Adaptive Memory for Smarter LLM Agents

The efficacy of Large Language Model (LLM) agents hinges critically on their ability to learn from experience. However, current LLM agents memory systems are hampered by rigid, pre-defined methods for interacting with external memory. This static approach fails to account for the dynamic nature of optimal memory behavior, which is inherently context-dependent.

Memory Management as a Learned Policy

The researchers introduce Memory as a Controlled Process (MemCon), a novel framework that reframes memory operations as a Markov Decision Process. This allows for the learning of an online policy that adaptively dictates retrieval timing, content, and volume, alongside strategic decisions on injecting distilled plans and consolidating or forgetting information. MemCon is designed to be backend-agnostic, meaning it can enhance any existing memory implementation.

Context-Aware Optimization Drives Performance

MemCon's adaptive strategy offers significant advantages across different task phases. Early task stages benefit from judiciously limited retrieval when memory is sparse. Recurring goal types are better served by reusing learned plans rather than performing generic nearest-neighbor lookups. Agents that encounter difficulties can re-retrieve information using alternative queries, and over extended task streams, the memory store itself is consolidated and pruned to maintain utility. This dynamic approach yields substantial gains: across six benchmarks, three agent frameworks, and three LLM backbones, MemCon consistently outperformed baseline memory strategies, achieving up to 15.2 points higher task success rates and reducing token consumption by 5% to 20%. Crucially, MemCon learns from simple task-by-task binary feedback, requiring no pretraining and no additional LLM calls, utilizing a lightweight tabular contextual bandit with UCB exploration that converges rapidly.