Visual TL;DR
LLM Context LimitsDriver
finite context window hinders long-horizon agentic tasks
Context CompactionContext
summarizing past states to overcome context limitations
From the article 5 mentionsContext compaction offers a solution by summarizing past states, but its integration with reinforcement learning has been largely unexplored.
CompactionRL IntroducedCore
novel RL strategy for context compaction in LLM agents
From the article 6 mentionsThis paper introduces CompactionRL, a novel reinforcement learning strategy designed to train long-horizon agentic LLMs with context compaction, as detailed on arXiv.
Joint OptimizationCore
optimizing task execution and summary generation simultaneously
Token Loss NormalizationContext
enabling effective learning from compacted, extended trajectories
From the articleThis is achieved through token-level loss normalization and cross-trajectory generalized advantage estimation, enabling LLM agents to learn effectively from compacted, extended trajectories.
Cross-Trajectory GAEContext
generalized advantage estimation across multiple interaction trajectories
From the articleThis is achieved through token-level loss normalization and cross-trajectory generalized advantage estimation, enabling LLM agents to learn effectively from compacted, extended trajectories.
Breaks Context LimitsEffect
enables LLM agents to handle much longer interaction histories
Boosts Coding TasksOutcome
demonstrates improved performance on complex coding challenges
From the articleThe effectiveness of CompactionRL is demonstrated through consistent performance gains on agentic coding tasks.
