Agentic LLMs Break Context Limits

The finite context window of Large Language Models presents a critical bottleneck for long-horizon agentic tasks. As interaction trajectories extend, they can exceed the maximum context length, hindering task completion. Context compaction offers a solution by summarizing past states, but its integration with reinforcement learning has been largely unexplored. This paper introduces CompactionRL, a novel reinforcement learning strategy designed to train long-horizon agentic LLMs with context compaction, as detailed on arXiv.

Visual TL;DR. LLM Context Limits problem CompactionRL Introduced. Context Compaction solution CompactionRL Introduced. CompactionRL Introduced uses Joint Optimization. Joint Optimization via Token Loss Normalization. Joint Optimization via Cross-Trajectory GAE. Joint Optimization enables Breaks Context Limits. Breaks Context Limits leads to Boosts Coding Tasks.

LLM Context Limits: finite context window hinders long-horizon agentic tasks
Context Compaction: summarizing past states to overcome context limitations
CompactionRL Introduced: novel RL strategy for context compaction in LLM agents
Joint Optimization: optimizing task execution and summary generation simultaneously
Token Loss Normalization: enabling effective learning from compacted, extended trajectories
Cross-Trajectory GAE: generalized advantage estimation across multiple interaction trajectories
Breaks Context Limits: enables LLM agents to handle much longer interaction histories
Boosts Coding Tasks: demonstrates improved performance on complex coding challenges

Visual TL;DRQuickExplainDeeper

!-- /sh-diagram -->

Joint Optimization for Long-Horizon Agents

CompactionRL tackles the context limitation by jointly optimizing task execution and summary generation. This is achieved through token-level loss normalization and cross-trajectory generalized advantage estimation, enabling LLM agents to learn effectively from compacted, extended trajectories. This approach represents a significant step in overcoming the inherent constraints of current LLM architectures for complex, multi-step reasoning.

Empirical Validation on Coding Tasks

The effectiveness of CompactionRL is demonstrated through consistent performance gains on agentic coding tasks. When applied to the open GLM-4.5-Air model (106B-A30B), CompactionRL achieved absolute improvements of 7.0 points on SWE-bench Verified (reaching 66.8% Pass@1) and 3.1 points on Terminal-Bench 2.0 (reaching 24.5% Pass@1). Further enhancements were observed with the GLM-4.7-Flash model (30B-A3B), boosting Pass@1 scores by 5.5 and 6.8 points to 56.0% and 20.2% respectively on the same benchmarks. These results underscore the practical benefits of the proposed strategy.

Scaling to State-of-the-Art Open Models

The strategic implications of CompactionRL are substantial, as evidenced by its deployment in the RL pipeline for training the open GLM-5.2 model (750B-A40B). This demonstrates the scalability and efficacy of the CompactionRL framework in pushing the boundaries of what open-source LLMs can achieve in complex, long-context scenarios, promising more capable and accessible agentic AI systems.