Agentic LLMs Break Context Limits

CompactionRL integrates context summarization into reinforcement learning for agentic LLMs, breaking context window limits and boosting performance on coding tasks.

Jul 7 at 8:00 PM4 min read

Diagram illustrating the CompactionRL process for agentic LLMs. — CompactionRL enables agentic LLMs to handle long-horizon tasks by effectively summarizing and compacting context.

Visual TL;DR

LLM Context LimitsDriver

finite context window hinders long-horizon agentic tasks

Context CompactionContext

summarizing past states to overcome context limitations

From the article 5 mentionsContext compaction offers a solution by summarizing past states, but its integration with reinforcement learning has been largely unexplored.

CompactionRL IntroducedCore

novel RL strategy for context compaction in LLM agents

From the article 6 mentionsThis paper introduces CompactionRL, a novel reinforcement learning strategy designed to train long-horizon agentic LLMs with context compaction, as detailed on arXiv.

Joint OptimizationCore

optimizing task execution and summary generation simultaneously

Token Loss NormalizationContext

enabling effective learning from compacted, extended trajectories

From the articleThis is achieved through token-level loss normalization and cross-trajectory generalized advantage estimation, enabling LLM agents to learn effectively from compacted, extended trajectories.

Cross-Trajectory GAEContext

generalized advantage estimation across multiple interaction trajectories

Breaks Context LimitsEffect

enables LLM agents to handle much longer interaction histories

Boosts Coding TasksOutcome

demonstrates improved performance on complex coding challenges

From the articleThe effectiveness of CompactionRL is demonstrated through consistent performance gains on agentic coding tasks.

The finite context window of Large Language Models presents a critical bottleneck for long-horizon agentic tasks. As interaction trajectories extend, they can exceed the maximum context length, hindering task completion. Context compaction offers a solution by summarizing past states, but its integration with reinforcement learning has been largely unexplored. This paper introduces CompactionRL, a novel reinforcement learning strategy designed to train long-horizon agentic LLMs with context compaction, as detailed on arXiv.

Joint Optimization for Long-Horizon Agents

CompactionRL tackles the context limitation by jointly optimizing task execution and summary generation. This is achieved through token-level loss normalization and cross-trajectory generalized advantage estimation, enabling LLM agents to learn effectively from compacted, extended trajectories. This approach represents a significant step in overcoming the inherent constraints of current LLM architectures for complex, multi-step reasoning.

Empirical Validation on Coding Tasks

The effectiveness of CompactionRL is demonstrated through consistent performance gains on agentic coding tasks. When applied to the open GLM-4.5-Air model (106B-A30B), CompactionRL achieved absolute improvements of 7.0 points on SWE-bench Verified (reaching 66.8% Pass@1) and 3.1 points on Terminal-Bench 2.0 (reaching 24.5% Pass@1). Further enhancements were observed with the GLM-4.7-Flash model (30B-A3B), boosting Pass@1 scores by 5.5 and 6.8 points to 56.0% and 20.2% respectively on the same benchmarks. These results underscore the practical benefits of the proposed strategy.

Scaling to State-of-the-Art Open Models

The strategic implications of CompactionRL are substantial, as evidenced by its deployment in the RL pipeline for training the open GLM-5.2 model (750B-A40B). This demonstrates the scalability and efficacy of the CompactionRL framework in pushing the boundaries of what open-source LLMs can achieve in complex, long-context scenarios, promising more capable and accessible agentic AI systems.

© 2026 StartupHub.ai. All rights reserved. Do not enter, scrape, copy, reproduce, or republish this article in whole or in part. Use as input to AI training, fine-tuning, retrieval-augmented generation, or any machine-learning system is prohibited without written license. Substantially-similar derivative works will be pursued to the fullest extent of applicable copyright, database, and computer-misuse laws. See our terms.

#AI Research #Large Language Models #Reinforcement Learning #Agentic AI #Context Compaction