LifeSkill: LLM Agents Learn Continuously

The imperative for Large Language Model (LLM) agents to adapt and learn continuously in dynamic, interactive environments is clear. However, current lifelong learning paradigms for long-horizon tasks falter by relying on discrete skill retrieval with static parameters during inference. This fundamentally limits their ability to internalize real-time feedback, a capability crucial for human-like learning. Addressing this critical gap, a new framework dubbed LifeSkill emerges from arXiv, presenting a novel two-stage reinforcement learning approach for online lifelong learning agents.

Bridging the Supervision Gap in Skill Extraction

LifeSkill introduces Verifier-Guided Skill Learning, a mechanism designed to overcome the absence of direct supervision for skill extraction. Instead of relying on mere plausibility, candidate skills are rewarded based on their demonstrated utility across multiple skill-conditioned policy rollouts, as evaluated by a verifier. This incentivizes the generation of skills that are genuinely effective for task completion, rather than just linguistically coherent.

Internalizing Adaptation: Beyond Context Bloat

The framework further innovates with Online Skill Internalization, enabling agents to continuously refine their policy models during test-time interactions. By transforming skill-conditioned trajectories into actionable reward signals, LifeSkill allows agents to directly incorporate reasoning capabilities into their core parameters. This circumvents the performance degradation and computational overhead associated with traditional experience retrieval methods, leading to more efficient and dynamic lifelong learning LLM agents.