OpenAI Unveils Agent RFT: Revolutionizing AI with Self-Improving Tool-Using Models

At the AI Engineer Code Summit, OpenAI's Will Hang and Cathy Zhou introduced Agent Reinforcement Fine Tuning (Agent RFT), a groundbreaking approach designed to dramatically enhance the performance of AI agents. Their presentation delved into how Agent RFT empowers models to interact more intelligently with external tools and environments, thereby pushing the boundaries of autonomous task completion. This innovation marks a significant leap for developers and enterprises aiming to deploy more capable and efficient AI systems.

Will Hang and Cathy Zhou, both members of OpenAI's fine-tuning team, spoke at the AI Engineer Code Summit about the latest advancements in fine-tuning code models. Their presentation focused on Agent RFT, detailing its architecture, benefits, and practical applications for improving AI agent performance. The core insight shared was how this method allows agents to learn and adapt more effectively within complex, real-world scenarios.

Agents, unlike traditional models, possess the unique ability to interact with the outside world by leveraging various tools to complete tasks autonomously. This process isn't merely a sequential execution of commands; it involves a sophisticated, interleaved cycle of reasoning and tool calls within the same context window. OpenAI's flagship coding agent, Codex, exemplifies this paradigm, utilizing tools like planning, terminal access, and `apply_patch` to perform complex coding tasks, from generating unit tests to submitting substantial code changes.

Improving agent performance typically begins with prompt optimization, where refined guidance and instructions steer the model's behavior. Further enhancements come from task optimization, simplifying objectives, adding guardrails to prevent improper tool use, or even modifying tool functionalities themselves. However, when these front-line techniques reach their limits, fine-tuning emerges as the ultimate method to squeeze more performance out of a task by altering the model's underlying weights.

Agent Reinforcement Fine Tuning represents the pinnacle of this fine-tuning hierarchy. It dynamically adjusts the model's weights based on a user-defined reward signal. During training, the agent actively explores numerous strategies for tool invocation to successfully navigate and solve complex tasks. A crucial development in Agent RFT is the unprecedented ability for models to call user-defined tools via public internet endpoints and, subsequently, to invoke custom reward signals through similar endpoints after each rollout. "These two additions actually mark the first time that we at OpenAI have allowed models to interact with the outside world during the training process," Hang explained, highlighting the profound implications of this capability.

The benefits of Agent RFT are compelling for sophisticated AI applications. It significantly improves the performance of reasoning models that must call tools and interact with external environments in a multi-step fashion. This method is also remarkably sample-efficient, often requiring only "10s to 100s of samples to get performance improvements," a testament to its practical utility. Ultimately, fine-tuned models exhibit lower latency and superior machine learning performance on agentic tasks.

One of the persistent challenges in deploying AI agents for specific business contexts is the "domain shift." This occurs when a user's operational environment and tools differ from the data on which the foundational models were initially trained, leading to suboptimal performance or incorrect tool usage. Agent RFT directly addresses this by readapting the model to the user's unique domain through its weight-changing training process. This results in an agent that genuinely comprehends and effectively operates within its designated environment, mitigating issues like redundant tool calls or erroneous inputs.

Beyond enhanced performance, Agent RFT also yields substantial improvements in operational efficiency. By imposing penalties for exceeding a defined tool-call budget during training, the models learn to accomplish tasks with significantly fewer steps, directly translating to lower latencies in real-world applications. This strategic optimization ensures that agents not only perform accurately but also do so with remarkable speed, a critical factor for production environments.

The path to successful Agent RFT implementation, as outlined by OpenAI, involves several critical steps. First, developers must construct a high-quality dataset with representative evaluations, ensuring that training and evaluation data accurately mirror production behavior. This meticulous preparation prevents unexpected performance degradation in live environments.

Second, it is essential to baseline the performance of frontier models against these datasets to establish a clear understanding of expected outcomes before applying RFT. Third, developers should exhaust traditional prompt and task optimization techniques. Only after these preliminary steps, when the desire for even greater performance persists, should one turn to Agent RFT.

OpenAI showcased several success stories. Cognition utilized Agent RFT for its Code Edit Planning Agent, achieving a 10-point improvement from GPT-5 on larger datasets and reducing tool calls from 8-10 to just 4 turns by enabling parallel tool execution. Qodo's Deep Research Agent for code review saw a 6 percentage point increase in facts recall and a 10-15% reduction in tool call steps, alongside a 50% decrease in output tokens. This also stabilized agent behavior, eliminating long-tail cases that caused latency spikes. Cosine, building coding agents for complex enterprise codebases with 30 diverse tools, achieved state-of-the-art performance on various benchmarks and significantly faster agents by converging to tighter, more efficient sequences of steps. Finally, Mako, a GPU kernel building agent, achieved a remarkable 72% improvement over existing frontier models in 3-shot runs, demonstrating RFT's power even with limited training examples.

To ensure success with Agent RFT, four key principles are paramount. Tasks must be well-specified and constrained, meaning there should be a clear, unambiguous definition of success without subjective grading criteria. Evaluation datasets must mirror production behavior to prevent domain shift. The model should demonstrate that its maximum performance improves with more tries, indicating its ability to learn from exploration. Lastly, reward functions must be unhackable and continuous, providing partial credit rather than binary feedback, which encourages the model to incrementally optimize towards optimal performance.