"Do I want to make 3% compute efficiency wins, or change behavior by 40%?" This succinct question, posed by OpenAI’s Josh McGrath, encapsulates the strategic pivot underway across leading AI labs: the high-leverage work has shifted from foundational pre-training optimization to the complex, behavioral engineering of post-training models. McGrath, a Member of Technical Staff who transitioned from data curation to post-training research, articulated this evolution during a discussion with Swyx of Latent Space at NeurIPS 2025. The conversation provided a rare glimpse into the operational realities of developing systems like GPT-5 and the newly released shopping model, highlighting how the bottlenecks in achieving superior model capabilities have become less about algorithmic novelty and more about rigorous engineering and data quality.
The industry’s focus has moved beyond the optimization debates that defined earlier reinforcement learning eras. McGrath noted that whether researchers use PPO or DPO, both are fundamentally policy gradient methods; the real difference lies not in the math, but in the quality of the input data and the resultant trust signal. DeepSeek Math’s use of GRPO was underappreciated, not for its optimization trickery, but for shifting the focus toward verifiable reward signals. This move away from subjective human preference toward objective correctness defines the new standard for signal trust in reinforcement learning.
Scaling this post-training infrastructure presents unique, severe engineering challenges that eclipse those faced during massive pre-training runs. Pre-training involves moving tokens across many machines to calculate gradients, a relatively streamlined process. Conversely, post-training reinforcement learning requires orchestrating complex tasks, diverse grading setups, and often integrating external partners, resulting in a system with "way more moving parts than pre-training." This complexity means researchers often find themselves babysitting runs late at night, needing to quickly gain context and debug code they don't necessarily own or understand intimately.
This engineering intensity is mirrored in the metrics that truly drive performance improvements. While wall-clock speed remains a factor, the shift from GPT-5 to 5.1 was defined by a revolution in token efficiency. The update boosted evaluation metrics while simultaneously "slashing tokens" required to complete tasks. McGrath emphasizes that thinking in terms of tokens—not just time—is essential, as it directly impacts the complexity and viability of agentic workflows by determining how many tool calls or internal thought steps a model can reasonably execute.
The development of advanced agents, such as the new shopping model launched around Black Friday, has dramatically altered the internal workflow of researchers. McGrath described how coding assistants like Codex have compressed design sessions, turning 40-minute planning periods into 15-minute agent sprints. Yet, this speed introduces a new form of cognitive overhead: the strange, "trapped" feeling of waiting for the agent to execute the complex sequence of steps the human has orchestrated. The shopping model itself served as a testbed for key interaction paradigms, particularly interruptibility and chain-of-thought transparency, where the model shows its internal reasoning to the user, allowing for course correction.
User preference around model personality also proves to be a surprisingly durable differentiator. The split between users who prefer a warm, helpful persona (the "Clippy" archetype) versus those who want a purely functional, detached tool (the "Anton" archetype) is real, driving the need for customizable interfaces. McGrath personally uses custom instructions to enforce the Anton style, viewing the model strictly as a tool for work. The challenge for developers is managing the resulting complexity, particularly where explicit top-level routers (like choosing a 'thinking' versus 'non-thinking' model) interact with implicit routing mechanisms, creating "weird bumps" in performance that must eventually be merged into cohesive, seamless abstractions.
Looking ahead, the discussion touched on the future of context and model autonomy. While the pursuit of ever-longer context windows—the "dream of 10M+ token windows"—continues, McGrath suggested that pushing agent capabilities, particularly through efficient "graph walks," might unlock more utility than simply expanding raw context length. The ability for an agent to perform complicated transformations across a vast context window, rather than just locating a single piece of information (the "needle in a haystack" problem), is the current focus of long-horizon research.
Ultimately, the most critical resource for pushing the AI frontier is not compute or data, but human capital capable of bridging disciplines. The education system is currently failing to produce enough people skilled in "both distributed systems and ML research," creating a persistent talent bottleneck in frontier labs. This hybrid skillset is vital because, as McGrath concludes, the technological landscape remains a "fog of war," where the core problem shifts every few weeks, demanding deep technical competence across both engineering and research domains.
