Cursor's Lee Robinson on Recursive Model Improvement

Lee Robinson, Machine Learning Engineer for Model Behavior at Cursor, took the stage at AI Engineer's World's Fair to discuss the intricate process of training AI models, with a particular focus on recursive model improvement. Robinson detailed Cursor's approach to building state-of-the-art models, emphasizing the iterative loop of gathering feedback, refining data, and scaling training processes.

The Model Training Loop

Robinson outlined a core training loop that begins with deploying a model into the world, followed by collecting user feedback. This feedback, along with online metrics and A/B testing, informs data scaling and improvement for subsequent training rounds. The process also involves increasing compute power to scale up overall training, ultimately leading to a new, improved model. This iterative cycle, while effective, is described as a serial process that can be time-consuming.

To accelerate this, Cursor employs a two-loop system: an outer loop for feedback and online metrics, and an inner loop focused on "climbing evaluations" and tackling more difficult training tasks. The goal is to generate higher-quality evaluations and create more challenging problems for models to solve, thereby shaping desired rewards during training.

Cursor's Progress and Future Ambitions

Cursor has been training models at scale for about a year, with its Composer 2.5 model, released in May, now being its most popular. Robinson highlighted that while Cursor initially focused on specialized models for tasks like tab completion, the past year has seen significant team expansion and a drive to develop state-of-the-art models. Composer is noted for its balance of speed, intelligence, and cost-effectiveness, filling a crucial niche in the market.

Looking ahead, Cursor aims to develop even larger and smarter models, gain full control over the training process by pre-training from scratch rather than relying on open-source bases, infuse new data to broaden model capabilities beyond coding, and scale up all aspects of training, particularly reinforcing the use of Reinforcement Learning (RL).

Enhancing the Inner and Outer Loops

Robinson elaborated on improving the outer loop by utilizing agent usage data, which constitutes the majority of Cursor's revenue. Feedback, both external (thumbs up/down) and internal (from heavy dogfooding), is classified to identify and address areas needing improvement.

The inner loop's acceleration is crucial. This involves creating high-quality, difficult evaluation tasks that accurately measure progress. Examples include models understanding user intent across numerous files, discerning when to ask for clarification versus trusting user input, and performing complex software engineering tasks like analyzing logs to diagnose issues. Robinson also touched upon the challenge of "reward hacking," where models find clever ways to exploit evaluations, such as by looking at Git history or external solutions.

To counter this, Cursor employs methods like deleting Git history during evaluations and implementing network allow-lists for agents. They have also developed private evaluation sets, like Cursor Bench, which are based on real-world tasks within their codebase and are held out from training data to ensure accurate performance measurement.

Innovative Training Techniques and Compute Scaling

The presentation also introduced novel learning methods, such as teaching models to "coach themselves." This involves providing specific textual feedback or hints within a model's rollout to guide its decision-making process, thereby improving its adherence to tool calling or stylistic consistency.

Scaling compute is central to these advancements. Cursor's partnership with SpaceX provides access to significant compute resources, enabling them to train large models from scratch. Robinson specifically mentioned SpaceX's Colossus supercomputer, which houses 200,000 GPUs, and Terafab for chip development, underscoring the full-stack approach to AI development.

The Future of AI Training: Recursive Self-Improvement

Robinson concluded by discussing the concept of the model training the next model, leading to recursive self-improvement. As models become smarter, they can generate or distill derivative models that accelerate training processes. This iterative improvement raises the baseline intelligence, enabling faster progress and the creation of more useful AI models. The ultimate goal is to automate more of the machine learning and research processes, freeing up researchers to focus on ambitious ideas and drive further innovation.