Cursor's Auto-review Tames AI Agent Autonomy

AI agents need autonomy to be productive, but this freedom introduces security risks, especially with local agents accessing sensitive systems. Cursor's new Auto-review aims to manage this by acting as a nuanced 'dial' rather than an abrupt 'switch'.

The core concept is to allow agents freedom for low-stakes actions while slowing down when potential risks increase. This is achieved through a specialized classifier agent that reviews actions in context before they run.

Judging Risk in Context

The risk of an agent's action is not absolute but depends heavily on the situation. A command harmless in one workflow could be unacceptable in another. The key is the relationship between the action, the user's intent, and the potential consequence of error.

This realization led to the development of a classifier agent designed to govern overall autonomy. The goal was a small, fast, and inexpensive model capable of nuanced judgments about an action's alignment with user intent.

The classifier operates with a simple rule: be more lenient when security stakes are low and more cautious when they are high. This forms the basis for a fast, contextual reviewer integrated directly into the agent's execution path.

Building the Classifier

The initial technical challenge involved selecting the right model. Since the classifier runs before tool execution, speed and accuracy were paramount. Cursor leveraged its multi-model capabilities to test various options.

An early surprise was that less sophisticated models weren't always faster. When models struggled with policy or tool calls, they consumed more resources for poorer outcomes. A small, capable model proved to be the optimal trade-off.

The classifier was designed to be agentic, allowing it to inspect the workspace using tools like ReadFile or Grep when a command alone isn't sufficient. For example, the safety of running a Python script depends on its contents.

Avoiding a separate classification endpoint was crucial to minimize latency. The classifier runs within the same RPC stream as the parent agent, akin to a subagent architecture.

Designing the Feedback Loop

The system's response to blocking an action was the next critical design choice. The aim was to avoid generating more user prompts. Instead, when blocked, the classifier returns an explanation to the parent agent.

The parent agent can then often use this feedback to select a safer alternative without interrupting the user. This works because the feedback is grounded in user intent, not just isolated risk assessment.

This approach allows normal development to proceed while ensuring higher-consequence actions still require explicit user confirmation. The classifier's effectiveness hinges on precise tuning, necessitating evaluation sets that cover both permitted and blocked actions.

Testing the Classifier

Initial evaluations used internal usage data to understand typical agent workflows. The classifier needed to catch risky actions without impeding routine development.

Approximately 12 hours of internal developer sessions were analyzed, with common actions deduplicated into 6,122 labeled rows. Synthetic data was also generated to cover rare but critical failure cases, such as reading secrets or interacting with production data.

Policy changes during development complicated data labeling, requiring relabeling or regeneration of evaluation sets to match the evolving understanding of the problem.

Evaluations were run through the full backend classifier loop, testing tool use, classification, overrides, and parse failures. This included checking the final allow/block decision and the contextual data used for workspace inspection.

The team also monitored 'flapping', cases where the same input yielded inconsistent results, to identify and resolve policy or prompt underspecification. This ensured stable, predictable behavior.

Minimizing Outright Blocks

In practice, only a fraction of agent actions require review. Many are already covered by allowlists or sandboxing. The classifier primarily engages when contextual judgment is needed.

Currently, the classifier blocks about 4% of actions. However, a block doesn't automatically trigger a user prompt. The parent agent often resolves the issue internally.

Globally, only about 7% of chats in Auto-review mode result in at least one user interruption. This contrasts with previous scenarios where some enterprise customers saw roughly 40% of actions blocked.

This early data validates the core design: minimal user interruption and robust internal resolution of blocked actions. The system allows agents to continue working safely by finding narrower paths.

Refining Agent Autonomy

Auto-review is an evolving system, with the understanding of agent autonomy continually refined as agents become more capable. Currently focused on local agents in the desktop app, the principles are expected to extend to other areas.

The goal is genuine agent autonomy, with decisions to slow down based on context, not rigid global settings. The classifier enhances safety without reverting to a constant stream of approval prompts.

It identifies actions needing scrutiny, provides feedback to the parent agent, and enables continued work when safer alternatives exist. Contextual AI action review, as implemented here, represents a significant step forward, differing from simpler approaches to contextual AI action review.

Auto-review is now the default for new users, and existing users can enable it in Settings > Agents.