Anthropic's Eugene Yan on LLMs Securing Source Code

Eugene Yan, a member of technical staff at Anthropic, recently shared insights into how large language models (LLMs) are transforming source code security. Speaking at the AI Engineer World's Fair, Yan outlined the evolving capabilities of AI in cybersecurity, highlighting the shift from simple vulnerability discovery to more complex tasks like verification, triage, and patching.

AI's Growing Role in Cybersecurity

Yan began by presenting data showing that AI model capabilities on cybersecurity tasks are doubling approximately every five months. This rapid advancement is evident in benchmarks that measure an AI model's ability to complete complex security tasks, mirroring human capabilities in areas like reverse engineering and web exploitation.

The impact of this increased capability is significant. Yan cited Mozilla Firefox's security bug fixes, noting a dramatic increase in vulnerabilities found and fixed in early 2026, with LLM-powered tools attributed to a substantial portion of this surge. He recalled major vulnerabilities like Log4Shell and Heartbleed to underscore the real-world impact of such security flaws.

The Bottleneck Shifts: From Finding to Fixing

Anthropic's own work scanning over a thousand open-source repositories revealed a key trend: finding vulnerabilities is becoming increasingly straightforward. The real challenge, Yan explained, lies in the subsequent steps of verification, triage, and patching. He shared the observation that the bottleneck has now shifted to these crucial, often human-intensive, processes.

To address this, Yan introduced the concept of 'agentic harnesses.' He quoted early experiments that showed promise but were hampered by high false positive rates, making them impractical for scaling. The introduction of agentic harnesses, however, has changed this dynamic, enabling models to reliably detect security issues and differentiate them from speculative findings. These harnesses, combining models with structured workflows, can significantly augment security teams.

A Six-Step Framework for AI-Powered Security

Yan distilled the lessons learned from working with numerous organizations into a six-step framework for building and utilizing AI for code security:

• Threat Model: This initial step involves defining what constitutes a vulnerability within a specific codebase. A well-documented threat model, Yan noted, can increase the true positive rate of AI findings to 90%. He emphasized that models excel at analyzing code they can read, but they lack the implicit context of system design, past fixes, or business logic that human engineers possess. This context, best captured in a threat model, is crucial for AI to act as a better security engineer.
• Sandbox: Isolation and reproducibility are paramount. Sandboxes, often run in microVMs with egress locked down, prevent models from performing malicious actions and ensure that vulnerability tests are reproducible.
• Discovery: This phase involves finding vulnerabilities. Yan highlighted three key elements: rich context (providing models with threat models, architecture docs, and past CVEs), simpler prompts (allowing models to infer vulnerabilities rather than being overly prescriptive), and providing useful tools (enabling models to actively interact with the system, like checking responses and reading logs).
• Verification: While discovery optimizes for recall, verification optimizes for precision. It's crucial to have an independent and adversarial verification agent that doesn't see the discovery agent's reasoning, aiming to confirm or refute findings.
• Triage: This step involves deduplicating findings, assessing severity based on impact and likelihood, and providing the threat model to the AI to refine its judgments. Yan stressed the importance of prioritizing critical and high-severity findings to avoid overwhelming developers.
• Patching: This final step includes building, reproducing, and regression testing patches. It also involves re-attacking the patched code and having a human in the loop to confirm the patch before merging.

Addressing Organizational Bottlenecks

Beyond the technical challenges, Yan pointed out significant organizational bottlenecks. He noted that while the harness scales with compute, human attention does not. Disagreements on severity calibration between product and security engineers, the implicit knowledge stored in people's heads, and the difficulty of fully automating patch review all present hurdles.

To overcome these, Yan advised that organizations should collaboratively define and document severity rules, and that the threat model needs to be explicitly written down. He also stressed the importance of moving towards AI-generated patches, verified by humans, to improve efficiency.

Getting Started

For those looking to implement these strategies, Yan recommended starting small with open-source dependencies. He advised an interactive approach to climbing the learning curve, emphasizing that understanding where models falter and what context is missing is crucial. Finally, he reiterated that the real bottleneck is not in scanning but in verification, triage, patching, and organizational processes.

Yan concluded by pointing to resources like Anthropic's Claude-based security tools and open-source GitHub repositories that can help developers get started with building their own agentic security harnesses.