David Brumley on Teaching AI to Find Real Zero Day Vulnerabilities

Large language models can write code, pass medical exams, and generate prose, but can they reliably exploit complex software vulnerabilities? David Brumley, a security researcher with two decades of experience training human hackers, argues that teaching AI to discover bugs requires the exact same structured path used for humans: a progressive ladder of exploitation tasks. Speaking on the mechanics of automated security research, Brumley explained why current evaluation setups fail and how real reinforcement learning environments can prove whether a model truly knows how to hack.

Who Is David Brumley

David Brumley is a veteran cybersecurity researcher and academic who has spent twenty years building security education systems. He founded picoCTF, a widely used capture-the-flag learning platform, and spent years recruiting top competitive hackers at Carnegie Mellon University. Brumley also played a central role in designing the scoring mechanics for the DARPA Cyber Grand Challenge, establishing deterministic methods to evaluate automated vulnerability detection and patching systems.

The Failure of Traditional AI Security Benchmarks

Evaluating standard AI models on security tasks is notoriously difficult. Unlike multiple choice tests or code completion benchmarks, offensive security has no single correct answer. When an AI model attempts to exploit a system, standard evaluation setups often rely on grading oracles that simply ask the language model if it succeeded. These self-reported oracles fail completely because models frequently hallucinate success.

The problem deepens when a target system contains multiple vulnerabilities. If a model is instructed to find a specific bug but triggers an entirely different flaw in the codebase, naive benchmarks count it as a success. This lack of precision creates a false illusion of capability, masking whether the model genuinely understood the target flaw or merely stumbled into a crash.

A Structured Task Ladder for Exploitation

To overcome these measurement flaws, Brumley advocates for a progressive ladder of exploitation tasks. Instead of expecting a model to instantly produce a working zero-day exploit, training environments break down offensive security into discrete, verifiable steps:

• Triggering a crash: Forcing the target application to crash deterministically on a specific input.
• Reading and writing memory: Manipulating program execution to achieve arbitrary memory access.
• Full working exploit: Combining primitive capabilities to achieve complete code execution or escape an isolated execution environment.

This sequential approach allows reinforcement learning algorithms to receive clear, honest feedback at each step of the offensive pipeline.

Deterministic Graders and Precision Scoring

Borrowing principles from his work on DARPA competitions like AIXCC, Brumley's team constructed isolated reinforcement learning environments. Rather than trusting language model output, these environments use deterministic graders to execute the exploit in a safe sandbox. The system checks whether the generated payload specifically triggers the targeted vulnerability, applying strict precision and recall metrics to score performance.

Testing AI on Google Chrome V8 Engine

Brumley tested this deterministic evaluation framework on V8, the open source JavaScript engine maintained by Alphabet Inc. (NASDAQ:GOOGL). His team evaluated leading artificial intelligence models across 41 real-world historical vulnerabilities in the engine.

The results demonstrated that top-tier models, when evaluated inside proper reinforcement learning environments, reached roughly a 95% success rate across the dataset. In the most challenging scenarios, the models generated fully functioning out-of-sandbox exploits, including the discovery of a real, previously unknown zero-day vulnerability.

The Danger of Benchmaxxing Security

Brumley closed with a direct warning against optimizing AI security models for artificial benchmarks. Optimizing models to pass superficial tests produces systems that sound convincing but fail against real targets. AI security research must rely on grounded execution environments and honest, deterministic evaluation. Only rigorous sandboxed testing can separate a model that merely looks like it can hack from one capable of real world security research.