The promise of AI coding agents is rapidly advancing, yet a critical blind spot persists: their susceptibility to sophisticated, multi-stage attacks that evade current safety protocols. While individual prompts may pass muster, the sequential execution of seemingly benign tasks can lead to exploitable code, a vulnerability that current safety alignment methods are ill-equipped to detect.
Emergent Exploits from Sequenced Innocuousness
Researchers Jonathan Steinberg and Oren Gal introduce MOSAIC-Bench, a novel benchmark designed to expose this structural weakness. The benchmark comprises 199 three-stage attack chains, paired with deterministic exploit oracles across diverse software substrates and common vulnerability classes (CWEs). This approach treats both exploit ground truth and reviewer protocols as critical evaluation axes. Astonishingly, nine leading production coding agents from major AI labs demonstrated end-to-end exploitability in 53-86% of scenarios, with minimal refusals. This starkly contrasts with direct-prompt evaluations where vulnerable-output rates drop significantly, highlighting how ticket staging effectively silences both refusal and hardening defense mechanisms.
Rethinking Code Review for Security
The findings underscore a fundamental gap in how AI-generated code is reviewed. Downstream code reviewer agents approved a significant portion (25.8%) of confirmed-vulnerable cumulative diffs, treating them as routine pull requests. Even full-context implementation protocols only partially closed the performance gap, indicating that context fragmentation is not the sole culprit. The study proposes a promising mitigation: reframing the reviewer as an adversarial pentester. Under this adversarial framing, evasion rates dropped substantially, and an open-weight Gemma-4-E4B-it reviewer achieved an 88.4% detection rate on real-world GitHub PRs, suggesting a path towards more robust MOSAIC-Bench coding agent security.
