Be Scared: AI Code Security Risks with Steve Yegge

Steve Yegge, speaking at the AI Engineer World's Fair, delivered a stark warning to the audience: "Be scared." His presentation, titled "Agentic Security: Permissions, Provenance, and the agent supply chain," underscored the escalating security risks associated with the rapid adoption of AI in code generation. Yegge, representing Sneak, highlighted that while AI can accelerate development, it also introduces new and complex security challenges.

The Amplified Attack Surface

Yegge painted a concerning picture of the current AI coding landscape. He stated that if development teams ship code 10 times faster with the same defect rate, the overall vulnerability surface area expands by a factor of 10. This issue is exacerbated by the fact that AI models, when writing code, are likely to introduce not just existing vulnerabilities like cross-site scripting, but entirely new classes of threats. He cited the example of 'slop squatting,' where AI models can hallucinate package names, leading developers to inadvertently download malicious code disguised as legitimate dependencies.

The Urgency of Early Detection

Drawing parallels to his experience at Google, Yegge emphasized the importance of surfacing bugs, especially security vulnerabilities, as early as possible in the development lifecycle. He explained that bugs typically have a 'half-life of urgency', the longer they persist, the less likely they are to be fixed. However, security vulnerabilities do not decay; they compound over time. This necessitates a proactive approach, surfacing these issues at the developer's fingertips, or even to the AI models themselves.

New Threats and AI's Limitations

Yegge pointed out that AI models, while powerful, are not inherently secure. He shared his experience with Fable, an AI model that, despite its capabilities, introduced an XSS vulnerability. He stressed that AI models, much like humans, require multiple passes to refine their work. Security cannot be an afterthought or combined with other tasks like correctness. Yegge advocated for a two-pass approach, separating security checks from other development concerns.

Leveraging Tools for AI Security

To combat these emerging threats, Yegge proposed integrating specialized security tools into the AI workflow. He specifically mentioned Snyk and Chain Guard as examples of solutions that can provide AI models with enhanced capabilities. By incorporating these tools as a pass within the AI's prompt, developers can ensure a more thorough security analysis. Yegge suggested a multi-tool approach, having various security tools check each other's work for a more robust defense.

The Future of Agentic Security

Looking ahead, Yegge warned that open-source models are rapidly catching up to proprietary ones in their ability to perform sophisticated tasks, including hacking. He highlighted the concerning prediction that this shift could occur within months. This underscores the need for immediate attention to AI security, including securing the agent supply chain. Yegge also touched upon a broader societal concern: the rise of AI-powered scams, such as deepfake calls designed to defraud individuals, emphasizing the need for personal vigilance and established 'code words' within families.

A Call to Action

Yegge concluded with a call to action, urging the audience to "dial it in" and take these threats seriously. He reiterated that while the situation is daunting, proactive steps can be taken. Leveraging existing tools, adopting multi-pass security checks, and fostering an adversarial mindset are crucial for navigating the evolving threat landscape. His message was clear: the race for AI security is on, and preparedness is paramount.