AI Agents as Supply Chain Actors: Patch Pilot's Security Model

The task of keeping software dependencies up-to-date, especially with security vulnerabilities, is a constant and often unglamorous challenge for development teams. While tools like Dependabot and Renovate automate much of this process, their limitations become apparent in complex, containerized environments. Moritz Johner from Form3 presented a case study at AI Engineer World's Fair on 'Patch Pilot,' a system designed to tackle these challenges by leveraging AI agents, while also addressing the critical security implications of granting such agents production code access.

The Limitations of Existing Tools

Johner highlighted that traditional tools primarily focus on manifest files, which is insufficient when vulnerabilities reside in OS packages within base images or in binaries pulled during the build process. These tools simply cannot 'see' or act upon these hidden vulnerabilities. Furthermore, dependency updates are rarely isolated events; updating one component can necessitate updates in others, leading to a cascade of changes and potential code invalidation. Johner emphasized that this isn't just a patching problem, but a reasoning problem, requiring a tool that understands the full context of a vulnerability.

Patch Pilot: A Two-Tiered Approach

To address these issues, Form3 developed Patch Pilot, a system with a dual architecture. The first layer is a deterministic Go application responsible for orchestration, discovering vulnerable artifacts (OCI images), scanning them, and linking images to their respective repositories. This part is intentionally kept simple and predictable. The second layer involves spawning AI agents, which are tasked with the complex reasoning required for remediation. When a CI failure occurs, an agent is invoked to diagnose the issue and determine the necessary actions, whether it's a simple retry or a more complex fix.

The Supply Chain Incident Question

A key concern raised by Form3's Infosec team was the potential for such automation to become a 'supply chain incident waiting to happen.' Johner's thesis posits that any useful coding agent, by virtue of its ability to modify production code, is a supply chain actor. Therefore, the same rigorous guardrails applied to human engineers must also be applied to these agents. This led to a crucial architectural decision: separating capabilities and credentials between the deterministic and agentic layers.

Separating Deterministic and Agentic Roles

Patch Pilot's Go application handles the deterministic tasks, such as cloning repositories, creating context for the agent, and importantly, committing, pushing, and creating pull requests. It also monitors CI status. The AI agent, on the other hand, is confined to modifying files on the filesystem, performing the reasoning and remediation steps. Crucially, the agent is not given direct production credentials for actions like pushing to GitHub or triggering CI builds. This separation limits the 'blast radius' in case of prompt injection or other agent misbehavior.

Mitigating Prompt Injection Risks

Acknowledging that prompt injection is an unsolved problem, Form3 implemented mitigation strategies. These include 'prompt steering,' where the agent is instructed to ignore or distrust certain directories or files containing untrusted information. Additionally, they developed end-to-end tests to specifically probe for and prevent prompt injection vulnerabilities. While these measures reduce risk, Johner admitted that some level of uncertainty remains, emphasizing the need for a layered security approach.

The Challenge of Agent Sandboxing

A significant challenge lies in securely sandboxing agents, especially when they need to interact with tools like Docker. Granting a Docker socket to an agent effectively gives it root access to the host, posing a critical security risk. Johner proposed a solution using micro-VMs, specifically Firecracker, to create a more robust isolation boundary. The agent and its Docker socket run within this micro-VM, with its own kernel, preventing escape to the host system. Network policies can also be applied granularly to both the deterministic and agentic components.

The Evolving Landscape of Agent Sandboxing

Johner noted that while existing agents often come with their own sandboxes, these are frequently insufficient, particularly when Docker access is involved. For agents without built-in sandboxing, like many open-source solutions, custom environments must be created. He highlighted the emergence of projects like 'micro-sandbox' as promising, offering comprehensive features, but acknowledged that many solutions are still in beta and require further development for enterprise-grade adoption. The key takeaway is that the architecture itself is the primary determinant of an agent's blast radius, and careful separation of deterministic and agentic functions is paramount.

Johner concluded by inviting the community to share their own experiences and strategies for running agents in production, emphasizing the collaborative effort needed to navigate this evolving field.