Trigger.dev's Eric Allam on Durable AI Agents

Eric Allam, co-founder of Trigger.dev, recently discussed the evolution of AI agents and the critical need for durability in their execution. In his presentation, Allam highlighted the shift from agents simply using existing backend infrastructure to becoming a fundamental part of that infrastructure themselves. This evolution necessitates new approaches to ensure agents can perform long-running, meaningful work reliably.

The Evolution of Web Backends and the Rise of Agents

Allam traced the history of web backends, starting with CGI in the early 1990s, which operated on a simple, stateless model where each request spawned a new process. This evolved into the LAMP stack and later serverless architectures, all largely adhering to a "shared nothing" or stateless compute model. However, as applications became more complex, they began incorporating "side effects" like sending emails or processing payments, which required more sophisticated handling of state and execution flow.

The emergence of AI agents, particularly with the advent of large language models (LLMs) in 2023, has further accelerated this trend. Allam explained that agents are fundamentally changing the dynamic, with the LLM now orchestrating the code, rather than the other way around. This shift demands a new approach to building "durable agents" that can maintain their state and recover from failures.

Two Roads to Durability: Replay vs. Snapshot

Allam outlined two primary methods for achieving agent durability: replay and snapshotting. The replay model, common in traditional software, involves logging every step of an execution. While this provides an audit trail and allows for retries, it can become cumbersome and inefficient for long-running, complex agent tasks, leading to issues with log size and versioning.

The alternative, snapshotting, involves capturing the entire state of a machine or process at a given moment and storing it. This approach, while conceptually simpler, has historically faced challenges with efficiency and size. Allam noted that early methods like CRIU (Checkpoint/Restore In Userspace) in 2011 offered a way to checkpoint and restore processes, but struggled with compatibility and performance for certain applications like media processing or headless browsers. He also pointed out the significant size of memory snapshots (512MB per snapshot) and the associated storage and network costs.

Firecracker MicroVMs and the Future of Stateful Compute

To address these limitations, Trigger.dev has been leveraging Firecracker microVMs. This technology allows for the snapshotting and restoration of the entire machine's state, providing a more robust and efficient solution. Allam highlighted that by using techniques like seekable zstd compression, lazy restore, and layered VMs, they have managed to reduce snapshot sizes significantly (from 512MB to 14MB compressed). This approach allows for "stateful compute," where the entire execution environment can be suspended and resumed with minimal overhead.

The performance metrics shared by Allam were impressive, with Firecracker snapshots taking under a second and restores just a few hundred milliseconds. This efficiency is crucial for enabling agents to perform complex, long-running tasks that could span days or weeks. Furthermore, the development of `fcrun`, a Docker-like CLI for managing Firecracker VMs, aims to simplify the deployment and management of these durable agents.

The Two Halves of a Durable Agent

Allam concluded by emphasizing that a durable agent has two key components: the context and the execution. The context is the append-only log of all interactions, including messages, tool calls, and LLM turns, which can be stored reliably in various storage systems. The execution state, encompassing files, memory, and processes, can be managed through snapshots. By combining these two aspects, developers can build AI agents that are not only powerful but also resilient and capable of handling complex, long-duration tasks, marking a significant step towards a more stateful and robust future for AI applications.