Agent Architectures Have a 6-Month Half-Life

Dan Farrelly, CTO and co-founder of Ingest, presented a compelling argument at the AI Engineer World's Fair, stating that the average AI agent architecture has a 'half-life' of just six months. This rapid obsolescence is driven by the fast-evolving AI landscape, where new models, framework versions, tool-calling standards, and architectural patterns emerge constantly. Farrelly, who leads a team building systems for reliable AI execution, emphasized that simply adopting new technologies without a solid architectural foundation leads to constant rewrites and technical debt.

The Problem with Rapid Evolution

Farrelly posed a critical question to the audience: "What part of your architecture survives the next six months?" He urged developers to consider whether the code they shipped half a year ago still functions as originally intended, and whether its survival was by accident or by design. He noted that many teams adopt frameworks or pre-built solutions without a deep architectural consideration, leading to merged layers and tightly coupled components. This makes it difficult to swap out or update individual parts without a complete system overhaul. The core issue, he explained, is that the rapid pace of change in the context and compute layers drags down the entire architecture.

Decoupling into Three Layers

To combat this, Farrelly proposed a layered mental model for architecting AI agents. He defined three discrete layers:

• Execution Layer (The Brain): This layer handles flow, state management, durability, and retries. It is the system responsible for reliably running code, independent of the underlying infrastructure.
• Context Layer (The Knowledge): This encompasses models, prompts, tools, and memory. This layer changes most frequently.
• Compute Layer (The Hands): This includes sandboxes, runtimes, and browsers that the agent automates.

Farrelly stressed that the Execution layer is the most critical for achieving architectural longevity. It should manage the agent's lifecycle, including planning, model calls, code execution, sub-agent invocation, loops, retries, and coordination. The goal is to make this layer swappable with different context and compute layers without requiring a rewrite.

Key Requirements for the Execution Layer

Farrelly detailed the essential functionalities of a robust Execution layer:

• Resumability: Systems must be able to resume execution after failures without restarting from the beginning. This is crucial for long-running tasks where failures in LLM calls or tool invocations are common. State must be durable and external to the work itself.
• Flexible Invocation and Coordination: The execution layer must support various invocation patterns, including CRON jobs, event triggers, APIs, human-in-the-loop workflows, and sub-agent delegation. It should handle synchronous and asynchronous calls, with delayed invocations and flexible orchestration primitives.
• Observability: Comprehensive visibility across the entire session, from trigger to finalization, is essential. This includes tracking LLM calls, tool calls, database errors, permissions issues, and performance metrics. Without full session traces, debugging and iterative improvement become significantly more challenging.

The Role of Sandboxes

Farrelly clarified the role of sandboxes, stating, "A sandbox does work. It doesn't think." Sandboxes are designed to be ephemeral and stateless, making them unsuitable for durability or state management. Using a sandbox for state persistence is an anti-pattern. Instead, the Execution layer should provide the sandbox with its context, sequence, and durability.

Emerging Architectures and the Need for Execution

Looking ahead, Farrelly highlighted emerging trends like background agents, dynamic workflows, autonomous loops, and agent factories. These systems are characterized by long-running, asynchronous, and delegated operations, all of which require robust observability and inspectability. He emphasized that execution is fundamental to building and evolving these complex systems.

For instance, background agents that run for hours with hundreds of tool calls are prone to failure, making proper infrastructure with observability crucial for debugging. Similarly, loop architectures require reliable execution, sub-agent delegation, and inspectable history to function effectively. Farrelly noted that frameworks from even a few months ago were not built to handle these complexities, necessitating custom solutions and a strong execution layer.

Ingest as the Solution

Farrelly concluded by introducing Ingest as a platform built to address these challenges. Ingest provides durable execution for AI agents, acting as the execution layer that allows developers to plug in any context or compute layer. Its features include durable, re-tryable, and cached steps, scheduling capabilities, agent-to-agent coordination, full session traces, and no infrastructure management. Ingest also enables instrumentation and scoring of agents by leveraging the data flowing through the execution layer, allowing for outcome-based scoring and faster iteration.

The core message was clear: by getting the execution layer right with the proper primitives, developers can build agent architectures that are resilient to the rapid pace of change in the AI field, making the next six months, and beyond, significantly easier to navigate.