4 Levels of AI Agent Maturity: Don't Build Slop

In a recent talk titled "Don't Build Slop (4 Levels of AI Agent Maturity)", Ara Khan outlined a framework for developing robust and effective AI agents. The presentation, delivered at AI Engineer Europe and sponsored by Google DeepMind, Braintrust, and WorkOS, emphasized a structured approach to agent development, moving beyond ad-hoc solutions towards more mature and maintainable systems.

Understanding AI Agent Maturity

Khan began by addressing a common pitfall in AI agent development: the tendency to create what he termed "slop." This arises from the factor of multi-agent orchestration, where the complexity of coordinating multiple agents can overwhelm even experienced engineers. Khan highlighted that many current AI agent flows from frontier models suffer from two main problems: inference bounds and take isolations.

Inference bounds refer to the situation where developers spend a significant amount of time waiting for an inference to complete, often in the background, which can be very time-consuming and reduce productivity. Take isolations occur when different agents are trained on the same data, leading to a "source code" conflict and eventual merge conflicts when integrating them.

The Four Levels of AI Agent Maturity

To combat these issues, Khan proposed a four-level model for AI agent maturity:

• Level 1: Use a Framework - At this foundational level, the focus is on leveraging existing AI agent frameworks like LangChain, LangGraph, CrewAI, AutoGen, or LlamaIndex. This allows developers to understand core concepts like how agents work at a surface level and their basic architecture.
• Level 2: Build It Yourself Agents - This stage involves building agents from scratch, focusing on architecture, modularity, and model independence. Developers learn to implement state machines, which are essentially recursive loops with conditions and end states, providing a mental model for agent behavior.
• Level 3: The Kanban: Visualize the Agent's Flow - This level emphasizes creating a clear, headline-level overview of each parallel agent's progress. It involves the ability to watch agents cleanly without context switching, and to establish dependency chains that link tasks autonomously. The concept of "diff review on click" is also introduced, allowing developers to see changes when an agent finishes a task.
• Level 4: Ship It to the Cloud: Parallelized, Production-Grade - The final level focuses on deploying agents as production-ready, cloud-native systems. Key characteristics include complete parallelizability (spinning up many instances), no local dependencies, programmatic orchestration via APIs, and horizontal scaling (more agents, not bigger agents).

Key Principles for Building Better AI Agents

Khan stressed that each new feature added to an agent risks making it worse by introducing new edge cases and complexities. He advocated for simplicity and a clear understanding of the agent's state machine, emphasizing that the goal is to create agents that are easy to build, test, and integrate into existing workflows.

He also highlighted a critical lesson learned: "Every single thing you add to an agent risks making it worse." This principle underscores the importance of thoughtful design and iterative development, rather than simply adding features for the sake of it.

Furthermore, Khan warned against the tendency of "frontier labs" to "lock you down" by making their proprietary models and APIs difficult to integrate with or customize. He advised developers to prioritize agents that are easily testable and allow for iterative improvements, ensuring that their creations are not only functional but also maintainable and scalable.

The Future of AI Agents

The presentation concluded with a look towards the future, suggesting that as AI models become more capable, the ability to orchestrate them effectively will become paramount. Khan's framework provides a roadmap for developers to move beyond basic agent functionality and build sophisticated, production-ready AI systems that can tackle complex tasks efficiently.