Cloudflare's Sunil Pai & Matt Carrie on Eval++ Compute Primitive

In a recent discussion at AI Engineer Europe, Cloudflare's Sunil Pai and Matt Carrie introduced and elaborated on Eval++, a new compute primitive they believe will be foundational for building and deploying AI agents. The platform is designed to offer durability, reproducibility, and scalability, addressing key challenges in the rapidly evolving field of AI development.

Meet the Innovators: Sunil Pai and Matt Carrie

Sunil Pai and Matt Carrie are key figures at Cloudflare, working on the forefront of their company's efforts in AI and edge computing. Their work focuses on abstracting complex infrastructure to enable developers to build sophisticated applications more easily. Their presentation on Eval++ highlights Cloudflare's commitment to providing the tools necessary for the next generation of AI-powered services.

Eval++: The Next Great Compute Primitive

Pai and Carrie positioned Eval++ as a significant advancement in how AI agents can be executed and managed. The core idea behind Eval++ is to provide a robust and scalable environment for AI agents, enabling them to perform tasks reliably and maintain state across multiple interactions. This is crucial for agents that need to remember context, interact with external services, and operate continuously.

The presenters emphasized the importance of stateful compute, a concept that allows agents to retain information and context over time, which is essential for building intelligent systems that can engage in meaningful, long-running interactions. Eval++ aims to abstract this complexity, providing developers with a simpler way to build these stateful agents.

Key Features and Benefits

The Eval++ platform offers several compelling features:

• Durability and Reproducibility: Agents built on Eval++ are designed to be durable, meaning their state is preserved even through system failures or restarts. This ensures that agents can pick up where they left off, maintaining continuity. Reproducibility is also a key focus, allowing developers to ensure that agents behave consistently under the same conditions.
• Zero Latency for Agents: By leveraging Cloudflare's global network of Workers, Eval++ aims to provide near-zero latency for AI agent interactions. This is achieved by running agents close to the user, reducing the time it takes for requests to be processed and responses to be delivered.
• Scalability: The platform is built to scale automatically, handling a massive number of concurrent agent executions without manual intervention. This elasticity is vital for applications that experience unpredictable traffic patterns.
• Cost-Effectiveness: The pay-as-you-go model of Cloudflare Workers contributes to the cost-effectiveness of Eval++, as users only pay for the compute resources they consume.

Pai and Carrie also touched upon the security aspects, noting that Cloudflare's existing security architecture provides a strong foundation for securing AI agents, including features like Web Application Firewalls (WAF) and DDoS mitigation, which are automatically applied to Worker deployments.

The Vision for AI Agents

The presenters shared their vision for the future of AI agents, emphasizing that they should be able to perform tasks and interact with the world in a more seamless and intelligent manner. They highlighted the potential for AI agents to become a new compute primitive, akin to how Docker containers transformed application deployment. By providing a reliable and performant platform, Cloudflare aims to empower developers to build a wide range of AI-powered applications, from simple chatbots to complex autonomous systems.

The discussion also touched upon the flexibility of the platform, allowing agents to be written in various languages that compile to WebAssembly, such as Rust and C++. While JavaScript is the primary language for Workers, the ability to use other languages broadens the appeal and capability of the Eval++ platform.