Context Layer: The Missing Piece for Production AI Agents

In the rapidly evolving AI landscape, a critical disconnect persists: while models have achieved remarkable leaps in raw intelligence, their ability to solve real-world business problems remains stubbornly constrained. Prukalpa Sankar, Founder & Co-CEO of Atlan, argues that the missing ingredient is context. Speaking at a recent event, Sankar outlined the architectural needs for giving AI agents the contextual intelligence required to move beyond impressive demos and achieve production viability.

The current state of AI agents presents a common frustration for businesses. Two years ago, AI models struggled with basic professional exams. Today, they rank in the top 1% of test scorers. Yet, most deployed agents still falter when faced with simple business questions. This leads to a cycle where a functional demo is shipped, only for the business to abandon it within a month. Sankar identifies this failure point not as a limitation of model intelligence, but as a deficit in understanding the nuances of business operations.

The Contextual Gap in AI Deployment

Sankar emphasizes that the value human experts bring to a business is not solely their analytical prowess, but their deep understanding of business definitions, procedural knowledge, and operational norms. These are the implicit, often unarticulated, rules and understandings that govern how work gets done. AI models, trained on vast datasets, often lack this specific, operational context. This leads to agents that can process information but cannot effectively act upon it within a specific business environment.

The problem is compounded by the fact that traditional methods like fine-tuning and simple prompting struggle to imbue AI agents with this deep contextual understanding. Fine-tuning can be costly and difficult to scale, while prompting offers only a surface-level interaction with the model's knowledge. Sankar proposes a more fundamental architectural shift: a dedicated 'context layer' designed to provide AI agents with the necessary business intelligence.

Architecture of a Context Layer

Drawing from hundreds of production deployments, Sankar detailed the components and principles of an effective context layer. This layer acts as a bridge between raw data and AI agent execution, translating business logic and operational knowledge into a format AI can utilize. Key elements include:

• Versioned, Testable, Portable Context Repositories: Just like code repositories, context needs to be managed, versioned, and tested. This ensures that the context applied to an AI agent is reliable, auditable, and can be easily shared and updated across different deployments and frameworks.
• Simulation Environments: Before deploying an agent into a live business environment, simulation plays a critical role. These environments allow for the testing of agent behavior against real-world scenarios, catching potential failures and unexpected outcomes related to context before they impact operations.
• Agent Traces for Contextual Compounding: The interactions and outcomes of AI agents in production can provide valuable feedback. By tracing agent behavior and feeding these insights back into the context layer, the system can learn and adapt, compounding shared context over time and improving future agent performance.

This approach, Sankar argues, is essential for scaling AI solutions beyond the demo phase. Context engineering, as opposed to solely relying on model training or prompt engineering, provides a more sustainable and effective path to production-ready AI agents.

Openness and Portability: The MCP Mandate

A crucial aspect of Sankar's vision for the context layer is its openness and portability. The context should not be locked into a specific AI framework or proprietary system. Instead, it should adhere to open standards, such as the Metadata, Compute, and Portability (MCP) framework, and integrate with data infrastructure like Apache Iceberg. This allows for context to be deployed across any framework and any cloud environment, preventing vendor lock-in and maximizing flexibility.

When context is not open and portable, it leads to fragmented systems and significant re-engineering efforts for each new AI initiative. Sankar highlighted the consequences of non-open context: increased costs, slower deployment cycles, and a reduced ability to adapt to changing business needs. The goal is to create a reusable and universally applicable layer of business intelligence that powers all AI agents within an organization.

Atlan's Role in Context Engineering

Prukalpa Sankar, as the Founder & Co-CEO of Atlan, is at the forefront of this movement. Atlan positions itself as the context layer for AI, aiming to provide enterprises with the necessary infrastructure to imbue their AI agents with operational intelligence. The company's work in metadata management and data governance has positioned it as a leader, serving over 300 enterprises including major financial institutions and automotive giants. Atlan's success in attracting significant funding, including over $200 million from prominent venture capital firms, underscores the growing recognition of context's importance in the AI era.

Before Atlan, Sankar co-founded SocialCops, a venture that built the world's largest government data lake to support the UN's Sustainable Development Goals. This prior experience likely informed her understanding of how data, context, and practical application intersect to drive real-world impact.