AI Agents Must Live Where Your Data Does

Enterprise AI agents often falter when they operate outside the secure, governed environment where an organization’s data resides. This architectural disconnect leads to compounding problems, including fragmented governance, escalating egress costs, and significant latency in multi-step operations. The core issue, as highlighted by Databricks, is the fundamental need for AI to operate within the data's existing control plane.

Companies working on this

StartupHub profiles of the companies this article names, with funding and a one-liner from our database.

Databricks

$190.0B

A unified data analytics and AI platform built on the lakehouse architecture.

poolside

$4K

AI coding models and pair programmer built for on-device execution with privacy-first architecture.

Sierra

$10K

Sierra helps businesses build better, more human customer experiences with AI agents that can understand, reason, and take action across various channels.

Inflection

$4.0B

Develops enterprise AI solutions and conversational AI models, formerly known for its personal AI assistant, Pi.

The common pilot project involves connecting a large language model (LLM) to data via a vector database. However, the real challenges emerge when scaling. Security teams flag governance gaps, user experience suffers from slow responses, and model provider bills surge. These issues typically stem from pulling data out of governed systems into a separate AI stack, which was never designed to enforce enterprise policies.

The High Cost of External Agents

Data possesses significant gravity. Moving compute is relatively easy, but relocating large datasets, especially with multiplying modalities, is not. Extracting data introduces a predictable set of penalties.

Governance weakens as access controls, lineage, and residency rules must be re-implemented for each integration, inevitably leaving gaps. Latency increases with every network hop to external vector stores and LLMs, compounding across multi-tool agents. Costs fragment across egress charges, duplicate storage, and per-token pricing from multiple vendors. Managing the lifecycle becomes a complex coordination task across disparate systems.

Observability suffers, requiring the stitching together of logs from multiple tools to trace a single request. Crucially, business context, like metric definitions or glossary terms, remains trapped within the governance layer, forcing external agents to guess or rebuild this essential context from column names.

Why Post-Hoc Governance Fails

Governance is the penalty that cannot be patched later. Most AI governance approaches attempt to filter data after an agent has already accessed it, such as redacting sensitive fields from responses or auditing logs afterward. This approach breaks down as soon as agents begin computing over data.

Consider an agent calculating a financial summary. If row-level security isn't enforced before the aggregation, the resulting sum or average is already shaped by data the user should not have influenced. No downstream redaction can undo this fundamental calculation. Policy enforcement must occur at query planning time, not at response rendering time.

Retroactive controls are incomplete because they assume data can be safely censored after reaching the agent. Once an aggregation or transformation occurs, the original governance intent is lost.

This breakdown also inflates costs. When governance is resolved after the fact, agents compensate by traversing audit logs, joining fragments from external systems, and re-evaluating data to determine its usability. This is not the agent’s intended task but a compensation for a poorly governed data answer.

Blocked or redacted outputs trigger retry loops, extending sessions and loading more context into the model. A single request can quietly balloon into thousands of billed tokens, a direct consequence of post-hoc governance spinning up the token-burning loop.

Data-native AI agents embed policy enforcement directly into query planning and computation, ensuring every intermediate result adheres to governance constraints. Policy is evaluated before and during execution, not as an afterthought.

Agent State and Memory Demand Governance

Beyond reading data, production agents write conversation history, task progress, user preferences, and cached results. This state layer is as critical as the data layer for end-to-end auditability.

State, the live conversation, the task in flight, and memory, customer interactions, user preferences, require transactional storage. Leaving this outside the governance boundary creates a critical hole.

A memory entry like "user X is a high-value EU customer" is sensitive data, subject to the same access and residency rules as the source record. Traditional workarounds like external PostgreSQL or Redis instances reintroduce the core problem: agent state leaves the governed perimeter into systems the governance layer cannot see.

This leads to a data-native agent with an ungoverned dependency. When multiple agents collaborate, sharing memory becomes essential. Without a single source of truth for state, agents diverge, writes collide, and each handoff becomes an ungoverned channel.

Lakebase, a managed PostgreSQL storage within the Databricks platform, addresses these gaps. Agent state becomes a governed asset, inheriting platform access controls and living alongside the agent's data and tools. This transactional layer serves as the swarm’s single source of truth, ensuring consistent state, atomic updates, and traceable memory across collaborating agents.

The Case for Data-Native Agents

The governance argument is compelling, but the advantages of data-native agents extend across security, quality, observability, deployment, latency, and cost.

Tools and data dependencies are versioned and logged with the model, making the entire system auditable and reproducible by default.

Data-native AI agents offer a single control plane for data and agents, embedding policy enforcement directly into query planning and execution. This contrasts sharply with external agents, which require replicating governance across separate AI components like warehouses, vector databases, and SaaS LLMs, leading to fragmented control and potential gaps in fine-grained access control (FGAC).

Security is enhanced as data and models remain within the organization’s cloud perimeter, avoiding the additional attack surfaces created when data leaves the secure environment. Agent quality benefits from end-to-end, platform-native evaluation, rather than fragmented and manual assessments across multiple external vendors.

Data quality consistency is built-in via shared data pipelines, with freshness managed at the platform level. Observability is holistic, capturing all steps centrally alongside model and agent versions, simplifying troubleshooting.

Crucially, agent memory, conversation history, user preferences, learned context, persists in a governed Lakebase, joinable to underlying business data and bound to the same identity model as the rest of the stack. In contrast, memory in separate Redis or PostgreSQL instances lacks lineage to source data and cannot be centrally governed.

The same Unity Catalog governance that protects data now extends to AI agent operations, providing a unified framework for trust and control.