Senior Staff Software Engineer Harish Gaggar told AI Engineer that enterprise agents fail without a governed stack first, and he put a number on one fix: his context optimizer cut a 24,000-token query to 2,400 tokens in a demo, a 90% saving, versus 70% to 78% typically, he said.
The missing piece is usually a gateway.
Gaggar, who works at Intuit Credit Karma, argued most production failures trace to direct exposure of MCP servers, thin tooling on those servers, and no durable way to track, evaluate or remember. His reference architecture routes every call through an MCP gateway for authentication and routing, stores state in a memory DB backed by Cloud Spanner on GCP, Azure or AWS, and logs to job tracking, quality and tool invocation tables with eight quality checks per run scored 0 to 1, he said. It also blocks PII above 90% severity before it reaches models, memory or traces, and adds a human-in-the-loop step, he said.
The framework leans on LangGraph from the LangChain community for graph-based loops and branching, OpenTelemetry with Tempo for traces and Prometheus with Grafana for metrics, plus JWT tokens, Redis and workload identity for access. Gaggar demoed the optimizer again on a Slack bot query, 24,000 tokens down to 1,500 tokens with the optimizer, he said, and recommended a dev portal that scaffolds repos with LangGraph, memory, tracking, PII blocking and context optimization prewired. For air-gapped shops he advised building custom MCP servers and gateways and having security review every tool before production. He also noted he recently released an MCP book available free on Kindle.
That push for a central, policy-enforced gateway mirrors what commercial vendors now sell as the control plane. Barndoor, for example, is described as a closed-source AI gateway with governed MCP access, an LLM gateway and data loss prevention available as SaaS, private cloud or on-prem for regulated teams needing per-call approval, per a 2026 Metorial comparison of enterprise gateways. Integrate.io separately lists Agent Monitor for visibility into MCP tool calls and token costs and Mint Guard with declarative rules and gateway middleware for runtime controls.
What the numbers do not show is cost in latency, accuracy or recall after compression, or failure rates before and after the gateway in live traffic. Gaggar gave token counts and severity thresholds but no controlled A/B on answer quality or PII escape rates at scale, and no price per thousand tokens after optimization.
One change would sharpen the read: publish eval scores and token use for the same tasks with and without the optimizer and the gateway, broken out by model. Until then the takeaway is architectural, as Gaggar put it: "Make sure that we build an enterprise framework, not just an agent, right? Which make sure that you have all the framework underneath," and his other warning that "The AI agent cannot be trusted without continuous monitoring, evaluation, and feedback." The open question is how much governance teams will pay in complexity to get that 70% token saving without losing context.
