Why Decagon Runs 90 Percent of Its Enterprise AI on Open Source

In the rapid rush to deploy enterprise AI agents, many founders assumed the primary battle would be won by calling frontier APIs like OpenAI and Anthropic. But on a recent episode of The a16z Show, Decagon co-founders Jesse Zhang and Ashwin Sreenivas outlined a starkly different operational reality: 90 percent of their platform workflow now runs on fine-tuned open-source models.

The Shift from Frontier APIs to Fine-Tuned Open Source

When Decagon launched its enterprise AI customer service platform, the engineering team relied almost exclusively on frontier closed-source models. The priority was getting a functional product to market quickly. However, as the company scaled to handle millions of customer interactions for global brands and launched voice agents, latency became an existential metric.

"When you want to go to smaller models, unfortunately, the frontier labs do have small models, but you can't really control them in the way that you want," Zhang explained during the interview. "Most small models out of the box are not going to be good enough at the task that we want them to do. So you have to fine-tune them."

By breaking complex agentic workflows into distinct, discrete sub-tasks, such as topic classification or fraud detection, Decagon realized that individual steps do not require general intelligence like coding or advanced mathematics. A smaller, open-source model trained specifically on that single task delivers equal or superior accuracy with significantly lower latency.

The False Trade-Off Between Intelligence, Cost, and Speed

A common belief in Silicon Valley holds that teams must choose between high-cost frontier intelligence and cheaper, dumber models. Sreenivas argued that this framing fundamentally misinterprets post-training capabilities in specialized enterprise domains.

"When we fine-tune smaller, dumber models, it's that they're just not as general purpose, but on the specific task we want them to do, they actually outperform the large, smart, state-of-the-art models," Sreenivas said. "So we end up getting all three things. It is better at the task, it is cheaper, and it is faster."

Decagon still reserves closed frontier models for the remaining 10 percent of its workload. These include open-ended, exploratory tasks such as Duet Autopilot, an auxiliary agent that analyzes millions of historical customer transcripts, identifies recurring failure patterns, and automatically drafts process updates and test simulations.

Why Software Layer Moats Persist in the Age of AGI

The conversation addressed the industry debate over whether general artificial intelligence will commoditize application software, turning application startups into thin user interfaces paired with manual implementation staff. Both founders pushed back against the narrative that frontier labs will absorb the entire software market.

Even if foundational model intelligence reaches general human capability, enterprise deployments require extensive software infrastructure around the model. Large organizations need granular permissioning, audit trails, system integrations, compliance monitoring, and mechanisms to encode complex business logic.

StartupHub.ai data gives Decagon a platform score of 65/100 as it competes in the enterprise intelligence sector. Among broader general intelligence startups tracked by StartupHub.ai data, AGI holds a score of 47/100 with $10M raised in verified seed funding, alongside peers such as Adept AI (71/100), Hebbia (72/100), Peak (70/100), Inflection AI (55/100), and Manus AI (75/100).

Redefining the Forward Deployed Engineering Model

With forward deployed engineers becoming a popular hiring trend across Silicon Valley, Sreenivas, a former deployment strategist at Palantir Technologies (NYSE:PLTR), warned that mismanaging the role turns software startups into service firms.

Because AI workflows are novel, forward deployed teams are initially essential to embed with customers and uncover how work actually gets done. However, those learnings must immediately flow back into the core product architecture.

"Forward deployed engineers eat pain and excrete product," Sreenivas noted, quoting an internal Palantir maxim. He emphasized that if forward deployed engineers spend their time writing custom, one-off code for individual client requests rather than building scalable platform features, the business risks becoming a glorified IT consultancy akin to Accenture (NYSE:ACN) rather than a scalable software business.