# DigitalOcean: Model Routing Beats Benchmarks _DigitalOcean's Archana Kamath and Tyler Gillam discuss model routing, arguing that preferences like cost and latency should dictate LLM choices over benchmarks, and showcase their open-source inference router._ **Published:** 2026-08-22 **Source:** https://www.startuphub.ai/ai-news/artificial-intelligence/2026/digitalocean-model-routing-beats-benchmarks --- In the rapidly evolving world of AI, the pursuit of the 'best' model often leads teams to chase benchmark scores. However, Archana Kamath, VP of Engineering for Inference Engine and AI Infrastructure at DigitalOcean, argues this is a flawed instinct. Speaking at the AI Engineer World's Fair, Kamath, alongside Tyler Gillam, who built parts of DigitalOcean's inference router, explained why preferences, not just benchmarks, should dictate model selection. One-Model Habit FlawedDriverchasing benchmarks for LLM selection often leads to suboptimal choicesdriven by3 Forces Break HabitDriverexploding inference costs, poor model fit, and single-model riskrequiresRight Model Per RequestContextpreferences like cost and latency should dictate LLM choicesFrom the article 4 mentionsDigitalOcean's commitment to an open-source, preference-driven approach aims to empower teams to build more efficiently and cost-effectively, ensuring they can always select the 'right' model for every specific request.implemented byDO Purpose-Built RouterCoreDigitalOcean's open-source inference router showcased at AI Engineer World's FairFrom the article 9+ mentionsThe system utilizes an open proxy plan and a purpose-built routing model, both open source, ensuring no vendor lock-in.Router Key FeaturesCoreenables dynamic model selection based on user-defined preferencesBeats BenchmarksOutcomemodel routing approach outperforms traditional benchmark-driven selectionFrom the article 3 mentionsIn the rapidly evolving world of AI, the pursuit of the 'best' model often leads teams to chase benchmark scores.enablesFuture AI FoundationEffectlays groundwork for more efficient and adaptable AI developmentFrom the article"Routing is the foundation, not the destination," she stated. ## The Three Forces Breaking the One-Model Habit Kamath highlighted three key reasons why the traditional approach of relying on a single, top-performing model is no longer sufficient: - **Cost:** Inference bills are exploding, with major companies like Walmart, Uber, and Microsoft actively capping usage to control costs. - **Fit:** A single, powerful model is often overkill for many tasks. Developers end up paying premium rates for work that smaller, more specialized models can handle effectively. - **Risk:** Relying on a single model introduces significant risk. If that model experiences downtime or degradation, an entire product can be compromised with no failover. Kamath emphasized that **"Model orchestration is actually the new FinOps."** She noted that while cloud cost optimization took years to mature, AI model cost optimization is arriving in months, driven by these critical factors. ## The Right Model Depends on the Request There isn't a single 'best' model for every job. The optimal choice depends on a combination of factors: - **Task:** What is the model actually trying to achieve? For classification and labeling, a small open model might suffice, offering significant cost savings. - **System Prompts & Tools:** The scaffolding around the model and how it's prompted influences its performance. - **Cost Limits:** The budget available for a specific request is a crucial consideration. - **Latency Needs:** Different use cases have varying latency requirements. For inline code completion, fast routing is essential, potentially requiring a faster, larger model. - **End-User Preference:** Ultimately, the user's experience and expectations drive the model selection. For tasks like code generation and bug fixing, a mid-weight open model can provide good cost optimizations. However, for accuracy-critical tasks such as code review and security, a more advanced, 'frontier' model might be necessary. ## DigitalOcean's Purpose-Built Router Tyler Gillam introduced DigitalOcean's inference router, explaining its different approach: **"Many builders have tried auto-routing before, but the problem was that it feels like a black box. The router makes a choice and if that choice results in poor performance, you really have no way of improving it. We built ours differently at the architecture level."** The system utilizes an open proxy plan and a [purpose-built routing model](/ai-news/insights/2026/best-multi-agent-frameworks-ai-teams-2026), both open source, ensuring no vendor lock-in. Users describe what matters for their workload, cost, latency, quality, preferred models, or hard rules, and the router uses this context to select the appropriate model per request. This specialized routing model is fast, operating in under 200 milliseconds, and is free of charge. **"Routing decision and under 200 milliseconds per request,"** Kamath added. **"It runs on a custom mixture of experts model purpose-built for routing. Zero application code changes needed from you to get it to adopt and it's free and included so you do not have to roll out your own router."** ## Live Demo and Key Features Gillam then demonstrated the router's capabilities, showcasing its UI for configuration and evaluation. He highlighted presets for common tasks like software engineering and general writing, as well as the ability to create custom routers. The demo illustrated how the router dynamically selects different models based on the task, leading to significant cost and latency improvements compared to using a single, high-end model like Opus for all requests. For instance, a code generation task was routed to a faster, cheaper model, while a more complex task might utilize a different, more powerful option. **"The difference is simple. This is routing you can customize, evaluate and improve without vendor lock in."** The presentation also touched on the importance of validation through evaluations, emphasizing that teams should trust their own benchmarks over external leaderboards. DigitalOcean's approach facilitates this by allowing users to define their preferences and validate the router's performance with their own data. ## The Foundation for Future AI Development Kamath concluded by positioning routing as a foundational layer for AI development, with further enhancements like caching and personalization building upon it. **"Routing is the foundation, not the destination,"** she stated. **"The more you route and evaluate, the better the router does for your workload."** DigitalOcean's commitment to an open-source, preference-driven approach aims to empower teams to build more efficiently and cost-effectively, ensuring they can always select the 'right' model for every specific request. --- Original analysis from [startuphub.ai](https://www.startuphub.ai), the #1 AI startup directory. © StartupHub.ai. All rights reserved. You may not republish this article in full without a license. Search engines and AI research tools may crawl and summarize for reference. Bulk reproduction or model training on this content requires a license. See https://www.startuphub.ai/terms.