Artificial Intelligence

Databricks Cuts AI Costs with Smart Routing

Databricks' new Smart Routing feature in Unity AI Gateway automatically matches AI coding tasks to the most cost-effective models, cutting expenses by over 30%.

StartupHub.ai Staff

By StartupHub.ai Staff, reviewed by Daniel Singer, Editor

4 min read

Databricks Unity AI Gateway Smart Routing interface showing cost optimization
Visual TL;DR
AI Model OverloadDriver
From the article 9+ mentionsThis move comes as the sheer volume of available AI models and the complexity of coding tasks create a costly dilemma for users.
High AI CostsDriver
defaulting every coding request to the most advanced, expensive AI model
From the article 5 mentionsDatabricks is rolling out a new feature designed to tackle one of the biggest headaches for AI developers and enterprises: runaway costs.
Smart RoutingCore
Unity AI Gateway feature automatically matches tasks to cost-effective models
From the article 9+ mentionsThe company announced its Unity AI Gateway Smart Routing, now in beta, which promises to slash the cost per AI coding task by more than 30% without sacrificing quality.
Task Complexity AssessedContext
From the article 3 mentionsInstead of defaulting every coding request to the most advanced, and therefore most expensive, AI model, the system intelligently assesses the complexity of each task.
Cost-Effective ModelsEffect
directs tasks to appropriate models, from top-tier to more economical options
From the article 9+ mentionsFuture research will focus on scenarios where task scoping is inherently clearer, such as PR reviews or batch jobs, and on enabling cost-effective model switching within a single session.
30%+ Cost SavingsOutcome
slashes cost per AI coding task by over 30% without sacrificing quality
From the article 7 mentionsEvaluating the effectiveness of Smart Routing involves monitoring both cost savings and developer productivity.
Quality MaintainedOutcome
internal workloads matched Opus 5 performance at 65% of the cost
From the articleThe company announced its Unity AI Gateway Smart Routing, now in beta, which promises to slash the cost per AI coding task by more than 30% without sacrificing quality.
Contents(4)

Databricks is rolling out a new feature designed to tackle one of the biggest headaches for AI developers and enterprises: runaway costs. The company announced its Unity AI Gateway Smart Routing, now in beta, which promises to slash the cost per AI coding task by more than 30% without sacrificing quality. This move comes as the sheer volume of available AI models and the complexity of coding tasks create a costly dilemma for users.

The core idea behind Smart Routing is simple yet powerful. Instead of defaulting every coding request to the most advanced, and therefore most expensive, AI model, the system intelligently assesses the complexity of each task. It then directs the task to the most appropriate model, whether that's a top-tier, frontier model for intricate problems or a more economical option for simpler jobs. Databricks reports that internal coding workloads saw performance matching a leading model like Opus 5, but at just 65% of the cost. Public benchmarks showed similar savings, with Smart Routing matching Opus 5's performance at less than half the price.

The Problem of Choice Overload

The AI development space is exploding with new models. In 2026 alone, Databricks noted the release of 33 new models. This proliferation, while a boon for innovation, presents users with overwhelming choices. Developers often resort to using the most capable model for every task to avoid the cognitive load of selection, leading to significant overspending on simpler operations. Databricks' own research found that much of everyday development work, like small code edits or bug fixes, doesn't require the most powerful, expensive AI.

How Smart Routing Works

Databricks opted for a "task-aware routing" approach. Unlike per-request routing, which can degrade performance by disrupting model cache efficiency, task-aware routing assesses the task at the session's outset. A small, fast model analyzes the task description and metadata, categorizing it by system area, code evidence, failure type, and project context. This classification helps determine a task-type and language family. The router then uses this information to select the best model class, defaulting to a medium-tier model and escalating to more powerful options for complex tasks or delegating to cheaper ones for simpler ones. This strategy aims to preserve cache hit rates, which are critical for cost-effective AI inference at scale.

This system was developed by Ankit Mathur, Ivan Zhou, Bryan Qiu, Rohit Agrawal, Elise Gonzales, and Kelly Albano, with the goal of optimizing AI coding costs without hindering developer productivity.

Beyond Model Routing: Omnigent Integration

Smart Routing's capabilities extend further through its integration with Omnigent, Databricks' meta-harness for coding agents. When developers use Omnigent, they can enable Smart Routing to automatically select not only the optimal model but also the most suitable coding harness for each task. This layered approach allows for nuanced routing decisions across complex workflows, such as assigning large codebase summarization to cheaper models while using more expensive ones for architectural design. This orchestration aims to deliver substantial savings by ensuring the right tools are used for every stage of a project.

The company reports that StartupHub.ai data indicates Databricks holds a strong position in the market with a score of 82/100, placing it among top-tier competitors like Alphabet Inc. (NASDAQ:GOOGL) with a score of 79/100 and Palantir (NASDAQ:PLTR) at 85/100. Databricks has also seen significant financial backing, with verified financials showing a $5B strategic financing round in 2026, valuing the company at $190B post-money.

Evaluating Success and Future Directions

Evaluating the effectiveness of Smart Routing involves monitoring both cost savings and developer productivity. Databricks logs coding session traces into Unity Catalog for analysis, using both AI and human review. Early findings confirmed that a significant portion of sessions were using expensive models for tasks that didn't require them. The company plans to refine the system further by improving its ability to accurately assess task complexity and potentially reassess routing decisions mid-session if a task evolves. Future research will focus on scenarios where task scoping is inherently clearer, such as PR reviews or batch jobs, and on enabling cost-effective model switching within a single session.

This development is significant for enterprises looking to control burgeoning AI expenditures. By automating the complex decision-making around model selection, Databricks is making advanced AI more accessible and cost-effective for developers.

Mentioned in this story

Sources

Get the free daily brief every morning

Every morning the day's AI news, new AI startups and new funding rounds. Free and takes a few minutes to read, unsubscribe at any time.

Join 64,000 other subscribers.

TopicsDatabricksUnity AI GatewaySmart RoutingAnkit MathurIvan ZhouBryan QiuRohit AgrawalElise Gonzales

StartupHub.ai StaffDaniel SingerWritten by StartupHub.ai Staff, Editorial teamReviewed by Daniel Singer, EditorAbout

StartupHub.ai Staff, Editorial team

The staff writers of StartupHub.ai, ranging from investment analysts to avid AI tool users, early adopters and critical enthusiasts. Backgrounds span engineering, business and the arts. We hold every piece to rigorous standards of research and review.

Daniel Singer, Editor

Daniel Singer is an AI expert who has been writing about, covering and building AI technology since 2018.

More from Daniel Singer

Test your Agent Readiness now

Paste your website. How well ChatGPT, Claude and the other agents can read and use a site, in about a minute.

Watch

The videos behind the news, one flip at a time

The feed mixes the videos from our articles with startup demos and you flip through them like TV channels for free with no signup.