AI Research

The Cheaper Model That Billed More

Vals AI's Rayan Krishnan told AI Engineer a cheaper list price can mean higher cost per task when weaker models burn extra tokens.

By Daniel Singer, Editor, StartupHub.ai

1 min read

Rayan Krishnan of Vals AI presenting benchmark charts on stage
Credit: AI Engineer

Rayan Krishnan of Vals AI told AI Engineer that Anthropic's $4-in/$20-out Opus 5.5 cost more per completed tax task than the $10-in/$50-out Fable 5.1 despite its lower list price.

The Cheaper Model That Billed More - AI Engineer
The Cheaper Model That Billed More, AI Engineer

The gap came from token burn, not pricing.

Krishnan said the smaller model scored 3% lower while trying to spend extra tokens to finish the work, so teams paid more for worse accuracy.

That jagged frontier repeats across finance tasks, where Excel modeling in aggregate hides that LBO subtasks run on a lower Pareto curve and drop the supposed frontier model off the efficient edge.

Vals AI now runs more than 6 trillion tokens of evals and ships those checks as Val Smith, which replays GitHub pull requests to test whether an agent could have done the human work and then routes each issue to the cheapest model that clears the bar.

Even the cheapest useful chat today still starts at $0 for free tiers and $8 for ChatGPT Go according to Overpaying for AI pricing data, so routing to the right model matters more than chasing the lowest sticker price.

Mentioned in this story

TopicsVals AIAI EngineerAnthropicOpus 5.5Fable 5.1Val SmithExcel modeling

Daniel Singer

Written by

Daniel Singer

Editor, StartupHub.ai

Daniel Singer is the editor of StartupHub.ai, a technology expert and thought leader on AI and its applications across sectors, from fintech and healthcare to developer tooling and consumer software. He writes and tests the tools covered here thoroughly and regularly, and built StartupHub.ai to give founders, operators and buyers a clearer read on what they are actually being sold.

Test your Agent Readiness now

Paste your website. How well ChatGPT, Claude and the other agents can read and use a site, in about a minute, free.

Watch

The videos behind the news, one flip at a time

The feed mixes the videos from our articles with startup demos and you flip through them like TV channels for free with no signup.