OutRival says tokenmaxxing means doing more with fewer tokens

Alvin Pane, head of agent intelligence at OutRival, told AI Engineer that reclaiming tokenmaxxing means extracting maximum intelligence from the minimum token payload, not burning the most compute, while his team processes billions of tokens daily across phone, text and email workers.

The tax is self-inflicted. Pane showed a synthetic HarborBite intent classifier bloated to nearly 5,000 tokens with 20 edge cases, pedantic definitions and hard-coded transcripts that now ships on millions of requests daily.

That bloat compounds into latency and millions of dollars, Pane said, and the fix starts with measurement not rewriting. Instrument the shared model client with OpenTelemetry, trace every call, tool and retry for a single task, and log prompt version, model settings, token usage, latency and cost to build a production benchmark.

Most teams then fail the same test. Pane said they handed the large prompt to a cheaper model and saw 18% disagreement against the expensive production baseline, so they rejected it.

Then they asked if prod agrees with prod. In a ceiling test they ran the production model three times on identical inputs and found 7% self-disagreement, or 93% agreement, meaning they demanded perfection the frontier system could not hit itself.

With that baseline Pane pulled three levers in sequence. First maximize prefix cache rate by structuring prompts for cache hits. Second shrink the prompt via search, defining the input, output schema and scoring function and letting an optimizer like DSPy with GEPA, transcribed as DSPI and Jeepa, strip worked examples and long rules for a 51% cut in instruction tokens with the same contract. Third downsize the model by routing simple steps to cheaper models or letting a small model search its own prompt space to match the large model's labels, though Pane warned downsizing introduces overfitting and new failures and many candidates were not shippable.

Ship decisions need paired shadow math. Run production against production, candidate against candidate, and candidate against production on identical inputs, then subtract average internal disagreement from cross-version disagreement to isolate true drift. For HarborBite that review found 443 regressions against 2,463 improvements, nearly six improvements per regression on a prompt half the size, but Pane said to ship only when the lower bound of the confidence interval for net gain sits above zero and critical intents hold separately. His homework: pick your most expensive task Monday, measure its ceiling, and search away the bloat.