IMEC shows when owning AI models beats API bills

Steven Latré, who leads AI at IMEC, told AI Engineer when running your own models beats paying an API per token.

An open-weight model is a model whose parameters you can download and run on your own servers instead of calling a closed API.

Latré said teams in 2024 and 2025 chased every new frontier model, then moved from subscriptions to token pricing and in the last year to routing tasks to the cheapest capable model.

To test the next phase of owning the stack, IMEC ran more than a thousand runs on long horizon SWE-bench Pro coding tasks using popular harnesses including Codex and Claude Code across open-weight models and matched hardware.

Smaller models up to a few hundred billion parameters like Qwen and DeepSeek solved 32.5% to 40% of tasks, GLM hit Opus level, and Kimi K3 hit 86%, Latré said, but Qwen needed four times the GPU rental of DeepSeek for similar accuracy and stepping from GLM to Kimi K3 added 20% accuracy for only 15% more rental cost.

That hardware math collapses under concurrency, Latré said, where even on optimal hardware Kimi K3 handled about four concurrent users at three times the latency of the Claude Code API baseline, small stacks needed 89% GPU utilization to beat APIs versus 15% for an 8x B200 system, and GLM class workloads were the threshold where owning infrastructure saved about 24% to 25% in total cost of ownership.

The harness choice did not move accuracy, Latré said, but it halved the bill, with Codex generating half the tokens and half the GPU wall time of peers on GLM.

Self-hosting is mostly fixed cost with near-zero marginal cost after hardware and setup while hosted open-weight APIs remain variable cost through providers like Together AI, Groq and Fireworks AI, per Layer3 Labs, and Latré said IMEC will publish its full benchmarking dataset and invite hardware and software partners into an alliance to keep measuring quantization, speculative decoding and new hardware.