Abdul Dakkak of Modular told AI Engineer the company's Max stack is 5.5 times faster on time to first token for Kimi K2.5 than vLLM and delivers 1.5 times more tokens per minute per GPU on Nvidia B200, according to benchmarks Dakkak said were independently verified by Artificial Analysis.
That is the claim for a hardware market he called a Cambrian explosion.
Dakkak said Modular is rebuilding the inference stack to hide hardware choice. At the bottom is Mojo, a Pythonic systems language he likened to C++, Rust or CUDA C++ that targets CPUs, GPUs and accelerators and where knowing Python covers about 90% of the syntax. Dakkak said Mojo competes with CUDA on performance and aims for portability without the usability tradeoffs of ROCm or OpenCL, while Triton hits about 70% of peak performance, and he pointed to more than 700,000 lines of kernel code the company says it has open sourced as evidence. Per I-Programmer, Mojo has separately been reported to have reached 1.0 and to be going open source, which would tighten the "next few months" timeline Dakkak outlined for stabilization and open sourcing.
On top of Mojo sits Max, which Dakkak framed as a replacement for the PyTorch plus vLLM or SGLang or TensorRT-LLM patchwork for generative inference, with native KV cache handling, graph compilation and eager execution for debugging. Dakkak said Max supports thousands of architectures from Gemma 4 to MiniMax and GLM with an OpenAI-compatible endpoint, and can schedule prefill on one accelerator and decode on another across multiple GPUs and nodes. For image generation, he said Max is 7 times faster than Diffusers with compile on Flux 2, and 3 times faster than the next competitor on Flux 2 dev per Artificial Analysis, while on AMD MI355 he cited a 4 times throughput advantage over the equivalent setup, noting AMD parts are typically about 30% cheaper than Nvidia equivalents, which changes any total cost comparison.
The numbers are stated as raw speed without distillation or quantization tricks, Dakkak said, and the company invites users to pip install modular to reproduce them. What is still gated matters. Mojo 1.0 and full open source were described as imminent, not available, during the talk. The comparisons are presented as best versus best but without published scripts, batch sizes, precision, or price per token, and the only external validation named is Artificial Analysis on a single image model. No production customer names, sustained multi-node latency curves, or failure modes were shown, even as Dakkak said modern inference is distributed and relies on routing, KV cache offloading and disaggregated execution.
