Five developers spending $150,000 a month to run 116 billion tokens cut that to less than $10,000 on a single node, AI Engineer guest Eugene Cheah said, and the math explains his first rule: measure before you optimize.
Cheah, chief executive of Featherless AI and a contributor to the RWKV project under the Linux Foundation, said one Featherless customer serves two trillion tokens a day at about $0.30 per million tokens, or about $60,000 a day, on 512 AMD MI325 GPUs running a private 4,800-billion-parameter mixture-of-experts model at what he described as Claude Opus 4.8 class performance. He said most teams cannot reproduce that price without dedicated capacity, and he warned the rest of the workshop only applies if you spend more than $10,000 a month on inference.
His mechanism starts with keys. Cheah said 30% to 40% of teams that contact Featherless arrive with one shared API key, which makes it impossible to track spend by department, team or use case. Splitting keys by workload lets teams see two opposite economics. Agentic workloads, like coding agents that run overnight with multiple compactions, maximize context and are cache heavy, with only about 1% of requests going to output tokens according to Cheah, while repeated input and cache reads drive cost. Process workloads, like call centers, email routing or scanning 10 million messages in a minute for a kid-safe phone feature, are short context and high volume.
Right sizing follows measurement. For agents, Cheah described a common harness pattern: the strongest model plans and reviews, a faster cheaper model implements, then the strong model checks again. He said teams rarely switch models to save money unless the cheaper model is also faster, and he recommended labeling models as "fast" versus "smart" or by department function so platform owners can swap backends without retraining users. He also advised separating interactive human traffic at higher tokens per second from CI/CD pipelines that can run slower and cheaper at the key level.
For process traffic the search is narrower. Cheah said teams should define evals and then find the cheapest model that passes, noting open models have closed the gap in 2025 and can be 10 times cheaper in narrow domains. He cited StepFun's Step 5-class models nearing frontier quality in finance despite far fewer active parameters, and he pointed to a new class of preview-only one-shot classifiers, known as Jeff and replicated as Simple Jeff in open source, that he said run about 10 times cheaper because they cannot produce chat output. Step 5 Preview itself is a 600B-parameter mixture-of-experts model with 27B active per token and a native 1M-token context window, slated for open weights on Oct. 15 and served by Featherless from that day, per Featherless.
