Rohan Pandey demoed an optimization agent on AI Engineer that hunts the best VLM serving config automatically instead of hand tuning recipes.
Why do settings that look good in a config file still lose on cost and speed?
Because manual recipes are rarely tested across real traffic mixes, and Pandey said the agent replaces that guesswork with Bayesian optimization, a search method that narrows the parameter space mathematically over repeated trials.
Pandey showed the run starting from the default YAML that ships with model weights, here GPT OSS 120B on a droplet, with tuner.py iterating across four regimes called chat short, chat long, rag and batch classify.
In the reported GPT OSS run the agent tested 50 trials over about four to five hours and lifted goodput tokens by 17.5 percent from the starting recipe, per Pandey.
In a nine trial run on DeepSeek R1 Distill the ninth trial was best, with token throughput doubling and inter token latency, a measure of delay between generated tokens, cut by almost half, per Pandey.
For teams serving VLMs the workflow shift is to let the tuner burn a droplet for hours and return an HTML report plus an optimized YAML instead of paying an engineer to grid search.
That fits DigitalOcean work beyond the demo to build benchmarking and regression infrastructure for TTFT, ITL and throughput with accuracy validation before ship, per its Principal Engineer listing, so watch whether this tuner becomes a hosted job rather than a script you babysit.
