Fuad Ali, product manager at Arize, demoed self-improvement loops for agents on AI Engineer. Signals is the out-of-the-box SRE agent, Ali said. It reads traces on a 6-hour schedule and proposes fixes, and in the demo it spent 6 minutes isolating a 14-17 second latency spike.
Teams can build agents faster than they can review their traces.
That gap is the workflow Arize wants to close for SREs, coders, marketers and customer-facing bots. The loop relies on traces with tool calls, latency and cost, then evals to score them, then datasets of failures for regression. Ali argued the pattern only works if humans stay in the loop, saying "Just because you can automate something doesn't mean it should be automated, but you should use that automation to really inform the reviewer."
Ali showed the rest of the stack live. Agent Studio lets teams define agents, point them at projects and attach skills. AX CLI bootstraps tracing without manual instrumentation. Evals run as code checks, LLM as a Judge, Agent as a Judge or cheaper decision models, and failed evals feed a task system that grows living datasets. Experiments then prove a fix clears the 88.9% success bar Ali showed without creating new regressions, a pattern Arize says helped one spam case reach 99.5% accuracy while routing 4.6% of cases to humans.
The pitch comes with costs and limits. Skills can pull GitHub, DataDog and APM data, but Ali warned more context can hurt triage. Every improvement has a price, he said, citing an $8 shopping assistant that cost $20 to improve, and Arize exposes harness and run-frequency controls to tune sampling and eval frequency. There is no hard budget rule, Ali said, it really just depends. The platform is API accessible, and even evals are traced so Signal can evaluate itself, a trick Ali called dogfooding that hits diminishing returns when ground truth is just transcripts.
Arize closed with next steps on the calendar. A talk on agent fixes and run-off implications is set for 3:20 tomorrow, according to the workshop, followed at 3:45 by a session on Alex, the in-house Arize agent for creating evals.
