AI Engineer host Fuad Ali, who works on product at Arize AI, walked through two years of rebuilding Alex, the in-product agent inside Arize AI's Arize AX, and said almost everything that survived was harness code, not model magic.
The model got better on its own schedule. The engineering did not.
Ali said the team started in late 2023 with a 3.5-era prototype that hallucinated about half the time, then shipped a copilot-style router over skills, then tore it up again in August 2025 for an iterative tool-calling loop with a to-do tool and again for Alex 2.0 in February 2026 and One Alex in June. Alex, per Ali, can now read traces, turn findings into tests and synthetic data, plan work and run multi-step tasks, and the platform itself is described by Arize as a managed AI engineering platform built on the same open standards.
Planning became the first fight, Ali said. When the team handed the model a to-do list it burned calls just checking boxes, sometimes 20 plus iterations on the list alone and losing focus after long tool sequences. They moved the plan into the harness piece by piece, first adding an in-progress marker, then rendering the plan as a message on every model call so rules sit next to data, then dropping the marker entirely, and they are now testing without a to-do tool at all while the harness keeps state and the completion check.
Memory got the same treatment. Ali compared a giant tool result to a book landing on a desk, then too many books filling the desk, and said the fix was to treat huge results like files and keep only a sticky note in context. For Alex that means truncating large JSON trace payloads and letting the agent fetch the full blob with code like JQ when needed, plus a fixed token budget for history and a checkpoint that the harness writes and then drops. For long-term memory, Ali said the team tried vector retrieval, episodic and semantic stores and temporal knowledge graphs and lost to a single markdown file per user capped at 8,000 characters, written by a background job after the session and loaded on every full request because it is exact, cheap and auditable.
Ali said the team also collapsed ten page-specific specialists into one agent that loads skill instructions and tools on demand, and stopped chasing the newest model by default. He argued fast and good enough wins if you invest in the harness, with the current default around Sonar 4.5, and noted Arize lets customers bring their own provider, which forces defensive design. The finish condition is now an explicit finish call that bounces back up to three times if plan items remain open, with a warning one step before the iteration limit and a force finish, after the old heuristic of text without tool calls equals done broke on models that announce a step in one turn and call in the next. Tracing stayed non negotiable, Ali said, with live Alex traces feeding regression data sets and session data sets played back through experiments, and with Signal, the trace-reading agent, flagging qualitative bugs like 43 identical tool calls or a 227 second latency that never throw an error. That debugging loop echoes the company's recent push to let agents flag and fix their own retry loops overnight, which we covered after Arize said its agents can fix themselves overnight.
Ali declined to claim the harness solves judgment. He said golden data sets, judge alignment and the speed versus quality trade still need people, and he left open how Arize will reliably auto-route per task or train small post-trained models for surfaces that may not need a loop at all.
