Debugging AI agents is becoming a significant hurdle as they move beyond simple chatbots to manage complex tasks. When these systems fail, tracing the root cause through long, probabilistic, and sometimes multi-agent interactions is an arduous manual process. Microsoft Research aims to solve this with the introduction of the AgentRx framework, an automated system designed to pinpoint the exact moment an agent's trajectory becomes unrecoverable.
Traditional metrics like task completion are insufficient for understanding agent failures. To build reliable and safe AI, developers need to identify the precise failure point and gather evidence. This is crucial for advancing capabilities in areas like managing cloud incidents and navigating complex web interfaces, moving beyond simple debugging AI agents.
AgentRx: A Structured Diagnostic Approach
AgentRx treats agent execution as a traceable system. Instead of relying on a single large language model to guess errors, it employs a multi-stage pipeline. This process begins with normalizing logs from various domains into a common format.
The core innovation lies in constraint synthesis. AgentRx automatically generates executable constraints from tool schemas and domain policies, such as ensuring API responses are valid JSON or that sensitive data isn't deleted without confirmation. These constraints are then evaluated step-by-step, only when their guard conditions apply, producing an auditable log of evidence-backed violations.
Finally, an LLM judge analyzes this validation log and a grounded nine-category failure taxonomy to identify the critical failure step and its root cause. This systematic approach allows for more precise identification of issues compared to current methods for autonomous systems debugging.