# Rayan Garg on Why Long Horizon AI Agents Need Better Verifiers _Rayan Garg from Theta Software explains why long horizon AI agent benchmarks need accurate environment design and final-state verifiers._ **Published:** 2026-07-31 **Source:** https://www.startuphub.ai/ai-news/ai-research/2026/rayan-garg-on-why-long-horizon-ai-agents-need-better-verifiers --- As autonomous AI agents attempt to handle tasks that stretch over hours or days, measuring their actual performance has become one of the hardest challenges in machine learning. In a technical talk, Rayan Garg from Theta Software broke down why existing benchmarks for long horizon work fall short and how the industry must rethink environment design and task verification. Rayan GargCore researcher at Theta Software focusing on dependable evaluations for autonomous agentsFrom the article 5 mentionsIn a technical talk, Rayan Garg from Theta Software broke down why existing benchmarks for long horizon work fall short and how the industry must rethink environment design and task verification.studiesLong Horizon AIContextAI agents handling tasks stretching over hours or days, hard to measure performanceFrom the article 5 mentionsIndustry benchmarks often measure long horizon performance using a simple time threshold.Flawed BenchmarksDrivercurrent industry benchmarks use simple time thresholds like 16-hour completionFrom the article 4 mentionsHis work addresses the growing gap between headline benchmark metrics and real-world agent performance.Cascading ErrorsDriversmall errors early in long tasks can lead to large failures laterState Space ComplexityDriverdifficulty in tracking all possible states an agent can enter during long tasksFrom the article 4 mentionsReal difficulty stems from state complexity and high-stakes choices, not just elapsed time.Noisy DataOutcometime threshold approach produces unreliable data for agent performance evaluationFrom the article 2 mentionsGarg argues that this approach produces noisy, unreliable data.Rethink VerificationEffectFrom the article 2 mentionsIn a technical talk, Rayan Garg from Theta Software broke down why existing benchmarks for long horizon work fall short and how the industry must rethink environment design and task verification.meansBetter VerifiersEffectneed accurate environment design and final-state verifiers for long horizon tasksFrom the article 2 mentionsRayan Garg is a researcher and engineer at Theta Software, where his focus centers on creating dependable evaluations, environments, and verifiers for autonomous agents. ## Who Is Rayan Garg Rayan Garg is a researcher and engineer at Theta Software, where his focus centers on creating dependable evaluations, environments, and verifiers for autonomous agents. His work addresses the growing gap between headline benchmark metrics and real-world agent performance. ## The Flaws in Current Long Horizon Metrics Industry benchmarks often measure long horizon performance using a simple time threshold. For instance, an evaluation might define a model as capable of long horizon tasks if it successfully completes work at the 16-hour mark. Garg argues that this approach produces noisy, unreliable data. Wall clock time hides the true difficulty of a task. Human time estimates vary wildly depending on skill level and experience. Furthermore, an artificially stretched task with serial dependencies might take hours to complete without testing complex reasoning. Real difficulty stems from state complexity and high-stakes choices, not just elapsed time. ## Cascading Errors and State Space Complexity True long horizon difficulty occurs when a single mistake early in a process cascades through every subsequent step. A bad initial database query or misconfigured environment setting corrupts all downstream actions, turning a simple task into an unrecoverable failure mode. Evaluating models under these conditions requires creating realistic, dynamic environments. Standardized evaluation becomes exceptionally difficult as state spaces grow. When state spaces balloon, traditional judge models often struggle to accurately evaluate whether an agent truly succeeded. ## Verifying Success from Final State To avoid relying on subjective judge estimates, Garg highlights the importance of verifying agent success from the actual final state of the environment. Rather than grading step-by-step trace files or trusting a judge model to guess correctness, system designers should evaluate concrete final artifacts. This approach involves collapsing massive state spaces using sample trajectories and structured tools. Evaluators must also ensure LLM judges do not see information they should not, preventing data leakage and unearned success scores. ## Reusing Agents for Automated QA Another major strategy Garg shares is using specialized agents to sift through artifacts like CI logs, build outputs, and execution traces. By pairing final-state verification with specialized agentic grading, teams can build clear rubrics and quality assurance pipelines. Ultimately, progress in long horizon autonomous work lives or dies on environment and verifier design. Simply pushing headline benchmark scores without honest evaluation frameworks will fail to produce reliable agents in production. --- Original analysis from [startuphub.ai](https://www.startuphub.ai), the #1 AI startup directory.