AI Analysts Lag on Real-World Reasoning

The current generation of AI agents excels at the rote mechanics of financial analysis, document retrieval, formulaic calculations, and spreadsheet updates. However, the true value lies in replicating the nuanced, open-ended reasoning that defines expert human analysts. Existing benchmarks fall short, particularly in evaluating this critical reasoning capability, often relying on noisy, circular model-judged outputs.

Bridging the Reasoning Gap with Grounded Evaluation

To address this deficiency, the authors introduce Hedge-Bench 1.0, a novel benchmark comprising 102 real-world tasks. These tasks are derived directly from the explicit reasoning traces of professional hedge fund analysts, grounded in their use of relevant information sources. This methodology enables deterministic and verifiable grading against established expert steps, circumventing the ambiguity of model-based evaluations.

Underwhelming Performance of Frontier Models

The initial evaluation of state-of-the-art frontier models and agents on Hedge-Bench 1.0 reveals a significant performance gap. These advanced systems scored below 16% on the benchmark, highlighting their current limitations in handling the complex, open-ended reasoning characteristic of expert financial analysis. The dataset and evaluation harness are publicly available to foster further research and development.