The relentless pursuit of LLM advancement has historically centered on scaling pre-training, post-training, and test-time compute. However, this paradigm overlooks a crucial capability: verification, the ability to rigorously assess solution correctness. This paper introduces LLM-as-a-Verifier, a novel framework that establishes verification as a new scaling axis, unlocking fine-grained feedback for agentic tasks without requiring additional model training.
Probabilistic Scoring for Enhanced Granularity
LLM-as-a-Verifier departs from traditional LM judges that output discrete scores. Instead, it computes the expectation over the distribution of scoring token logits, yielding continuous scores. This probabilistic approach unlocks scaling along multiple dimensions: score granularity, repeated evaluation, and criteria decomposition. Crucially, increasing scoring granularity demonstrably improves the separation between correct and incorrect solutions, leading to more calibrated comparisons. Furthermore, repeated evaluations and criteria decomposition consistently boost verification accuracy by reducing variance and complexity.