Large language models are plagued by a critical flaw: once a reasoning error occurs mid-generation, subsequent tokens often compound the mistake, leading to unrecoverable outputs. This paper introduces a novel solution, Latent Phase-Shift Rollback (LPSR), designed to address this fundamental limitation without requiring fine-tuning or additional forward passes.
Halting Compounding Errors with Latent Phase-Shift Rollback
LPSR operates by monitoring the residual stream at a critical layer during each generation step. It employs a dual gate, combining cosine similarity and entropy, to detect abrupt directional reversals, akin to phase shifts, in the model's internal state. Upon detection, LPSR rolls back the KV-cache and injects a pre-computed steering vector, effectively correcting the erroneous trajectory. This mechanism bypasses the need for gradient computation or further training, offering an inference-time fix. On the MATH-500 benchmark, an 8B model equipped with LPSR achieved a remarkable 44.0% accuracy, a substantial +15.2 percentage point improvement over standard autoregressive generation (28.8%). Crucially, LPSR significantly outperforms prompted self-correction, which scores only 19.8%, by a margin of +24.2 percentage points.
Efficiency and Scalability Beyond Current Paradigms
The efficacy of LPSR extends to efficiency and scalability. It surpasses the Best-of-16 sampling strategy by +7.8 percentage points while operating at 5.4x lower token cost. Furthermore, a standard 70B model's performance (35.2%) is exceeded by LPSR, even with an 8.75x smaller parameter count and approximately 3x the token budget. This demonstrates a potent combination of accuracy improvement and resource optimization. The researchers also identified a fascinating 'detection-correction dissociation,' observing that the optimal layer for error detection (layer 14, AUC 0.718) differs from the optimal layer for task accuracy (layer 16, 44.0%). This suggests that separate layers might be specialized for identifying errors versus implementing corrective actions, a key insight for future architectural designs.