Hill Sampling for Test-Time Scaling: A Simple and Better Alternative to Repeated Sampling, Evolution, and Training argues that much of the machinery built around test-time compute is unnecessary. Instead of elaborate evolutionary search or updating model weights at test time, the authors repeatedly sample candidate program edits from a frozen LLM and keep the best program found so far. On verifiable scientific and algorithmic problems, that simple hill climbing loop matches or outperforms far more complex harnesses, suggesting that retention and selection matter more than sophisticated exploration operators.
The result reframes test-time scaling as a search problem where the generator is already strong enough. If a frozen model can propose useful edits, the cheapest way to improve is to sample, score, and retain, not to orchestrate populations or train during deployment. For teams shipping reasoning systems, that means lower latency, lower cost, and fewer moving parts at inference time.
Why naive sampling and compression break reasoning
Beyond Repeated Sampling: Learning Search Policies for LLM Reasoning makes the complementary case that naive repeated sampling is inefficient because it explores only through local decoding noise, producing near duplicates rather than genuinely different ideas. The paper learns search policies that first sample problem specific concepts, hints, or strategies to steer exploration at a semantic level, showing that directed diversity beats independent draws.
Train Where the Quantized Model Goes: On-Policy Distillation for Low-Bit Reasoning explains a second failure mode when trying to ship efficient reasoning models. Quantization-aware distillation restores short-form QA under sub-3-bit quantization but leaves math and code reasoning impaired, with long generations collapsing into repetitive loops. The authors trace this to quantization-amplified exposure bias: training on fixed corpus prefixes while the quantized model drifts along its own autoregressive trajectory. Their fix is on-policy distillation that trains where the quantized model actually goes, closing the gap between training prefixes and deployed behavior. Together, the three papers point to a simpler, more faithful way to scale reasoning: sample smarter, keep what works, and train compressed models on their own trajectories.
