The conventional wisdom treats pretrained models as mere starting points for iterative adaptation. However, a new perspective from Yulu Gan and Phillip Isola, detailed on arXiv, reframes pretraining's outcome not as a single vector, but as a distribution rich with task-specific experts.
The Dense Landscape of Large Model Experts
In small models, specialized solutions are rare, requiring structured optimization like gradient descent to find. The researchers observed a notable shift in large, well-pretrained models: the density of these task-experts increases dramatically. This means diverse, task-improving specialists are not outliers but populate a substantial fraction of the neighborhood around the pretrained weights. This insight fundamentally alters how we approach adapting these powerful models.
Random Sampling: A Surprisingly Potent Post-Training Strategy
Motivated by the dense expert landscape, the authors explore a simple, fully parallel post-training method. It involves sampling a number of parameter perturbations at random, selecting the top performers, and ensembling their predictions via majority vote. Astonishingly, this straightforward approach proves competitive with established, more complex post-training AI optimization techniques like PPO, GRPO, and ES for contemporary large-scale models. This suggests a paradigm shift towards more accessible and efficient post-training AI optimization.