#Transformer Models

Faster LLMs by Reshaping Sparsity

Sakana AI and NVIDIA unveil a new method that reshapes sparsity in LLMs to boost GPU efficiency, achieving over 20% speedups.

AI Brains vs. Human Minds

Exploring the fundamental differences between transformer AI models and the human brain's continuous learning and sensory grounding.

AI's Consciousness Debate

Vishal Misra and Martin Casado discuss LLM functionality, the path to AGI, and the role of data in AI development.

Predicting Transformer Training Instability

Researchers introduce RKSP, a method to predict transformer training divergence from a single forward pass, and KSS, a technique to actively prevent it, saving compute and enabling higher learning rates.

TabICLv2: Spreadsheets Meet AI's Future

TabICLv2 emerges as a breakthrough tabular foundation model, challenging traditional methods with zero-shot, in-context learning on massive datasets.

AlphaProof system proves its worth at the Math Olympiad

Nested Learning AI Tackles Catastrophic Forgetting