#Transformer Architecture

Recurrence Enhances Transformer Reasoning

Transformers with Temporal Middle-Layer Recurrence (T2MLR) enables persistent intermediate reasoning by fusing cached middle-layer states, outperforming baselines and allowing efficient model retrofitting.

Predictive vs. Generative AI: Key Differences Explained

IBM's Martin Keen clarifies the distinction between predictive AI (forecasting outcomes) and generative AI (creating new content), outlining their core mechanics and use cases.

Transformer Artifacts Unpacked

Research demystifies massive activations and attention sinks in Transformers, revealing them as architectural artifacts enabled by pre-norm configurations.

Beyond the Token Tax: Why Dynamic Large Concept Models (DLCM) are the Future of LLM Efficiency

Bolmo Advances Byte-Level Language Models with Practicality