#Transformer Architecture

3 articles with this tag

Recurrence Enhances Transformer Reasoning
AI Research

Recurrence Enhances Transformer Reasoning

Transformers with Temporal Middle-Layer Recurrence (T2MLR) enables persistent intermediate reasoning by fusing cached middle-layer states, outperforming baselines and allowing efficient model retrofitting.

18 days ago
Predictive vs. Generative AI: Key Differences Explained
Artificial Intelligence

Predictive vs. Generative AI: Key Differences Explained

IBM's Martin Keen clarifies the distinction between predictive AI (forecasting outcomes) and generative AI (creating new content), outlining their core mechanics and use cases.

3 months ago
Transformer Artifacts Unpacked
AI Research

Transformer Artifacts Unpacked

Research demystifies massive activations and attention sinks in Transformers, revealing them as architectural artifacts enabled by pre-norm configurations.

5 months ago