Visual TL;DR
Autoregressive Decoding LimitsDriver
From the articleThe inherent limitations of autoregressive decoding in Transformers often lead to the compression of rich hidden states, hindering the persistence of intermediate reasoning across tokens.
T2MLR IntroducedCore
From the article 2 mentionsA novel approach, Transformers with Temporal Middle-Layer Recurrence (T2MLR), directly addresses this by fusing cached middle-layer representations from previous tokens into earlier layers of the current token's computation.
Decouples ComputationEffect
From the article 2 mentionsThis architectural shift allows abstract intermediate computations to persist across decoding steps, effectively bypassing the token-space compression issue with minimal inference overhead.
Targeted RecurrenceContext
From the article 3 mentionsThe researchers found that applying recurrence to a localized middle-layer block, even as little as 20% of the network, often yields performance superior to full-layer recurrence.
Efficient RetrofittingEffect
allows existing models to adopt T2MLR with minimal inference overhead
From the articleBy retrofitting the recurrent pathway into an existing 1.7B parameter Transformer and performing brief fine-tuning, significant improvements in math reasoning were observed.
Enhanced Latent ReasoningEffect
often yields superior performance compared to full-layer recurrence for complex tasks
From the articleThis suggests that effective latent reasoning in Transformers can emerge more powerfully from strategically placed middle-layer recurrence rather than broad, network-wide looping.
Outperforms BaselinesOutcome
T2MLR consistently shows better results on various multi-step inferential tasks
