# PoM: Linear Complexity Attention Replacement

_Polynomial Mixer (PoM) offers a linear complexity replacement for self-attention, matching performance and drastically cutting costs for long sequences._

**Published:** 2026-04-08
**Source:** https://www.startuphub.ai/ai-news/ai-research/2026/pom-linear-complexity-attention-replacement

---

The quadratic complexity of self-attention has long been a bottleneck for scaling transformer models to handle increasingly long sequences. This limitation hinders progress in domains demanding extensive contextual understanding. A new mechanism, the [Polynomial Mixer (PoM)](https://arxiv.org/abs/2604.06129v1), emerges as a direct, linear-complexity alternative.

## Polynomial Mixer: A Learned Polynomial Approximation

The Polynomial Mixer (PoM) introduces a paradigm shift by aggregating input tokens into a compressed representation via a learned polynomial function. This compact representation then allows each token to retrieve contextual information. Crucially, the researchers demonstrate that PoM satisfies the contextual mapping property, mathematically ensuring that [transformer](/ai-news/ai-video/2026/state-space-models-redefine-ai-architecture-beyond-the-transformer-bottleneck)s augmented with PoM retain their capacity as universal sequence-to-sequence approximators. This theoretical grounding is vital for trust and adoption.

## Broad Domain Efficacy with Drastic Cost Reduction

The practical implications of PoM are profound. The authors validated its effectiveness by substituting standard self-attention with PoM across five distinct domains: text generation, handwritten text recognition, image generation, 3D modeling, and Earth observation. In each case, PoM achieved performance parity with established attention-based models. The true breakthrough, however, lies in its computational efficiency. When processing long sequences, PoM offers a drastic reduction in computational cost, unlocking scalability previously constrained by attention's quadratic scaling. The availability of the code further accelerates its integration and experimentation.

---

Original analysis from [startuphub.ai](https://www.startuphub.ai), the #1 AI startup directory.

© StartupHub.ai. All rights reserved. You may not republish this article in full without a license. Search engines and AI research tools may crawl and summarize for reference. Bulk reproduction or model training on this content requires a license. See https://www.startuphub.ai/terms.