# AdaCodec: Efficient Video MLLM Encoding
_AdaCodec revolutionizes video MLLMs by using predictive visual coding to drastically cut tokenization costs and latency, achieving superior performance at a fraction of the budget._
**Published:** 2026-06-02
**Source:** https://www.startuphub.ai/ai-news/ai-research/2026/adacodec-efficient-video-mllm-encoding
---
The inherent temporal redundancy in video, where adjacent frames largely overlap, presents a fundamental inefficiency for current video multimodal large language models (video MLLMs). These models typically process each sampled frame as an independent image, leading to redundant visual tokens and inflated computational costs. A new approach, detailed on [arXiv](https://arxiv.org/abs/2606.02569v1), challenges this paradigm by proposing a more dynamic and efficient video interface.

Video MLLM InefficiencyDriver

processing adjacent frames as independent images leads to redundant tokensFrom the articleThe inherent temporal redundancy in video, where adjacent frames largely overlap, presents a fundamental inefficiency for current video multimodal large language models (video MLLMs).problemTemporal RedundancyContextadjacent video frames largely overlap, causing inflated computational costsFrom the articleThe inherent temporal redundancy in video, where adjacent frames largely overlap, presents a fundamental inefficiency for current video multimodal large language models (video MLLMs).AdaCodec IntroducedCorea new dynamic and efficient video interface for MLLMsFrom the article 3 mentionsInstead of encoding every frame fully, this system, instantiated as AdaCodec, selectively transmits a full reference frame only when scene prediction is unreliable.usesPredictive Visual CodingCoreintelligently manages visual token transmission based on scene predictionFrom the articleThe core innovation lies in a 'predictive visual code' that intelligently manages visual token transmission.generatesSelective Frame EncodingContextFrom the articleInstead of encoding every frame fully, this system, instantiated as AdaCodec, selectively transmits a full reference frame only when scene prediction is unreliable.Compact P-tokensCoreFrom the articleOtherwise, it encodes inter-frame changes, encompassing motion and prediction residuals, using compact 'P-tokens'.contributes toReduced Token CountEffectsignificantly minimizes visual tokens required for video understandingFrom the articleEven at a drastically reduced token budget (1/7th), AdaCodec with 32k tokens outperforms the 224k baseline on all long-video benchmarks.Efficiency GainsOutcomedrastically cuts tokenization costs and latency for video MLLMsFrom the articleThis efficiency leap makes real-time video analysis and interaction far more feasible.Superior PerformanceOutcomeachieves better results at a fraction of the computational budget

## Predictive Visual Coding for Reduced Redundancy

The core innovation lies in a 'predictive visual code' that intelligently manages visual token transmission. Instead of encoding every frame fully, this system, instantiated as AdaCodec, selectively transmits a full reference frame only when scene prediction is unreliable. Otherwise, it encodes inter-frame changes, encompassing motion and prediction residuals, using compact 'P-tokens'. This adaptive strategy significantly minimizes the number of visual tokens required for video understanding.

## Substantial Gains in Efficiency and Performance

AdaCodec demonstrates marked improvements over the baseline [Qwen3-VL-8B](/ai-news/ai-research/2026/beyond-rgb-grounding-vision-language-on-raw-sensor-data) model across eleven benchmarks. Even at a drastically reduced token budget (1/7th), AdaCodec with 32k tokens outperforms the 224k baseline on all long-video benchmarks. Furthermore, for general-video benchmarks, it not only elevates average scores but also slashes the time-to-first-token from 9.26s to a mere 1.62s. This efficiency leap makes real-time video analysis and interaction far more feasible.
---
Original analysis from [startuphub.ai](https://www.startuphub.ai), the #1 AI startup directory.