Unifying Audio: The Rise of the Real-Time LALM

The current generation of Large Audio Language Models (LALMs) operates in discrete, offline modes, handling single tasks like ASR or voice chat in isolation. This fragmented approach fails to capture the inherently interactive and continuous nature of audio. A significant leap forward is proposed by the researchers, who introduce the concept of an 'always-on' LALM capable of real-time perception, decision-making, and response.

The Audio Interaction Model: Perceive, Decide, Respond in Real-Time

This paradigm shift is formalized as the Audio Interaction Model. It envisions a unified streaming architecture that integrates offline task performance with online, general audio instruction following. Crucially, this model can discern the semantics of a continuous audio stream to decide precisely when to interject or respond, moving beyond simple turn-based interactions. This capability is realized through a novel model called Audio-Interaction, which maintains offline task execution while enabling dynamic, real-time audio understanding and engagement.

SoundFlow: A Streaming-Native Framework for Real-Time Audio

To operationalize the Audio Interaction Model, the authors propose SoundFlow, a comprehensive framework designed for end-to-end streaming audio processing. SoundFlow addresses the entire pipeline from data construction to training and deployment. Key innovations include streaming-native data construction, comprehension-aware training methodologies, and asynchronous, low-latency inference mechanisms. This ensures stable, real-time interaction essential for applications requiring immediate audio comprehension and reaction.

StreamAudio-2M and Proactive-Sound-Bench: Enabling New Audio Capabilities

The practical advancement of real-time audio interaction is underpinned by new data and evaluation tools. The researchers have constructed StreamAudio-2M, a substantial 2.6 million-item streaming corpus covering seven fundamental audio abilities and 28 sub-tasks. Complementing this is Proactive-Sound-Bench, a benchmark specifically designed to assess proactive audio intervention capabilities. Experiments across eight benchmarks demonstrate that Audio-Interaction not only achieves competitive performance on conventional audio tasks but also unlocks novel functionalities, such as real-time ASR and proactive assistance, previously unattainable with offline LALMs.