ABot-World-0: Real-time Video World Models

The challenge of creating AI agents capable of understanding and interacting with complex visual environments in real-time, over extended periods, has been a significant hurdle. Traditional approaches often struggle with maintaining coherence and controllability as interactions lengthen. The introduction of ABot-World-0 marks a substantial step forward in addressing this by presenting an action-conditioned video world model engineered for long-horizon closed-loop interaction.

Unifying Diverse World Dynamics for Controllable Agents

ABot-World-0 leverages a sophisticated multi-source data infrastructure that integrates data from AAA games, simulation engines, and internet videos. This diverse dataset is crucial for learning robust and controllable world dynamics. The system employs WorldExplorer for agent-driven data collection, guided by training feedback, and a rigorous unified pipeline featuring deterministic quality checks and VLM-based assessments to ensure data integrity and annotation quality. This comprehensive data strategy underpins the agent's ability to understand and navigate complex environments.

Distillation and Long-Horizon Alignment for Coherent Interaction

A core innovation lies in the progressive distillation process. A bidirectional action-conditioned teacher model is distilled into a causal student model using techniques like teacher forcing and ODE distillation. To combat the accumulated distribution shift and autoregressive drift inherent in long rollouts, the paper introduces LongForcing. This method aligns extended student self-rollouts with the teacher's horizon, significantly improving coherence and controllability over long interaction sequences. The use of raw keyboard actions as a unified control interface and reference-character memory for identity consistency further enhances the agent's interactive capabilities.

An Optimized Inference Stack for Real-Time Performance

Deployment of such a powerful world model requires an efficient inference stack. The researchers co-designed a streaming inference solution that includes a lightweight VAE decoder, efficient attention mechanisms, memory-aware scheduling, and low-bit DiT inference. This optimization allows ABot-World-0 to stream 720P video at up to 16 FPS on a single NVIDIA RTX 5090 desktop GPU, with a remarkably low 1.2s action-to-first-frame latency and approximately 19GiB peak VRAM. These performance metrics demonstrate the practical viability of deploying sophisticated AI agents in real-time interactive scenarios.