Marionette: Decoupling Appearance from State

Current interactive game world models struggle with long-term consistency. By autoregressively generating visual observations directly, they implicitly encode structured properties like pose and geometry. This leads to error accumulation over time, compromising controllability and coherence. The Marionette world model introduces a novel paradigm: explicitly modeling the evolving world state and delegating exact geometric computation to a fixed, zero-parameter renderer.

Separating Structure from Synthesis

Marionette operates in three stages to achieve this separation. First, a two-stage autoregressive dynamics model predicts an explicit, interpretable 276-dimensional 3D world state. This state encapsulates multi-entity articulated skeletons, metric root trajectories, and rotations. Second, a graphics bridge, requiring zero parameters, translates this predicted state into pose-control videos. Crucially, this bridge computes world-space geometry and occlusion in closed form, offering exactness without neural computation. Finally, a control-conditioned video-diffusion observation model synthesizes photorealistic RGB observations based on these structured controls.

Controllability and Repairability in the State Space

The core innovation lies in the explicit world state, which proves directly controllable. Experiments demonstrate that forcing a mismatched action stream alters root-aligned joint error by 31% across held-out segments, validating the state's responsiveness. Furthermore, long-horizon behaviors are determined and can be repaired within this state representation. Left unconstrained, generated characters drifted apart significantly, and a third of frames exhibited ground penetration. However, by imposing simple rules on the explicit state, such as a terrain collider and a separation cap, ground penetration was reduced by 66%, and character engagement was maintained. These improvements were achieved without any modifications to the observation model, highlighting the power of state-level control.

Uncompromised Fidelity, Enhanced Utility

Routing appearance through the predicted state incurs no detectable loss in fidelity. The system achieves a Frame-to-Frame Video Diffusion (FVD) score of 831, a marginal increase compared to the 799 FVD for recorded pose, suggesting photorealism is preserved. This architectural choice provides the significant benefit of a controllable and repairable world model, without sacrificing the visual quality expected from generative approaches. The Marionette world model offers a promising direction for building more robust and predictable interactive AI systems.