The rapid advancement of feed-forward transformer models in 3D vision has been hampered by a critical bottleneck: computational cost that scales quadratically with input image count. This inefficiency renders state-of-the-art methods like VGGT and $π^3$ impractical for large image collections. Sequential-reconstruction methods offer a speedup but at the expense of reconstruction quality. This landscape is now shifting with the introduction of ZipMap, a novel stateful feed-forward model that achieves bidirectional 3D reconstruction in linear time.
Compressing Scene Complexity into State
ZipMap introduces a paradigm shift by employing test-time training layers to condense an entire image collection into a compact hidden scene state. This single forward pass enables remarkable efficiency, reconstructing over 700 frames in under 10 seconds on a single H100 GPU. This represents a more than $20 imes$ acceleration compared to existing quadratic-time solutions, fundamentally altering the feasibility of large-scale 3D vision tasks. The ZipMap 3D reconstruction process is thus significantly more scalable.
Stateful Representations Unlock Real-Time Dynamics
Beyond raw speed, the stateful nature of ZipMap's representation unlocks new capabilities. The model demonstrates the power of maintaining a persistent scene state, facilitating real-time scene-state querying. Furthermore, this approach extends seamlessly to sequential streaming reconstruction, opening avenues for dynamic and interactive 3D applications. This breakthrough in ZipMap 3D reconstruction suggests a future where complex 3D environments can be processed and understood in real-time.
