ELSA3D: Structured Text-3D Reasoning

Unified 3D foundation models promise to streamline 3D asset generation and understanding, but current approaches struggle with implicit text-3D interaction. Existing methods often flatten text and 3D tokens, leading to a loss of structural cues and fine geometric detail. This paper introduces ELSA3D, a novel unified 3D model designed to address this by structuring language and geometric reasoning through matched abstraction scales.

ELSA3D employs an 'elastic semantic anchoring' strategy to enable precise and efficient interaction between text and 3D representations. It utilizes a scale-aware octree tokenizer for geometry and introduces Anchor Tokens. These sparse, cross-modal units are crucial for selecting semantic cues, routing them to the appropriate 3D abstraction scale, retrieving relevant geometric evidence, and then integrating this fused signal back into the unified representation. This approach ensures that interaction remains sparse yet highly accurate, avoiding the information collapse seen in previous flat-sequence methods.

Optimized Reasoning with Lightweight Routing

A key innovation in ELSA3D is its lightweight per-block router. This component dynamically selects which text tokens instantiate anchors at which geometric scales. By concentrating cross-modal capacity only where alignment is most critical, ELSA3D achieves remarkable computational efficiency. This elastic computation and reasoning mechanism allows the model to significantly reduce FLOPs and inference latency, roughly halving them compared to non-elastic versions of similar architectures, while simultaneously pushing the boundaries of performance.

State-of-the-Art Performance and Efficiency Gains

The effectiveness of ELSA3D is demonstrated through its state-of-the-art performance across critical benchmarks, including image-to-3D generation, text-to-3D generation, and 3D captioning. The model not only surpasses the strongest unified baselines but does so with substantial gains in efficiency. This dual achievement of superior performance and reduced computational overhead positions ELSA3D as a significant advancement for researchers and investors in the 3D AI space, offering a more practical and scalable path forward.