ELSA3D: Structured Text-3D Reasoning

Unified 3D foundation models promise to streamline 3D asset generation and understanding, but current approaches struggle with implicit text-3D interaction. Existing methods often flatten text and 3D tokens, leading to a loss of structural cues and fine geometric detail. This paper introduces ELSA3D, a novel unified 3D model designed to address this by structuring language and geometric reasoning through matched abstraction scales.

Visual TL;DR. 3D Model Limitations solves ELSA3D Introduced. ELSA3D Introduced uses Elastic Semantic Anchoring. Elastic Semantic Anchoring employs Anchor Tokens. Anchor Tokens enables Structured Reasoning. Elastic Semantic Anchoring leads to SOTA Performance. SOTA Performance and Efficiency Gains.

3D Model Limitations: current methods flatten text and 3D tokens losing structural cues
ELSA3D Introduced: novel unified 3D model for structured language and geometry
Elastic Semantic Anchoring: precise text-3D interaction via scale-aware octree and anchor tokens
Anchor Tokens: sparse cross-modal units for semantic cue selection and routing
Structured Reasoning: matched abstraction scales for language and geometric reasoning
SOTA Performance: revolutionizes 3D generation and captioning with precise alignment
Efficiency Gains: halving computational costs for unified 3D models

Visual TL;DRQuickExplainDeeper

!-- /sh-diagram -->

Elastic Semantic Anchoring for Precise Cross-Modal Alignment

ELSA3D employs an 'elastic semantic anchoring' strategy to enable precise and efficient interaction between text and 3D representations. It utilizes a scale-aware octree tokenizer for geometry and introduces Anchor Tokens. These sparse, cross-modal units are crucial for selecting semantic cues, routing them to the appropriate 3D abstraction scale, retrieving relevant geometric evidence, and then integrating this fused signal back into the unified representation. This approach ensures that interaction remains sparse yet highly accurate, avoiding the information collapse seen in previous flat-sequence methods.

Optimized Reasoning with Lightweight Routing

A key innovation in ELSA3D is its lightweight per-block router. This component dynamically selects which text tokens instantiate anchors at which geometric scales. By concentrating cross-modal capacity only where alignment is most critical, ELSA3D achieves remarkable computational efficiency. This elastic computation and reasoning mechanism allows the model to significantly reduce FLOPs and inference latency, roughly halving them compared to non-elastic versions of similar architectures, while simultaneously pushing the boundaries of performance.

State-of-the-Art Performance and Efficiency Gains

The effectiveness of ELSA3D is demonstrated through its state-of-the-art performance across critical benchmarks, including image-to-3D generation, text-to-3D generation, and 3D captioning. The model not only surpasses the strongest unified baselines but does so with substantial gains in efficiency. This dual achievement of superior performance and reduced computational overhead positions ELSA3D as a significant advancement for researchers and investors in the 3D AI space, offering a more practical and scalable path forward.