Grounding VLMs: VAORA's Leap in Physical AI

Vision-language models (VLMs) consistently falter in interactive physical reasoning, especially when confronted with novel tasks and unfamiliar environments. This generalization gap stems from two critical failure modes: the generation of hallucinatory chain-of-thought (CoT) reasoning that defies physical laws, and a pronounced disconnect between the model's internal reasoning and its executed actions.

Rewriting Reality: Suppressing Hallucinated CoT

The core challenge in VLM physical reasoning has been their tendency to generate CoT that contradicts the visual and physical world. arXiv introduces VAORA (Visual Action Outcome Reasoning Alignment), a novel reward design engineered to directly combat this issue. VAORA integrates a Visual Alignment Reward, which meticulously anchors VLM reasoning to the visual context, independent of the agent's immediate action. This critical component acts as a truth serum for the model's internal monologue, forcing its reasoning to align with observable reality rather than fabricated narratives. The result is a significant suppression of hallucinatory CoT, paving the way for more physically coherent reasoning.

Bridging the Gap: Reasoning-Action Alignment

Beyond internal coherence, VLMs often struggle with a fundamental misalignment between their reasoning and their subsequent actions. VAORA tackles this by incorporating a Visual-Action Alignment Reward. This complementary reward grounds the model's reasoning in the visual outcome directly induced by its action. By penalizing discrepancies between predicted outcomes and actual visual results, the VAORA reward design dramatically reduces the gap between a VLM's conceptual understanding and its behavioral execution. This dual reward structure ensures that not only is the model's reasoning sound, but its actions are also a direct, logical consequence of that reasoning, enhancing reliability in complex interactive environments.

Stabilizing Intelligence: Dense Rewards for Generalization

Training stability is paramount for robust AI, especially when dealing with nuanced physical interactions. VAORA enhances its training efficacy by employing smooth, dense rewards. This is achieved by estimating success probabilities using a pre-trained in-domain expert agent. This technique provides a continuous learning signal, which is crucial for navigating the complexities of physical reasoning tasks and improving overall training stability. Experiments conducted on the challenging PHYRE and Virtual Tool benchmarks decisively demonstrate VAORA's superior performance across novel-task and unseen-environment settings. This confirms that the VAORA reward design can indeed induce grounded and generalizable physical intelligence, marking a significant step forward for interactive AI systems.