# CLAP cross-embodiment action-conditioned video generation _CLAP trains action-conditioned video world models across human and robot video and matches single-embodiment models on DROID with zero-shot generalization._ **Published:** 2026-08-28 **Source:** https://www.startuphub.ai/ai-news/ai-research/2026/clap-cross-embodiment-action-conditioned-video-generation --- Single-embodiment video world models cannot scale. [CLAP](https://arxiv.org/abs/2608.27406v1) proposes CLAP cross-embodiment action-conditioned video generation to train on internet-scale human and robot video instead. Embodiment lock-inDriver single-robot video models cannot use heterogeneous data containing rich physics signalsFrom the articleAccording to arXiv, the work from Kechen Liu and Ola Shorinwa treats universal physical laws as the common substrate across embodiments.CLAP frameworkCoreFrom the article 5 mentionsCLAP proposes CLAP cross-embodiment action-conditioned video generation to train on internet-scale human and robot video instead.DROID zero-shot matchOutcomematches single-embodiment baseline performance on DROID with zero-shot generalization across platformsusesLatent actions channelContextinferred action representations for unlabeled human video, completing the unified three-channel designFrom the article 2 mentionsCLAP reconciles this with three channels: end-effector poses, language instructions, and latent actions.Embodiment lock-inDriversingle-robot video models cannot use heterogeneous data containing rich physics signalsFrom the articleAccording to arXiv, the work from Kechen Liu and Ola Shorinwa treats universal physical laws as the common substrate across embodiments.motivatesUniversal physics substrateContextspatiotemporal dynamics obey the same laws regardless of whether the actor is human or robotFrom the articleAccording to arXiv, the work from Kechen Liu and Ola Shorinwa treats universal physical laws as the common substrate across embodiments.inspiresCLAP frameworkCoreFrom the article 5 mentionsCLAP proposes CLAP cross-embodiment action-conditioned video generation to train on internet-scale human and robot video instead.usesEnd-effector poses channelContextprecise robot action control, but limited to embodiment-specific kinematic structures and measurementsFrom the articleCLAP reconciles this with three channels: end-effector poses, language instructions, and latent actions.Language instructions channelContextnatural language goals bridging human and robot videos, but lacks fine-grained motor specificationFrom the articleCLAP reconciles this with three channels: end-effector poses, language instructions, and latent actions.Latent actions channelContextinferred action representations for unlabeled human video, completing the unified three-channel designFrom the article 2 mentionsCLAP reconciles this with three channels: end-effector poses, language instructions, and latent actions.Unlabeled video curriculumCoretraining schedule turns raw internet human video into implicit physics priors without manual annotationFrom the articleIt first learns foundational physical priors across unlabeled video data using latent actions.producesDROID zero-shot matchOutcomematches single-embodiment baseline performance on DROID with zero-shot generalization across platforms According to [arXiv](https://arxiv.org/abs/2608.27406v1), the work from Kechen Liu and Ola Shorinwa treats universal physical laws as the common substrate across embodiments. ## Why Embodiment Lock-In Breaks World Models State-of-the-art action-conditioned video models are tied to one robot. That prevents them from using heterogeneous video that already contains rich physics signals. CLAP is built on the observation that spatiotemporal dynamics obey the same laws regardless of the actor, whether human or robot. ## How CLAP Unifies Three Incompatible Action Spaces Cross-embodiment learning fails because action representations do not align and human video has no action labels at all. CLAP reconciles this with three channels: end-effector poses, language instructions, and latent actions. Each has a clear limitation alone. Poses are precise but embodiment specific. Language is universal but underspecified. Latent actions are scalable but ungrounded. Together they cover the tradeoff. ## The Curriculum That Turns Unlabeled Video Into Physics Priors CLAP introduces a curriculum-based recipe to resolve those limitations. It first learns foundational physical priors across unlabeled video data using latent actions. It then grounds those priors in end-effector action spaces for zero-shot deployment to real tasks. This two-stage design lets the model absorb internet-scale dynamics without requiring action labels everywhere. ## So What: From Generalist Pretraining to Specialist Supremacy CLAP approaches or surpasses state-of-the-art single-embodiment video models on challenging environments like DROID. The advantage compounds with few-shot adaptation, establishing a new paradigm for training single-embodiment video world models. The suite spans end-effector, language, and latent conditioning across DROID, Bridge, bimanual YAM robots and G1 humanoids, with all code and models open-sourced. For founders and investors, this inverts the standard world-model bet. Instead of collecting more DROID or Bridge data to make a better DROID model, build a cross-embodiment generalist on cheap human video and few-shot it down. It mirrors what happened in language and vision: scale on heterogeneous data, then specialize. Robotics world models are now following the same curve. --- Original analysis from [startuphub.ai](https://www.startuphub.ai), the #1 AI startup directory.