Introducing NVIDIA Cosmos 3: The Open Model That Thinks, Generates, and Acts

Summary
This official launch video from NVIDIA introduces Cosmos, an open frontier omni-model designed for physical AI. Narrated over conceptual diagrams and video demonstrations, the video outlines Cosmos's architecture—a Mixture of Transformers combining an autoregressive reasoning transformer and a diffusion generator—and its applications across reasoning, synthetic data generation, simulation, and robotic policy execution.

What is shown

• Autonomous Driving Edge Cases [00:01–00:09]: Real-world driving in a Mercedes-Benz test vehicle identifying a rolling ball and a pedestrian child crossing, displaying live "Reasoning" and "Meta Actions" overlays.
• Architecture Overview [00:14–00:34]: A schematic showing Cosmos processing text, image, video, audio, and action inputs through a "Mixture of Transformers" architecture consisting of an Autoregressive Reasoner connected to a Diffusion Generator.
• World Reasoner (VLM) [00:39–00:48]: Cosmos analyzing drone timelapse footage of an urban traffic intersection to generate a structured traffic report with observations and actionable engineering insights.
• Data Generator & World Model [00:49–00:57]: Physics-accurate synthetic video generation depicting an unusual road hazard (a mattress flying off a truck on a highway).
• Simulator & OmniDreams [00:58–01:12]: Cosmos operating within simulation runtimes (AlpaSim) and NVIDIA OmniDreams as an action-conditioned world model, generating predictive sensor output for extreme scenarios (an elephant crossing a residential road, cone navigation at night, and heavy snow).
• Policy Model / World Action Model [01:13–01:29]: Integration with Alpamayo 2 Super and robotic manipulation, demonstrating multi-step tool grasping (picking up a screwdriver and placing it on a rack) with live step-by-step reasoning and motion planning.

Claims & numbers

• The narrator claims real-world physical data cannot scale on its own, asserting that "compute is data" for physical AI [00:07–00:12].
• Cosmos is described as an "open frontier omni-model for physical AI" [00:16].
• Cosmos utilizes a "Mixture of Transformers" architecture where an autoregressive transformer plans and instructs a diffusion transformer that generates downstream frames/actions [00:19–00:33].
• Cosmos serves as the underlying foundation for NVIDIA OmniDreams, an action-conditioned world model predicting future sensor outputs frame by frame [01:03–01:10].

Notable quotes

• [00:11]: "For physical AI, compute is data."
• [00:16]: "An open frontier omni-model for physical AI, built on a new Mixture of Transformers architecture."
• [01:37]: "Cosmos: the foundation for developers of the age of physical AI."

Assessment
This is a polished official marketing and architecture announcement from NVIDIA. While it showcases real video samples, simulated robotics rollouts, and software interface mockups, it is heavily produced and cut for promotional impact rather than providing live unedited developer workflows or technical benchmark disclosures.

Described by gemini-3.8-flash on 2026-09-29 from the video's audio and frames.