Meet Cosmos 3: Our Latest Frontier Model for Physical AI

Summary
Ming-Yu Liu, Vice President of Cosmos Lab at NVIDIA, announces and details the release of Cosmos 3, NVIDIA's foundation model for physical AI. He explains that Cosmos 3 unifies prediction, transfer, physical reasoning, and policy generation into a single "omni" model architecture available in two sizes: Nano and Super.

What is shown

• [00:00] Ming-Yu Liu introduces Cosmos 3 from NVIDIA.
• [00:09] Visual recap of previous Cosmos components: robotic arm tea/powder preparation (Cosmos Predict), simulation-to-real domain transfer (Cosmos Transfer), drone inspection of wind turbines with text Q&A reasoning (Cosmos Reason), and tabletop manipulation ("put purple eggplant on plate", "put brown chicken wing on plate") (Cosmos Policy).
• [00:32] Diagram of the Omni model interface handling text, image, video, audio, and action for both inputs and outputs.
• [00:40] Architecture diagram detailing the "Mixture-of-Transformer" framework featuring an autoregressive Reasoner tower and a diffusion Generator tower sharing multimodal attention.
• [01:08] Physical AI downstream robotics and autonomous driving clips, including dual-arm manipulation, tool sorting, race car telemetry, and night driving lane prediction.
• [01:28] Robotic bread-toasting demo evaluating next-best action and generating step-by-step reasoning tokens.
• [01:48] Leaderboard benchmark tables shown: VANTAGE-Bench, Traffic Anomaly Reasoning (TAR), PAI-Bench (Physical AI Bench), R-Bench, RoboLab-120 Overall, and Artificial Analysis Image-to-Video Leaderboard.
• [03:03] Announcement of open availability via Hugging Face and GitHub.

Claims & numbers

• The presenter claims Cosmos 3 is NVIDIA's strongest and most versatile model built to date, unifying previous discrete models into a single architecture.
• The model is released in two sizes: the smaller Nano model (tailored for edge device deployment) and the Super model (optimized for high accuracy in physical AI tasks).
• The architecture is a novel "Mixture-of-Transformer" with two towers: an autoregressive tower and a diffusion tower.
• Benchmark claims highlighted:
  • Ranked #1 on reasoning benchmarks including VANTAGE-Bench and TAR (Traffic Anomaly Reasoning).
  • Top performance on generation benchmarks including PAI-Bench and R-Bench.
  • Ranked #1 in RoboLab (RoboLab-120) for policy evaluation (Cosmos Nano-Policy shown at top with 476/1200, score 73.1).
  • Ranked #1 for open-source models on the Artificial Analysis Image to Video Leaderboard (Cosmos3-Super-Image2Video shown with 1,212 ELO).
• The presenter states Cosmos 3 is open, with weights available on Hugging Face, code examples on GitHub, and training scripts and datasets provided.

Notable quotes

• [00:24] "In Cosmos 3, we bring all of them together in a single model. The latest Cosmos 3 model is the Omni model."
• [00:40] "And it's based on a novel architecture called Mixture-of-transformer, where you have two towers. The left tower runs autoregressive, the right tower runs diffusion."
• [02:43] "At NVIDIA, we want to help accelerate the physical AI revolution. We are doing our part to build high quality, open physical AI foundation models to unlock all the developers."

Assessment
This is an official NVIDIA product launch presentation featuring an executive walkthrough accompanied by motion graphics, benchmark tables, and pre-recorded robotics/driving test clips. While the performance metrics are backed by standard third-party and community benchmark leaderboards, the robot clips and simulations are curated highlight reels rather than unedited live interactive demonstrations.

Described by gemini-3.8-flash on 2026-09-29 from the video's audio and frames.