Teaching a humanoid robot to play cornhole with Gemini 4 Argon

Summary
Google DeepMind research engineer Rohan Thakker presents an experiment testing whether Gemini 4 Argon can control an Apptronik Apollo humanoid robot to learn and execute playing cornhole. The model observes human demonstrations via the robot’s cameras, models and practices throws inside a MuJoCo physics simulator, and iteratively refines its real-world throwing technique from visual feedback.

What is shown

• [00:00 - 00:05] Apollo humanoid robot throwing a beanbag cleanly into the cornhole board; title overlay: “Gemini 4 Argon x Robotics”.
• [00:05 - 00:08] Introduction by Rohan Thakker, Research Engineer at Google DeepMind.
• [00:08 - 00:12] Web prompt UI (Apollo real2sim Cornhole, running Gemini 4 Argon) prompting the robot to record a demonstration using its left eye camera.
• [00:13 - 00:16] A researcher tossing a beanbag while the robot watches.
• [00:17 - 00:20] Side-by-side comparison of Apollo 2’s left eye camera feed and the MuJoCo simulator mirroring the motion.
• [00:21 - 00:30] Initial trial throws on hardware, showing misses alongside telemetry and code logs of joint positions, velocities, and error analysis.
• [00:31 - 00:36] Successful beanbag throw directly into the target hole; engineers cheer.
• [00:37 - 00:41] Apollo celebrates with arm gestures and speech.
• [00:42 - 00:46] User prompt requesting a 20-second summary of the methodology with accompanying gestures in sim.
• [00:47 - 01:25] Apollo articulates its 4-step approach (motion mapping from video, stereo camera 3D estimation & physics setup, simulation tuning including widening stance to 52 cm for a 7.2 cm balance margin, and real-world iterative correction).
• [01:26 - 01:36] Thakker detailing Gemini 4 Argon’s multimodal reasoning, error diagnosis from video, and experiential learning.
• [01:37 - 01:43] Montage of direct cornhole shots and the closing Gemini 4 Argon title card.

Claims & numbers

• The presenter claims they tested whether Gemini 4 could control a humanoid robot to solve a dynamic physical task (cornhole).
• In simulation, the model widened the robot’s stance to 52 cm to boost its balance margin to 7.2 cm (shown at [01:07]).
• Apollo claims it adjusted its release on the physical robot by a few centimeters until sinking 7 consecutive shots straight into the hole.
• The presenter claims Gemini 4 can autonomously reason over video footage of failures to diagnose mechanical errors and improve control policies iteratively.

Notable quotes

• [00:05] “In this experiment we’re basically trying to see if we can teach Gemini 4 to control a humanoid robot to solve cornhole.” — Rohan Thakker
• [00:38] “Woohoo! Airmail, straight in the hole. Let’s go!” — Apollo / Gemini 4
• [01:20] “...and adjusted my release by a few centimeters until we sank seven shots straight in the hole.” — Apollo / Gemini 4

Assessment
This is an official demonstration video produced by Google DeepMind showcasing Gemini 4 Argon’s multimodal reasoning in a real-to-sim-to-real robotics framework. The footage showcases real robot hardware and simulation environments, though the presentation is polished and time-condensed with cuts across the trial and error iterations.

Described by gemini-3.8-flash on 2026-10-13 from the video’s audio and frames.