Google AI Teaches Humanoid Robot to Play Cornhole by Watching Human Videos
Google DeepMind's AI taught a humanoid robot to play cornhole using human videos, simulation and real-world practice.

Google DeepMind has demonstrated how AI can teach a humanoid robot a new physical skill just by watching a video of a person performing it, instead of having to rely on engineers to program every movement.
On October 13, in conjunction with the unveiling of the Gemini 4 Argon AI model from Google, the company made public a demonstration of Apptronik’s Apollo humanoid being taught the game of cornhole, where one must throw beanbags at an elevated board with an opening in it.
The robot was able to land seven straight shots once its AI had done its homework: studying human form and function, some practice in virtual space and then applying those lessons to the machine itself, Google reports.
Gemini 4 Argon set the whole thing in motion by way of a video analysis; the model would make note of the environment and figure out the body mechanics required for the throw. From there it wrote the simulation code so the robot could have a go at it virtually first.
Google describes the “Video-to-Sim-to-Real” technique as a means of saving engineers the hours of manual labour that goes into building simulations and robotic programmes; they claim it can cut development time from weeks to a matter of hours.
There were no instant successes, though. Norman Di Palo of Google DeepMind, who was involved in the experiment, took to social media to say that after the robot’s first efforts, more training was called for on the physical unit. The seven-shot run came only after some tweaking, a testament to the value of feedback from the real world.
It is an exercise in the kind of embodied reasoning Google is after in its AI systems. On the ERQA benchmark, which puts to the test a system’s grasp of spatial and physical conundrums, Gemini 4 Argon scored 87%, though that is a measure of reasoning, not of a robot’s aptitude in the field.
To that end, Apptronik, the Texas firm that makes Apollo, has been in collaboration with Google DeepMind, making use of large-scale training and data from the field to further humanoid intelligence. In June, the Austin developer put in an expansion at its Robot Park so the Apollo robots can amass data from their activities for future AI deployments.
Cornhole is hardly an industrial application but the method has its uses. Should a robot be able to pick up a physical skill off a video, it may be possible for developers to put them to work in all manner of things without having to programme every move.
For now, these are internal results and Google has yet to determine if the approach holds up in other settings or when the task is more complex. Yet it is indicative of the part that observation and simulation will come to play in schooling a humanoid.
More articles on this topic
278 articles
ReportWhat it Really Costs to Own a Personal Assistant Robot at Home
ReportHumyn Labs Raises $47 Million to Build Real-World Training Data for Robotics
ReportBoston Dynamics Selects GlobalFoundries as First Semiconductor Customer for Atlas Humanoid Trials
ReportGenesis AI Raises $500 Million at a $3.5 Billion Valuation to Scale General-Purpose Robots
ReportMultiway Robotics Showcases Intralogistics Solutions and Wheeled Humanoid at CeMAT ASIA
ReportFORT Robotics Files Registration Statement for Public Listing Partnership
ReportRobot Baristas are Learning to Reproduce the Art of Professional Coffee Making
ReportTARSBOT and Aptiv Plan to Deploy 100 Industrial Robots Across 33 Factory Applications
ReportSouth Korea Proposes $39 Million for Robot Actuators, Humanoid Batteries and AI Sensors
ReportLoongson Completes ROS 2 Integration on Its Domestic Computing-Powered Intelligent Robot Vehicle
ReportBoston Dynamics Puts Wheels on Atlas Humanoid to Test Reinforcement Learning and Motor Limits
ReportAnt Group Open-Sources LingBot-Touch to Give Robots a Shared Sense of TouchBUSINESS NEWS WEEKLY LETTER
The Weekly Letter for Robotics Professionals, Summarizing the Most Important Industry Moves, Launches, Deals and Signals.