OP3 Soccer (DeepMind)

DeepMind uses deep reinforcement learning to teach the small humanoid robot OP3 to play one-on-one soccer.

This is work from Google DeepMind, posted to arXiv in April 2023 and published in Science Robotics in April 2024. The robot is Robotis's OP3, a low-cost small humanoid with 20 driven joints, and the task is simplified one-on-one soccer. Training happens entirely in MuJoCo simulation: skills like getting up and shooting are trained separately first, then distilled into a single policy, which keeps improving by playing against its own past versions (self-play). A relatively high control frequency, targeted dynamics randomization, and disturbances applied during training let the policy transfer to the real robot zero-shot. Compared to a scripted controller, it walks 181% faster, turns 302% faster, gets up 63% quicker, and kicks 34% faster. It's an early representative example of deep reinforcement learning producing agile, whole-body motion on a bipedal robot.

ExampleDuring a match, the robot anticipates where the ball will go and turns sideways to block an opponent's shot; if knocked down, it gets back up quickly and keeps chasing the ball.

Also called

Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learning

Related

Reinforcement Learning · Sim-to-Real Transfer · Domain Randomization · Self-Play · Policy Distillation · Google DeepMind

Sources

Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learning (arXiv 2304.13653)
OP3 Soccer 项目页 (Chinese)

As of

2024-04