Robot Parkour Learning

A parkour policy that lets a low-cost quadruped robot dog climb, jump, crawl, and squeeze through obstacles using only a depth camera.

Robot Parkour Learning was released by the Shanghai Qi Zhi Institute, Tsinghua University, Stanford, CMU, ShanghaiTech, and other institutions in September 2023, receiving an oral presentation at CoRL 2023 and reaching the finals for the Best Systems Paper Award. It teaches low-cost quadruped robots like Unitree's A1 and Go1 to climb high obstacles, leap wide gaps, crawl under low barriers, squeeze through narrow gaps, and run. The method has three steps: first, reinforcement learning pretraining in simulation, where the robot is allowed to 'pass through' obstacles, treating physics violations as only a soft penalty to encourage exploration; then fine-tuning each individual skill with full physical constraints restored; finally, using DAgger to distill the several specialized skills into a single vision-based policy that relies only on an onboard depth camera. Once deployed on a real robot, it can choose the appropriate action for the obstacle in front of it on its own, without relying on animal motion references.

ExampleA robot dog encounters a low horizontal bar; based on its depth image, the policy automatically chooses to lower its body and crawl under it rather than trying to jump over it.

Also called

Quadruped Robot Parkour Learning

Related

Parkour · Perceptive Locomotion · Teacher-Student Distillation · DAgger · Sim-to-Real Transfer · Extreme Parkour

Sources

arXiv 2309.05665: Robot Parkour Learning
Robot Parkour Learning 项目主页 (Chinese)

As of

2023-11