Humanoid Parkour Learning

An end-to-end parkour policy that lets a humanoid jump onto platforms, cross gaps, and clear hurdles using just its head depth camera.

This was released in June 2024 by Ziwen Zhuang, Shenzhe Yao, and Hang Zhao at the Shanghai Qi Zhi Institute, ShanghaiTech University, and Tsinghua University, published at CoRL 2024, and is the humanoid counterpart of the same group's quadruped work, Robot Parkour Learning. It needs no human motion reference at all, training a vision-based, end-to-end whole-body control policy with reinforcement learning: first, flat-ground walking is trained on terrain with fractal noise (the noise naturally encourages the robot to lift its feet), then a “privileged” parkour policy that reads ground-truth terrain directly is trained across 10 obstacle types, and finally that's distilled with DAgger into a student policy that only sees images from a head-mounted depth camera. Deployed on a Unitree H1, it can jump onto a 0.42 m platform, cross a 0.8 m gap, run outdoors at 1.8 m/s, and autonomously choose which parkour move to use when given only a steering command.

ExampleThe operator only uses a joystick to steer; when the H1 sees a platform ahead, it decides on its own to jump up onto it.

Related

Parkour · Perceptive Locomotion · Teacher-Student Distillation · DAgger · Unitree H1 · Robot Parkour Learning

Sources

Humanoid Parkour Learning (arXiv 2406.10759)
Humanoid Parkour Learning project page

As of

2024-09