Extreme Parkour

A 2023 CMU project where a low-cost quadruped does parkour — jumping high and far — using just one depth camera and a single network.

Extreme Parkour is work released in September 2023 by Xuxin Cheng, Deepak Pathak, and colleagues at Carnegie Mellon University, published at ICRA 2024. They used a low-cost Unitree A1 quadruped, whose motor control isn't especially precise, with just one Intel RealSense D435 depth camera on its head producing low-frequency, jittery, noisy footage. Where traditional approaches carefully design perception, mapping, planning, and control as separate modules, this work goes end-to-end: a neural network is trained with large-scale reinforcement learning in simulation to output joint actions directly from the depth image. Training happens in two stages — first using precise terrain information only available in simulation, then distilled into a policy that works from the depth image alone and can even decide which direction to jump; the reward is mostly unified into a single term, the dot product between the velocity direction and the goal direction, with no need to hand-design a reward for each type of obstacle. It pushed the difficulty ceiling for legged-robot parkour up substantially, and is often cited alongside the contemporaneous Robot Parkour Learning.

ExampleThe A1 robot dog can jump onto a 0.5-meter-tall box (about twice its own height), leap across a 0.8-meter-wide gap (about twice its body length), and even walk upside down on just its two front legs.

Also called

Extreme Parkour with Legged Robots

Related

Parkour · Robot Parkour Learning · Perceptive Locomotion · Teacher-Student Distillation · Unitree A1 · Sim-to-Real Transfer

Sources

Extreme Parkour with Legged Robots (arXiv 2309.14341)
Extreme Parkour 项目主页 (Chinese)

As of

2023-09