ANYmal RL Locomotion Series

Three Science Robotics papers (2019, 2020, 2022) from ETH Zurich that took simulation-trained RL locomotion out into the real wild on the ANYmal quadruped.

This refers to three papers published in Science Robotics by ETH Zurich's Robotic Systems Lab (Marco Hutter's group), all using the ANYmal quadruped as their platform. In 2019, Hwangbo and colleagues introduced actuator networks: a neural network trained on real-robot data to simulate motor response, plugged into the simulator to shrink the sim-to-real gap, letting a policy trained purely in simulation run directly on the real robot — 25% faster than the previous record, and able to get back up after falling. In 2020, Lee and colleagues used teacher-student distillation: a teacher is trained with access to privileged information such as ground-truth terrain, and a student that uses only proprioception learns to imitate it, combined with a terrain curriculum, so the robot can walk over mud, snow, and rubble without seeing the terrain at all (blind locomotion); this controller was used in the DARPA Subterranean Challenge. In 2022, Miki and colleagues used an attention-based recurrent encoder to fuse a height map with proprioception, automatically leaning more on proprioception whenever vision becomes unreliable.

ExampleThe 2022 perceptive-locomotion controller let an ANYmal complete a 2.2-kilometer hiking route rated “difficult,” with 120 meters of elevation gain, on Switzerland's Mount Etzel in 78 minutes — almost exactly the 76 minutes a hiking-planning tool would recommend for a person — stopping only once to repair a dislodged foot cover and swap batteries.

Also called

Learning Agile and Dynamic Motor Skills for Legged Robots, Learning Quadrupedal Locomotion over Challenging Terrain, Learning Robust Perceptive Locomotion for Quadrupedal Robots in the Wild

Related

RL-based Locomotion Control · Actuator Modeling (Actuator Network) · Teacher-Student Distillation · Privileged Information · Terrain Curriculum · ANYbotics ANYmal

Sources

Learning agile and dynamic motor skills for legged robots (Hwangbo et al., Science Robotics 2019, arXiv 1901.08652)
Learning Quadrupedal Locomotion over Challenging Terrain (Lee et al., Science Robotics 2020, arXiv 2010.11251)
Learning robust perceptive locomotion for quadrupedal robots in the wild (Miki et al., Science Robotics 2022, arXiv 2201.08117)

As of

2022-01