GNM

A general visual-navigation model trained on mixed data from 6 robot types that can drive different robots.

GNM was proposed in October 2022 by Sergey Levine's group at UC Berkeley (Dhruv Shah and others), published at ICRA 2023. It pools about 60 hours of navigation data from 6 robot types — TurtleBot2, Jackal, Spot, an RC car, an all-terrain vehicle, and more — to train a single image-goal navigation policy: given the current image, a few past frames (used to infer embodiment context, i.e., “which kind of robot am I”), and a goal image, it outputs the time-distance to the goal and the next 5 normalized waypoints. A normalized action space is what lets it work across different robots. The finding is that a single policy trained on this heterogeneous data outperforms any policy trained on a single dataset, and it can even deploy directly on robots absent from the training set, such as a quadrotor drone. Follow-up work includes ViNT and NoMaD.

ExampleAt deployment, a topological map made of images along a route is built first; GNM estimates how far the current view is from each node, and Dijkstra's algorithm then plans a sequence of sub-goals for the robot to navigate to one by one.

Also called

General Navigation Model

Related

ViNT · NoMaD · Cross-Embodiment · Image-Goal Navigation · Topological Map · Berkeley Artificial Intelligence Research

Sources

GNM: A General Navigation Model to Drive Any Robot (arXiv:2210.03370)
GNM 项目主页 (Chinese)

As of

2023-05