Motion Policy Networks

A neural motion planner that generates collision-free robot-arm motion directly from a single depth camera's point cloud.

Motion Policy Networks was released in October 2022 by Adam Fishman, Byron Boots, Dieter Fox, and colleagues at the University of Washington and NVIDIA, published at CoRL 2022. Classical motion planning (such as sampling-based planning like RRT) needs a complete, accurate model of the environment and is slow to compute in cluttered scenes. MπNets replaces this with an end-to-end neural network: given the point cloud a single depth camera sees of the scene plus the robot's current state, it outputs joint motion toward the target pose step by step, which strung together forms a smooth, obstacle-avoiding trajectory. All training data is generated automatically in simulation using classical planning tools (OMPL and Geometric Fabrics), covering more than 500,000 environments and more than 3 million planning problems. The result beats prior neural planners by 46%, is much faster than a global planner, can handle dynamic scenes, and, even trained only on simulation data, transfers to noisy partial point clouds on a real robot. Code, weights, and data are open-source.

ExampleTo reach into one compartment of a cabinet to grab something, MπNets just looks at the depth camera's point cloud once and gives a real-time joint trajectory that routes around the cabinet's panels, with no need to first build a complete map and then run a planner.

Also called

MπNets, MPiNets

Related

Neural Motion Planning · Motion Planning · Point Cloud · Obstacle Avoidance · Sampling-Based Planning · Geometric Fabrics

Sources

Motion Policy Networks (arXiv 2210.12209)
MπNets project page
NVlabs/motion-policy-networks (GitHub)

As of

2022-10