Point Cloud Completion / Shape Completion

Given a partial point cloud, inferring and filling in the parts of an object that were occluded or never scanned.

A camera or lidar only sees the side of an object facing it, so the resulting point cloud — a set of points with 3D coordinates — is always incomplete. Point cloud completion takes such a partial point cloud as input and produces the full shape; when the output is a point cloud this is usually called point cloud completion, and when the output is a mesh or voxel grid it is often called shape completion. PCN, from Carnegie Mellon in 2018, was an early deep network that completed shapes by operating directly on point sets, first generating a coarse shape and then progressively densifying and refining it. For robots, completion lets a system estimate the geometry of an object’s hidden back side, making grasp planning more reliable: Varley et al. used a 3D convolutional network in 2016 to complete a single-view point cloud and plan grasps from it, verified on a real robot. Evaluation commonly uses Chamfer distance, the average nearest-point distance between two point sets.

ExampleA wrist camera sees only the front of a mug; a completion network infers the shape of its back side and handle, and the grasp planner uses that to choose where to grip.

Also called

Point Cloud Completion, Shape Completion, 3D Shape Completion

Related

Point Cloud · Occlusion · Single-Image 3D Reconstruction · Chamfer Distance · Grasp Planning · Point Cloud Encoder

Sources

PCN: Point Completion Network (arXiv 1808.00671)
Shape Completion Enabled Robotic Grasping (arXiv 1609.08546)