Depth Completion

Fills in missing or sparse pixels in a depth map to produce complete, dense depth.

Depth completion takes an incomplete depth map — one with holes, or sparse like lidar output — usually paired with a matching color image, and predicts dense depth with a value at every pixel. The RGB-D depth cameras commonly used on robots often fail to measure depth on transparent, reflective, overly bright, or distant surfaces, yet grasping and obstacle avoidance both depend on complete geometry, so completion is a common preprocessing step. The simplest approach interpolates from neighboring pixels to fill holes; Zhang and Funkhouser at CVPR 2018 proposed first predicting surface normals and occlusion boundaries from the color image, then jointly solving for complete depth together with the raw depth; ClearGrasp (2019) specifically completes depth for transparent objects to support grasping. Recent work also completes depth by combining a monocular depth foundation model with sparse depth input.

ExampleA gripper needs to grasp a glass cup on a table, but the depth camera returns almost entirely invalid values over the cup’s region; a method like ClearGrasp first fills in the cup surface’s depth, which is then passed to the grasp-detection network.

Also called

Depth Inpainting, Depth Map Completion

Related

Depth Holes · Depth Camera · Transparent & Reflective Object Perception · Monocular Depth Estimation · Prompt Depth Anything · LingBot-Depth

Sources

Deep Depth Completion of a Single RGB-D Image (arXiv 1803.09326, CVPR 2018)
ClearGrasp: 3D Shape Estimation of Transparent Objects for Manipulation (arXiv 1910.02550)