Transparent & Reflective Object Perception Datasets and Benchmarks

Depth and segmentation datasets collected specifically for glass, metal, and other objects that depth cameras struggle to measure.

This category of dataset targets transparent or highly reflective objects such as drinking glasses, plastic bottles, and stainless-steel tableware. Ordinary depth cameras depend on light reflecting back normally, and fail on these materials with depth holes, or by mistaking the surface behind the object (such as a tabletop seen through a glass) for the object’s own surface, causing grasp failures. Representative work includes ClearGrasp, released in 2019, which provides more than 50,000 synthetic RGB-D images plus 286 real images with ground truth, and repairs depth by predicting surface normals, a transparent-region mask, and occlusion boundaries; and TransCG (RA-L 2022), from Cewu Lu’s group at Shanghai Jiao Tong University, which used two RealSense cameras to capture 57,715 real RGB-D images across 130 scenes covering 60 transparent objects, with ground-truth depth, normals, and masks included. These datasets are mainly used to train and evaluate depth-completion networks, whose repaired depth is then handed to a grasping algorithm.

ExampleIn a kitchen scene captured by a wrist-mounted RealSense, the region covering a drinking glass is almost entirely depth holes; only after a depth-completion network trained on TransCG fills it in can the grasp-detection module produce a usable grasp pose.

Also called

Transparent Object Depth Completion Datasets, ClearGrasp, TransCG

Related

Transparent & Reflective Object Perception · Depth Completion · Depth Holes · Depth Camera · Grasp Pose Detection · RealSense Depth Camera (D435i / D405)

Sources

ClearGrasp: 3D Shape Estimation of Transparent Objects for Manipulation (arXiv)
TransCG: A Large-Scale Real-World Dataset for Transparent Object Depth Completion and a Grasping Baseline (arXiv)
TransCG 数据集主页 (GraspNet) (Chinese)

As of

2022