DexYCB

NVIDIA's multi-view dataset of human hands grasping objects, with 3D pose annotations for both the hand and the object.

DexYCB is a real human-hand object-grasping dataset released by NVIDIA and the University of Washington, published at CVPR 2021. The authors used 8 synchronized RealSense D415 depth cameras to film 10 subjects grasping 20 YCB objects (YCB is a standard set of everyday objects widely used in robotics research) from multiple angles, yielding 1,000 sequences and about 580,000 RGB-D frames in total. Hand pose is annotated using the MANO parametric hand model, and each object is given a 6D pose (3D position plus 3D orientation). It's used as a benchmark for 2D detection, 6D object pose estimation, and 3D hand pose estimation, and the paper also proposes an evaluation for generating safe grasps when a person hands an object to a robot. In embodied AI, it's commonly used as a source of human grasping demonstrations, which are converted into robot dexterous-hand trajectories through motion retargeting.

ExampleUsing dex-retargeting's position-retargeting mode, the MANO hand poses of people grasping objects in DexYCB are converted into grasping trajectories for robot hands such as Allegro and Shadow.

Also called

DexYCB: A Benchmark for Capturing Hand Grasping of Objects

Related

YCB Object and Model Set · MANO · Hand Pose Estimation · 6D Object Pose Estimation · Human-Robot Handover · dex-retargeting

Sources

DexYCB 项目主页 (Chinese)
DexYCB: A Benchmark for Capturing Hand Grasping of Objects (arXiv)

As of

2021-04