DexUMI

A wearable exoskeleton that lets a human hand collect dexterous-hand data directly, then digitally replaces the hand in footage with the robot hand.

DexUMI is a dexterous-hand data collection and policy-learning framework from Shuran Song's group at Stanford, together with Columbia University, NVIDIA, and others, from 2025; it was a best-paper finalist at CoRL 2025. It extends the idea behind UMI (Universal Manipulation Interface, which collects data with a handheld gripper) to multi-fingered dexterous hands, using the human hand itself as the collection interface, and narrows the embodiment gap between human and robot hands in two ways. On the hardware side, a wearable exoskeleton is custom-built for the target robot hand, constraining human hand motion to what the robot hand can actually do while letting the operator feel contact directly; wrist pose is recorded with an iPhone, a wide-angle camera sits below the wrist, and the exoskeleton carries tactile sensors. On the software side, the human hand and exoskeleton are digitally erased from the footage and the background is inpainted, then an image of the robot hand in the matching pose is composited in, so the training footage matches what the robot will actually see at deployment. Across two robot hands, Inspire and XHand, it reaches 86% average success.

ExampleOn a tea-leaf-scooping task, the paper reports DexUMI's data-collection throughput is about 3.2 times that of conventional teleoperation.

Also called

DexUMI: Using Human Hand as the Universal Manipulation Interface for Dexterous Manipulation

Related

Universal Manipulation Interface · Exoskeleton · Embodiment Gap · DexCap · Dexterous Hand · DEXOP

Sources

DexUMI 项目主页 (Chinese)
DexUMI (arXiv)

As of

2025-05