Robot-free (Embodiment-free) Data Collection

Collecting data with no real robot present, by having a human hold or wear a device shaped like the robot's end effector.

Ordinary teleoperation requires a human to operate an actual robot, which is expensive equipment that's hard to bring into a real home. Robot-free collection instead has a person hold or wear a device designed to match the robot's end effector and perform the task directly; a camera and sensors on the device record the footage and motion, which is later converted into robot actions for training. The best example is UMI (Universal Manipulation Interface), proposed by Cheng Chi, Shuran Song, and colleagues in 2024: a handheld gripper with a wrist-mounted GoPro, each demonstration taking about 30 seconds, claimed to let collection start in any home or restaurant within two minutes, with the resulting policy deployable to different robot arms. DexUMI pushes the same idea to dexterous hands, using a hand exoskeleton to align kinematics and then inpainting a robot hand over the human hand in the video. The key is keeping the device's shape, viewpoint, and action capability as close as possible to the real robot, or an embodiment gap is left behind.

ExampleUMI's cup-placement task: collectors carry a handheld gripper and demonstrate placing a cup on a saucer in different locations; back at the lab, that data trains a policy that deploys straight onto a robot arm.

Also called

Embodiment-free Collection

Related

Universal Manipulation Interface · Handheld Gripper Data Collection · DexUMI · Wearable Data Collection · Capture-Execution Isomorphism · Embodiment Gap

Sources

Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robots
UMI 项目主页 (Chinese)
DexUMI: Using Human Hand as the Universal Manipulation Interface for Dexterous Manipulation

As of

2025-10