MimicGen

Common

A system that segments a few human demonstrations by object, transforms and stitches them, and auto-generates many new demonstrations.

MimicGen is an automatic data-generation system proposed by NVIDIA and UT Austin at CoRL 2023. It splits a human demonstration into object-centric subtask segments (such as “grasp the cup” and “place it on the plate”); when applied to a new scene where the object is in a different position, it transforms each segment's end-effector trajectory to match that object's new pose, stitches the segments back together, executes the result in simulation, and keeps only the successful runs. Using fewer than 200 human demonstrations, the paper generated more than 50,000 demonstrations across 18 tasks, easing imitation learning's dependence on human-collected demonstrations. NVIDIA's Isaac Lab Mimic, built into Isaac Lab, follows the same idea — annotate subtasks, then generate automatically — and also supports humanoid robots.

ExampleFor one simulated manipulation task, giving MimicGen just 10 human demonstrations lets it generate 1,000 demonstrations across a wider range of object placements, which are then used to train a policy.

Related: DexMimicGen · DemoGen · Synthetic Data · Demonstration Data · NVIDIA Isaac Lab · Behavior Cloning
Sources: MimicGen 项目主页 (Chinese)
Isaac Lab Docs: Teleoperation and Imitation Learning with Isaac Lab Mimic
NVIDIA Technical Blog: Building a Synthetic Motion Generation Pipeline for Humanoid Robot Learning