Digital Cousin

A simulated scene that only needs to be geometrically and semantically similar to reality, not an exact one-to-one replica.

Digital cousin was proposed by Fei-Fei Li and Jiajun Wu's group at Stanford in their CoRL 2024 paper “Automated Creation of Digital Cousins for Robust Policy Learning,” as a counterpart to the “digital twin” (a virtual copy that precisely replicates one real scene down to the last detail). A digital cousin doesn't replicate any specific real scene; it only needs to be geometrically and semantically similar — for a kitchen cabinet, for instance, it's enough for the shape, size, and opening mechanism to be close, built directly by picking similar objects from an existing asset library. This costs less than precise modeling, and generating several cousin scenes at once brings built-in diversity, making a trained policy less prone to overfitting to one specific scene. The paper's ACDC pipeline builds a simulated environment automatically from a single real RGB photo, through three steps: extracting object information, matching similar assets, and generating an interactive scene. In its reported zero-shot sim-to-real transfer, policies trained on digital cousins reached about 90% success versus about 25% for digital twins.

ExampleGiven one photo of a kitchen cabinet at home, ACDC automatically picks several cabinets from its asset library with similar shapes and opening mechanisms, generates several cousin scenes, trains a cabinet-opening policy in simulation, and deploys it directly to the real cabinet.

Also called

ACDC, Automated Creation of Digital Cousins

Related

Digital Twin · Real-to-Sim-to-Real · Sim-to-Real Transfer · Domain Randomization · Simulation Assets · OmniGibson

Sources

Automated Creation of Digital Cousins for Robust Policy Learning (arXiv 2410.07408)
Digital Cousins 项目主页 (Chinese)

As of

2024-10