Open X-Embodiment

A Google-led open dataset pooling more than a million real-robot trajectories from 22 kinds of robots.

Open X-Embodiment (OXE) is a cross-embodiment robot dataset released in October 2023 by Google DeepMind together with 21 other institutions. It consolidates 60 existing datasets from 34 labs into a unified RLDS format — a format Google proposed for storing robot data by episode — totaling more than 1 million real-robot trajectories, 22 robot embodiments, and 527 skills. Before this, each lab used its own data format and action definitions, making it hard to combine data for training. The team trained RT-1-X and RT-2-X on the pooled data: on robots that had less data of their own, RT-1-X's success rate was on average about 50% higher than a model trained only on that robot's own data, showing that mixing data from many kinds of robots helps them all. OXE went on to become the main pretraining data for open generalist policies such as Octo and OpenVLA.

ExampleAfter training on OXE's mixed data, RT-2-X's success rate on an emergent-skills evaluation — skills absent from Google's own robot data, learned only from other robots' data — was roughly 3 times that of RT-2, which was trained only on Google's own data.

Also called

OXE, Open X, RT-X Dataset

Related

Cross-Embodiment Data · RT-X · RLDS (Reinforcement Learning Datasets) · Octo · OpenVLA · OXE Magic Soup

Sources

Open X-Embodiment: Robotic Learning Datasets and RT-X Models（项目主页） (Chinese)
google-deepmind/open_x_embodiment（GitHub）

As of

2023-10