RT-1 Robot Action Dataset

Google's real-robot manipulation dataset, collected over 17 months using 13 robots.

The RT-1 dataset is the real-robot data Google collected to train its RT-1 model: 13 Everyday Robots single-arm mobile robots, operating in office-kitchen-like settings, recorded via human teleoperation over 17 months, producing more than 130,000 episodes (one complete task execution) covering over 700 language instructions. Inside Open X-Embodiment it's named fractal20220817_data; the TFDS version contains 87,212 episodes, stored in RLDS format, with each step carrying an image, a language instruction, and an action (end-effector displacement, gripper open/close, base movement, and a termination signal). It was among the largest public real-robot datasets of its time, and it's also the training data that SimplerEnv's “Google Robot” evaluation tasks are built to match.

ExampleLoading fractal20220817_data from Open X-Embodiment gives frame-by-frame images and actions for the robot picking up a soda can from a table under an instruction like “pick coke can”; SimplerEnv's Google Robot “pick up the coke can” task was recreated directly from this setup.

Also called

Fractal, fractal20220817_data, Google Robot Dataset

Related

RT-1 · Open X-Embodiment · SimplerEnv · Everyday Robots Mobile Manipulator · RLDS (Reinforcement Learning Datasets) · Real-Robot Data

Sources

RT-1: Robotics Transformer for Real-World Control at Scale（项目页） (Chinese)
TFDS Catalog: fractal20220817_data

As of

2022-12