Real-Robot Data

Data recorded from an actual robot physically performing tasks in the real world.

Real-robot data refers to data recorded from an actual robot body performing tasks in the physical world, including camera footage, joint state, and action commands, usually collected through teleoperation, though it also includes data recovered when a robot acts autonomously. It carries no sim-to-real gap (the physical and visual mismatch between simulation and reality), and its actions can be replayed directly on the same model of robot, which is generally considered the highest-quality data available. Its downside is that it is expensive and slow: it requires buying robots, building scenes, and hiring data collectors to record data one demonstration at a time. The industry often describes this with a “data pyramid”: massive internet and human video data at the bottom, simulation and synthetic data in the middle, and a small amount of precious real-robot data at the top. Notable datasets include DROID, AgiBot World, and Open X-Embodiment.

ExampleDROID was collected by 50 people across 13 institutions teleoperating Franka arms with Quest 2 headsets over 12 months, gathering 76,000 trajectories totaling 350 hours across 564 scenes.

Also called

Real-World Robot Data

Related

Teleoperation · Simulation Data · Data Pyramid · DROID (Distributed Robot Interaction Dataset) · AgiBot World · Real-Robot-Data Camp vs. Sim-Data Camp

Sources

DROID: A Large-Scale In-the-Wild Robot Manipulation Dataset
AgiBot World Colosseo (arXiv 2503.06669)

As of

2025-03