Data Collection SOP

The written procedure that specifies exactly how each demonstration should be collected and what counts as an acceptable one.

A data collection SOP is the operating manual a data team writes for its collection workers. It typically spells out: equipment checks and camera calibration; how the scene should be arranged and how often it should be changed; where task instructions come from; how the start and end of one demonstration are defined; the pace and technique for the motion; how to handle and flag failures; file naming and upload; and quality-control criteria. Robot learning is sensitive to inconsistency in the data — if different people perform a task differently, start and end points aren't standardized, or long pauses creep in, the model struggles to learn — so an SOP is the foundation for keeping the proportion of valid data high. A good SOP also builds in diversity: DROID's shared collection protocol has operators change scenes roughly every 20 minutes and randomly prompts them to adjust lighting, move the camera, or add and remove objects. China is also pushing for standardization: the Beijing Humanoid Robot Innovation Center is leading a proposed industry standard for the Ministry of Industry and Information Technology, titled “Artificial Intelligence — Embodied Intelligence — Data Collection Specification.”

ExampleDROID's 50 collection workers are spread across multiple universities and all use the same collection protocol and GUI: the interface randomly draws an instruction from a list of feasible tasks the worker has entered, and after each trajectory the worker marks it as a success or failure.

Also called

Data Collection Standard Operating Procedure, Data Collection Protocol

Related

Data Collector (Teleoperator) · Valid (Usable) Data · Data Quality Control · Demonstration Data · DROID (Distributed Robot Interaction Dataset) · Crowdsourced Data Collection

Sources

DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset (arXiv 2403.12945)
国地共建具身智能机器人创新中心打造具身智能标准体系 (Chinese)

As of

2024-11