Distractor Objects

Extra objects in a scene that are irrelevant to the current task but can throw a policy off.

Distractors are objects in a manipulation or navigation scene that are unrelated to the current instruction — for example, a bowl, a toy, and other cups sitting on the table when the robot is told to pick up the red cup. Evaluations often deliberately add or remove distractors to see whether a policy grasps the wrong target or gets thrown off by occlusion or visual changes, making this a standard way to measure visual generalization and robustness. The Colosseum benchmark (RSS 2024) tests manipulation policies along 14 kinds of perturbation and finds that a single perturbation alone can drop success rates by 30–50%, with the number of distractors, the target object's color, and lighting having the largest effects. Common countermeasures include training with cluttered scenes and data augmentation, or first using open-vocabulary detection to box in the actual target.

ExampleThe training data shows only a single apple on the table; at test time an orange and a small red ball are added next to it, and an imitation-learning policy may reach for the red ball instead.

Also called

Distractors

Related

Visual Generalization · Robustness · Out-of-Distribution · The Colosseum: A Benchmark for Evaluating Generalization for Robotic Manipulation · Generalization / Robustness Evaluation · Open-Vocabulary Object Detection

Sources

THE COLOSSEUM: A Benchmark for Evaluating Generalization for Robotic Manipulation (arXiv 2402.08191)