RLBench

Common

A simulation benchmark of 100 robot-arm manipulation tasks from Imperial College London, built on CoppeliaSim.

RLBench was released in 2019 (IEEE RA-L 2020) by Stephen James, Andrew Davison, and colleagues at Imperial College London's Dyson Robotics Lab. It contains 100 hand-designed manipulation tasks, ranging from simple reaching to multi-step tasks like opening a door or an oven, using a Franka Panda arm by default and running on the CoppeliaSim simulator. Each task provides multi-camera RGB, depth, segmentation-mask, and proprioceptive observations, can automatically generate any number of demonstrations via motion planning, and many tasks also have variants that change color, position, and similar attributes. It was originally aimed at reinforcement learning, imitation learning, and few-shot learning, and later became one of the main simulation benchmarks for 3D manipulation policies: the 18-task subset selected by PerAct has since been reused by a large body of work including RVT and 3D Diffuser Actor.

ExamplePerAct trained a single multi-task Transformer on 18 RLBench tasks (249 variants), voxelizing multi-view RGB-D observations to predict the end effector's next keyframe pose; later work such as RVT-2 and 3D Diffuser Actor compares success rates on the same set of tasks.

Also called: RLBench-18 (PerAct subset)
Related: PerAct · RVT-2 · 3D Diffuser Actor · CoppeliaSim · Keyframe Action Prediction · The Colosseum: A Benchmark for Evaluating Generalization for Robotic Manipulation
Sources: RLBench: The Robot Learning Benchmark & Learning Environment (arXiv 1909.12271)
stepjam/RLBench (GitHub)
Perceiver-Actor (arXiv 2209.05451)