RoboDojo (A Unified Sim-and-Real Benchmark for Comprehensive Evaluation of Generalist Robot Manipulation Policies)

A unified sim-and-real benchmark for general-purpose manipulation policies, with 42 simulated tasks and 18 real-robot tasks.

RoboDojo was released and open-sourced in July 2026 by the University of Hong Kong's MMLab together with 18 institutions including Berkeley, Tsinghua, Peking University, and Stanford, with Tianxing Chen as first author. It targets the problem that existing benchmark tasks tend to be too easy, get their scores saturated quickly, and lack real-robot validation. The simulation half is built on Isaac Sim / Isaac Lab, with 42 tasks on an ARX X5 dual-arm platform covering generalization, memory, precision, long-horizon execution, and open-ended instructions, with different tasks able to run in parallel across heterogeneous setups; the real-robot half sets 6 tasks each on three platforms — ARX X5, AgileX Piper, and Piper X — using standardized hardware and a remote cloud evaluation system called RoboDojo-RealEval to ensure reproducibility. Individual policies all connect through a shared interface called XPolicyLab; the first batch evaluated 30 policies, where the best in simulation, Hy-Embodied-0.5-VLA, averaged only an 8.80% success rate, and the best on real robots, π0.5, reached 12.8%, against human-expert scores of 76.03% and 100% respectively. The leaderboard is maintained and continuously updated by a nonprofit organization.

ExampleAfter connecting a self-trained VLA through the XPolicyLab interface, one command runs it through all 42 tasks in Isaac Sim and produces a results table in the same format as the leaderboard.

Also called

RoboDojo-RealEval

Related

RoboTwin · Benchmark · Simulation-Based Evaluation · Real-World Evaluation · RoboChallenge · π0.5

Sources

RoboDojo: A Unified Sim-and-Real Benchmark for Comprehensive Evaluation of Generalist Robot Manipulation Policies (arXiv 2607.04434)
RoboDojo-Benchmark/RoboDojo GitHub

As of

2026-09