dex-retargeting

An open-source library that converts human hand keypoint poses into joint angles for many different robotic dexterous hands.

dex-retargeting is an open-source Python library, MIT-licensed and pip-installable, spun out of the AnyTeleop project (UC San Diego and NVIDIA, RSS 2023). Human hands and robot hands differ in finger count, length, and joint structure, so joint angles can't just be copied over — they need retargeting: the library takes hand keypoints from a MediaPipe or MANO hand model as input, and solves an optimization problem for the robot hand's joint angles that keeps fingertip positions, or the vectors between fingertips, as close as possible to the human hand's. It offers three retargeting modes — vector retargeting (suited to real-time video teleoperation), position retargeting (suited to offline conversion of hand-object interaction datasets), and DexPilot — along with a model library covering hands such as Allegro, Shadow, LEAP, and Inspire. Many dexterous-hand teleoperation and human-hand data-collection projects use it to convert human hand motion into robot hand motion.

ExampleAn official example uses position retargeting to offline-convert the MANO hand poses of people grasping objects in the DexYCB dataset into grasping trajectories for robot hands such as Allegro and Shadow.

Also called

dex_retargeting, AnyTeleop Hand Retargeting Library

Related

AnyTeleop: A General Vision-Based Dexterous Robot Arm-Hand Teleoperation System · Motion Retargeting · MANO · Dexterous Hand · MediaPipe · DexYCB

Sources

dex-retargeting GitHub 仓库 (Chinese)
AnyTeleop: A General Vision-Based Dexterous Robot Arm-Hand Teleoperation System (arXiv)

As of

2026-09