Dexterous Manipulation

Using a multi-fingered hand, coordinated finger movement and force control to grasp, flip, and handle objects.

Dexterous manipulation refers to a robot hand — usually a multi-fingered dexterous hand — using fine coordination and force control across several fingers to grasp, reorient, and manipulate objects: rotating a block in the hand, unscrewing a bottle cap with the fingers, adjusting a grip on a pen. Compared with a two-finger parallel gripper that just opens and closes, this is much closer to a human hand, and it is key to letting robots use human tools and handle complex objects. The challenges are the many joints and high degrees of freedom, frequent contact that is hard to model, and the difficulty of both tactile sensing and data collection. A landmark project is OpenAI's Dactyl (2018), which trained a Shadow dexterous hand in simulation using reinforcement learning plus domain randomization (randomly varying friction, appearance, and other parameters), then transferred it directly to the real hand; the policy spontaneously learned human-like tricks such as finger gaiting, where the fingers release and reposition in turn to keep rotating an object. More recent work increasingly uses teleoperation or human hand videos for imitation learning instead.

ExampleOpenAI's Dactyl used a Shadow dexterous hand to reorient a block held in its palm to a target orientation: block pose was estimated from three ordinary cameras, fingertip positions were tracked with a motion-capture system, no tactile sensing was used, and the policy was trained entirely in simulation.

Also called

Dexterous Hand Manipulation, Multi-fingered Dexterous Manipulation

Related

Dexterous Hand · In-hand Manipulation · Contact-rich Manipulation · Tactile Sensor · Dactyl · Bimanual Manipulation

Sources

Learning Dexterous In-Hand Manipulation (OpenAI)
Dexterous Manipulation through Imitation Learning: A Survey