Extrinsic Contact Sensing

The perception task of inferring where and how an object held in the hand is making contact with the outside environment.

When a robot holds an object to do work, there are two kinds of contact: contact between the fingers and the object is called intrinsic contact, while contact between the held object and the environment — a tabletop, the wall of a hole — is called extrinsic contact. Extrinsic contact happens outside the hand, is often occluded, and can’t be felt by the fingers directly, so it has to be inferred indirectly. In an ICRA 2021 paper, MIT’s Ma, Dong, and Rodriguez used a distributed tactile sensor to track the object’s tiny relative motion inside the hand, and combined this with rigid-body constraints like non-penetration and friction to estimate the location of a point or line contact, arguing that distributed tactile sensing suits this better than looking only at the net force from a six-axis force sensor. Later work also infers contact location from touch, vision, or even sound using neural networks. This is crucial for contact-rich tasks like peg-in-hole assembly and tool use: knowing where contact happened is what tells the robot which way to adjust.

ExampleHiguera and colleagues’ NCF-v2 infers the contact between a held object and the environment from gripper tactile sensing; plugged into an insertion policy, it raised the success rate of placing a cup into a saucer by 33% and made execution 1.36 times faster, and raised the success rate of placing a bowl on a rack by 13%.

Also called

Extrinsic Contact Estimation, Tool-Environment Contact Sensing

Related

Contact Detection · Tactile Sensor · Peg-in-Hole Insertion · Contact-rich Manipulation · Tool Use · Extrinsic Dexterity

Sources

Extrinsic Contact Sensing with Relative-Motion Tracking from Distributed Tactile Measurements (ICRA 2021)
Perceiving Extrinsic Contacts from Touch Improves Learning Insertion Policies