Vision-Based Tactile Sensor

A tactile sensor that uses a built-in camera to photograph how a soft surface deforms on contact.

A vision-based tactile sensor coats the surface of a transparent elastic gel with a reflective membrane, with LEDs and a tiny camera inside; when an object presses on it, the gel surface deforms, and what the camera captures is essentially a “tactile image.” Photometric stereo — reconstructing surface normals from how shading changes under lights from different directions — can recover fine geometry of the contact surface, while the displacement of markers embedded in the gel reveals shear force and slip. The best-known examples are MIT’s GelSight, first prototyped in 2009, and Meta’s DIGIT, an open-source, small, low-cost design released in 2020 that’s compact enough to mount on a dexterous fingertip. The advantage is high resolution with image output that vision networks can process directly; the downsides are that the gel surface wears down, the sensor has some thickness, and force values still need separate calibration.

ExampleResearchers mounted two DIGIT sensors on two fingers of an Allegro dexterous hand and used a tactile model-predictive controller to roll a glass marble back and forth between the fingers, predicting contact changes from the tactile images.

Also called

VBTS, Visuotactile Sensor, Optical Tactile Sensor, GelSight-Style Sensor

Related

GelSight · DIGIT · Tactile Sensor · Photometric Stereo · Marker Tracking · Tactile Image

Sources

GelSight: High-Resolution Robot Tactile Sensors for Estimating Geometry and Force (Sensors 2017)
DIGIT: A Novel Design for a Low-Cost Compact High-Resolution Tactile Sensor (RA-L 2020)