Marker Tracking

Prints a dot pattern on a vision-based tactile sensor’s soft gel and tracks the dots’ displacement to estimate shear force and slip.

Marker tracking is a common signal-processing method for vision-based tactile sensors (which photograph a soft gel’s deformation with a camera, like GelSight): rows of black dots are printed between the elastomer and its reflective coating, and on contact the camera captures how these dots shift laterally across the gel surface, with point-by-point tracking giving a displacement field. The 2017 GelSight paper by Yuan, Dong, and Adelson summarized how to read it: under normal pressure the markers spread outward from the contact center; under shear force they shift together in the shear direction; a rotational pattern indicates an in-plane torque; and displacement magnitude is roughly proportional to force. When an object is about to slip, points near the edge of the contact region start moving first while the center stays still — this non-uniformity can be used to detect slip before it fully happens. Looking only at the gel’s surface bumps (photometric stereo) gives shape alone; markers add tangential-force information. Sensors like TacTip instead use the displacement of internal pins as their main signal.

ExampleGripping a cup with a gripper fitted with GelSight Mini sensors, if the outer-ring markers start moving in the same direction while the center markers stay roughly still, that signals the cup is about to slip, so the gripper can increase its grip force.

Also called

Tactile Markers, Marker Displacement, Marker Flow

Related

Vision-Based Tactile Sensor · GelSight · Slip Detection · Photometric Stereo · Normal Force and Tangential (Shear) Force · Tactile Image

Sources

GelSight: High-Resolution Robot Tactile Sensors for Estimating Geometry and Force (Sensors 2017)