9DTact

An open-source, compact vision-based tactile sensor from Tsinghua’s Huazhe Xu group that measures contact shape and 6D force.

9DTact is a vision-based tactile sensor from Huazhe Xu’s group at Tsinghua University along with the Shanghai Qi Zhi Institute and other collaborators; the paper was published in IEEE RA-L and presented at ICRA 2024. The “9D” in the name refers to 3D shape plus 6D force. It’s a GelSight-style design: an internal camera photographs the back of a soft gel pad, and when an object presses on it, the gel deforms and reflects less light, so the image’s shading pattern reflects contact depth. It exploits the optical properties of a translucent gel to estimate three-axis force and three-axis torque directly from the image with a neural network, without needing markers on the gel surface. It was trained on about 100,000 paired image-force samples from 175 objects and generalizes to unseen objects. It measures about 32.5 × 25.5 × 25.5 millimeters, with an average shape-reconstruction error of about 0.046 millimeters. Both hardware and software are open source, with a build guide included.

ExampleMounted on a gripper fingertip, 9DTact reads the normal and shear forces at the fingertip during a grasp, which can be used to judge whether an object is starting to slip and whether to increase grip force.

Related

Vision-Based Tactile Sensor · GelSight · DIGIT · Six-Axis Force/Torque Sensor · Photometric Stereo · Slip Detection

Sources

9DTact: A Compact Vision-Based Tactile Sensor for Accurate 3D Shape Reconstruction and Generalizable 6D Force Estimation (arXiv 2308.14277)
9DTact project page
9DTact GitHub repository

As of

2024-05