HOT3D

Meta's first-person hand-object interaction dataset, recorded with Aria glasses and Quest 3, with motion-capture ground truth.

HOT3D is a first-person hand-object interaction dataset released by Meta, with the paper appearing at CVPR 2025. 19 participants manipulated 33 rigid objects in staged kitchen, office, and living-room settings, recorded with two kinds of device, Project Aria research glasses and a Quest 3 headset, for a total of over 833 minutes and 3.7 million images. The data includes multi-view RGB and monochrome images, eye gaze, scene point clouds, and 3D poses for the cameras, both hands, and the objects; the pose ground truth comes from a motion-capture system using optical markers, which is more accurate than pose estimated algorithmically. Hand annotations are provided in both UmeTrack and MANO format (a commonly used parametric hand model), and objects come with 3D meshes carrying PBR (physically based rendering) materials. It's mainly used for research on 3D hand and object tracking and pose estimation, and also serves as a data source for learning dexterous manipulation from first-person human video. Using it requires agreeing to the HOT3D license.

Also called

HOT3D: Hand and Object Tracking in 3D from Egocentric Multi-View Videos

Related

Hand-Object Interaction · Egocentric Video · Project Aria Glasses · MANO · Optical Motion Capture · Hand Pose Estimation

Sources

HOT3D: Hand and Object Tracking in 3D from Egocentric Multi-View Videos (arXiv)
HOT3D 项目主页 (Chinese)

As of

2025-06