6D Object Pose Estimation

Computing an object's 3D position and 3D orientation relative to the camera — six degrees of freedom in total.

6D pose estimation takes an RGB or RGB-D image and solves for a rigid object's 3 translations and 3 rotations relative to the camera — usually output as a rotation matrix plus a translation vector. Methods are grouped by how much is known about the object beforehand: instance-level (the specific object and its CAD model have been seen before), category-level (only that it belongs to a class such as ‘cup’), and methods for novel objects, which further split into model-based (given a CAD model) and model-free (given only reference images). The BOP benchmark has organized evaluations for this task since 2018. NVIDIA's FoundationPose, from 2023, handles both familiar and novel objects within one framework and can also track a pose continuously over time. Grasping, assembly, and AR all rely on it: knowing where an object is and which way it faces is what lets a system compute where a gripper needs to go.

ExampleA drill with a known CAD model sits on a table. A pose estimation model computes its position and orientation in the camera's frame from an RGB-D image, and combining that with the hand-eye calibration result gives the target pose the arm needs to grasp the handle.

Also called

6DoF Pose Estimation, Object Pose Estimation

Related

Pose · FoundationPose · BOP (Benchmark for 6D Object Pose Estimation) · Category-Level Pose Estimation · Pose Tracking · Average Distance of Model Points

Sources

BOP: Benchmark for 6D Object Pose Estimation
arXiv 2312.08344: FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects