Hand-Eye Calibration

Finding the fixed coordinate transform that relates a camera to a robot arm.

Hand-eye calibration solves for the fixed, unchanging transform between the ‘eye’ (camera) and the ‘hand’ (robot arm). There are two configurations: eye-in-hand, where the camera is mounted on the end-effector and the goal is the camera's pose relative to the end-effector flange; and eye-to-hand, where the camera is fixed nearby and the goal is the camera's pose relative to the robot's base. The method moves the arm through several poses, recording the end-effector's pose and the calibration board's pose as seen by the camera at each one; the problem is written as AX = XB, where X is the transform being solved for. Classic solutions such as Tsai-Lenz (1989) are already implemented in OpenCV's calibrateHandEye. At least two motions with non-parallel rotation axes are required — meaning at least 3 poses — though more are collected in practice. An inaccurate calibration means objects the camera sees get mapped to the wrong arm coordinates, and grasping simply fails.

ExampleTo calibrate a wrist camera, the arm carries it through a dozen or more angles photographing a fixed ChArUco calibration board. Forward kinematics gives the end-effector's pose at each shot, PnP gives the board's pose relative to the camera, and OpenCV's calibrateHandEye then solves for the transform from the camera to the flange.

Also called

AX=XB, Camera-to-Arm Calibration

Related

Eye-in-Hand · Eye-to-Hand · Camera Extrinsics · Calibration Board · Homogeneous Transformation Matrix · Wrist Camera

Sources

OpenCV calib3d.hpp：calibrateHandEye 与 AX=XB 说明 (Chinese)
Wikipedia: Hand eye calibration problem