Image-Based Visual Servoing

A closed-loop vision-control method that computes camera or arm velocity directly from the pixel error of image feature points.

Visual servoing uses camera feedback to close a control loop around robot motion, splitting into image-based (IBVS) and position-based (PBVS), with IBVS proposed by Weiss and Sanderson in the 1980s. IBVS doesn't reconstruct the target's 3D pose at all; it defines error directly on the image, e = s − s*, where s is the current feature (e.g., pixel coordinates of corner points) and s* the desired feature. Feature velocity and camera velocity v are related by ṡ = L·v, where L is the interaction matrix (or image Jacobian), depending on pixel coordinates and depth Z. The control law v = −λ·L⁺·e (L⁺ the pseudoinverse, λ a gain) drives the error down at roughly an exponential rate. It needs no object model and is fairly robust to calibration error; the drawback is that depth can only be estimated, the camera can retreat unexpectedly during a large rotation about the optical axis, and it can get stuck in a local minimum. PBVS instead estimates pose first and controls in 3D space, depending more heavily on calibration and a model.

ExampleA wrist camera aimed at 4 marker corners on an assembly part: at the aligned position, a reference image records the 4 corners' pixel coordinates as s*; at runtime the controller continuously compares current against desired pixels and outputs the camera's 6D velocity until the four points coincide — at which point the gripper is aligned with the hole too.

Also called

IBVS, 2D Visual Servoing

Related

Visual Servoing · Position-Based Visual Servoing · Jacobian Matrix · Eye-in-Hand · Camera Intrinsics · Closed-loop Control

Sources

Wikipedia: Visual servoing
Chaumette & Hutchinson: Visual servo control, Part II: Advanced approaches (IEEE RAM 2007，含 Part I 基本公式回顾) (Chinese)