Object Tracking

Continuously finding the same object across video frames and keeping its identity consistent over time.

Object tracking locates the same object frame by frame through a video, deciding whether ‘the object in this frame is the same one as in the last frame.’ Single-object tracking follows one object given its box in the first frame; multi-object tracking (MOT) tracks many objects at once while keeping each one's own identity. A common recipe is ‘detect then associate’: run a detector every frame, predict each object's motion with a Kalman filter, and match the new detections to existing tracks with the Hungarian algorithm — SORT, from 2016, does exactly this. The difficulty lies in occlusion, changes in appearance, fast motion, and similar-looking objects getting confused with one another. Robots use tracking to keep a lock on an object they intend to grasp, to follow a pedestrian, or to track the object a hand is manipulating in a human video; video segmentation models such as SAM 2 can also track objects in the form of masks.

ExampleOn a sorting line, packages keep moving on the conveyor belt. The system detects each package every frame and uses tracking to maintain its ID, so the arm can grab the correct one at the right moment.

Also called

Multi-Object Tracking, MOT, Single-Object Tracking, Visual Tracking

Related

Object Detection · Kalman Filter · Video Object Segmentation · Tracking Any Point · Embodied Visual Tracking · Pose Tracking

Sources

Wikipedia: Video tracking
Simple Online and Realtime Tracking (SORT, ICIP 2016)