SuperPoint / SuperGlue / LightGlue

A line of neural-network models for detecting and matching feature points across images, replacing hand-crafted methods like SIFT.

This is a lineage of learned feature-matching techniques. SuperPoint, proposed by Magic Leap in 2018, uses a single network to jointly output keypoint locations and descriptors (vectors describing what the area around each point looks like) in an image, replacing hand-designed features such as SIFT and ORB. SuperGlue (2020) uses a graph neural network with attention to match keypoints between two images, and can also tell which points have no correspondence in the other image. LightGlue (2023, ETH Zurich) is an improved version of SuperGlue that the paper describes as using less memory and compute, being more accurate, easier to train, and able to adaptively stop inference early depending on how hard the match is. These models are commonly plugged into the front end of SfM, visual SLAM, and relocalization pipelines, to keep finding reliable correspondences even under large changes in lighting or viewpoint.

ExampleSwapping the default SIFT for SuperPoint keypoints and LightGlue matching inside an hloc or COLMAP pipeline finds more correct matches between photos taken in daylight and at night.

Also called

Learned Feature Matching

Related

Feature Points · Feature Matching · Structure from Motion · Visual SLAM · Relocalization · Random Sample Consensus

Sources

LightGlue: Local Feature Matching at Light Speed (arXiv 2306.13643)
SuperPoint: Self-Supervised Interest Point Detection and Description (arXiv 1712.07629)
SuperGlue: Learning Feature Matching with Graph Neural Networks (arXiv 1911.11763)

As of

2023-06