Plücker Coordinates

Represents a line in 3D space with 6 numbers — a direction plus a moment — and is often used to encode camera rays per pixel.

Plücker coordinates, introduced by the 19th-century German mathematician Julius Plücker, describe a line in 3D space with 6 numbers: a direction d, and a moment m = p × d (p is any point on the line, × is the cross product). The moment m doesn't depend on which point p on the line is chosen, and d·m = 0. The representation is fundamental in robotics: joint axes in screw theory and Featherstone's spatial vector dynamics are both built on Plücker coordinates. In recent years it has also shown up in generative models as the 'Plücker ray embedding': for every pixel in an image, compute (o × d, d) from the camera center o and that pixel's ray direction d, producing a 6-channel map used as a camera-pose conditioning input. CameraCtrl (2024) follows the approach of Light Field Networks (2021), arguing that this is easier for a network to associate with per-pixel information than directly feeding in the intrinsic and extrinsic matrices; it's commonly used in camera-controlled video generation and world models.

ExampleA camera at the origin looking along the z-axis has a ray direction d = (0, 0, 1) for the image's center pixel, with o × d = 0, giving the embedding (0, 0, 0, 0, 0, 1); after the camera translates sideways, o changes, and the same pixel's o × d changes with it, which is how the network senses camera motion.

Also called

Plücker Ray Embedding, Plücker Embedding, Plücker Ray Map

Related

Screw Theory · Spatial Vector Algebra · Camera Extrinsics · Camera Intrinsics · Video Generation Model · World Model

Sources

Wikipedia: Plücker coordinates
CameraCtrl: Enabling Camera Control for Text-to-Video Generation (arXiv 2404.02101)
Roy Featherstone: Spatial vector teaching materials（Plücker Basis Vectors）