Occupancy Network

A neural network that predicts, for every location in 3D space, whether it is occupied by an object.

“Occupancy network” originally referred to work by Mescheder et al. at CVPR 2019: a network that takes an image or point cloud as input and, for any queried 3D point, outputs the probability that the point lies inside an object — an implicit representation used to reconstruct 3D shapes at a resolution not limited by a voxel grid. A second, unrelated sense of the term comes from self-driving cars: in 2022 Tesla publicly described an occupancy network that uses multiple onboard cameras to predict whether each voxel of space around the vehicle is occupied, and “3D occupancy prediction” has since become a popular task in autonomous-driving research. Its advantage is that it does not depend on a predefined list of object categories, so it can represent oddly shaped obstacles; it is closely related to occupancy grid maps and bird’s-eye-view perception.

ExampleA camera-only self-driving system predicts whether each voxel of space around the car is occupied using multiple cameras, letting it avoid oddly shaped obstacles that aren’t in its list of detectable categories.

Also called

Occupancy Networks, 3D Occupancy Prediction, Occupancy Grid Prediction

Related

Occupancy Grid Map · Implicit vs. Explicit 3D Representation · Bird’s-Eye View · Autonomous Driving · Vision-Only Approach · Tesla AI Day

Sources

Occupancy Networks: Learning 3D Reconstruction in Function Space (arXiv)