Image Signal Processor

The processing unit that turns an image sensor’s raw output into a normal color image.

An ISP is a dedicated processor in a camera’s imaging pipeline — sometimes a standalone chip, but more often integrated into a system-on-chip (SoC) like a phone’s or a Jetson’s. A CMOS image sensor outputs raw Bayer-pattern data, where each pixel records only red, green, or blue; the ISP performs demosaicing (interpolating to recover a full-color image), noise reduction, white balance, auto-exposure, gamma and color correction, sharpening, and more, in sequence, outputting an RGB or YUV image. When a robot connects a bare sensor over a MIPI interface, it needs the host chip’s ISP to process and tune the image; USB cameras and some camera modules with a built-in ISP output an already-processed image directly. Auto-exposure and auto-white-balance make the brightness and color of the same scene drift over time, which affects training-data consistency, so they are often locked manually while collecting data.

ExampleNVIDIA Jetson developer kits’ OV5693 camera module has no ISP of its own — reading it directly over V4L2 gives raw Bayer data; going through libargus (nvarguscamerasrc) routes it through the Jetson’s built-in ISP, producing a normal color image.

Also called

ISP, Image Processor

Related

CMOS Image Sensor · MIPI CSI-2 · RGB Camera · NVIDIA Jetson · Global Shutter / Rolling Shutter

Sources

Wikipedia: Image processor
NVIDIA Jetson Linux Developer Guide: Camera Software Development Solution