Neural Processing Unit (NPU)

A low-power chip block specialized for the matrix math inside neural networks.

An NPU is an accelerator designed specifically for neural-network inference. Its core is a large array of multiply-accumulate units, well-suited to convolution and matrix multiplication, and it usually runs at low numerical precision such as INT8, giving lower power draw than a GPU at the same compute throughput — which is why it's commonly integrated as one block inside an SoC (system-on-chip) used in phones, cars, and robots. Different vendors use different names: Horizon Robotics calls its NPU architecture the BPU (Brain Processing Unit), and D-Robotics' RDK line of dev boards keeps that same name. Using an NPU usually means first converting and quantizing a model with the vendor's own toolchain (such as RKNN-Toolkit or Horizon's Tiangong Kaiwu), and falling back to the CPU or GPU for any operator the NPU doesn't support — this step is often where deploying large models onto real hardware gets stuck.

ExampleOn a Rockchip RK3588 dev board, a YOLO detection model is quantized to INT8 with RKNN-Toolkit and run on the NPU, freeing the CPU for other work.

Also called

BPU (Brain Processing Unit), AI Accelerator, Neural Network Accelerator

Related

System on Chip (SoC) · TOPS (Tera Operations Per Second) · Onboard Compute Platform · Rockchip RKNN-Toolkit · Horizon Robotics OpenExplorer · Post-Training Quantization

Sources

Neural processing unit - Wikipedia