Intel OpenVINO

Intel's open-source inference-optimization and deployment toolkit for speeding up models on Intel CPUs, integrated graphics, and NPUs.

OpenVINO (Open Visual Inference and Neural network Optimization) is Intel's open-source deep-learning inference toolkit. It converts a model from PyTorch, ONNX, TensorFlow, or another format into its own intermediate representation, then applies operator fusion and low-precision quantization (such as INT8) tuned specifically for Intel CPUs, integrated graphics, and NPUs (neural processing units) before running it. It occupies a similar niche to NVIDIA's TensorRT, just aimed at Intel hardware instead. Many robots run on an industrial PC or mini PC with no discrete GPU, and deploying vision detection or segmentation models with OpenVINO in that situation can raise frame rate and cut latency without adding a GPU.

ExampleExport a YOLO detection model to OpenVINO format and run real-time tabletop object detection on the CPU of an Intel Core mini PC mounted on the robot.

Also called

OpenVINO Toolkit

Related

Inference Deployment · On-Device / Edge Deployment · NVIDIA TensorRT · Open Neural Network Exchange (ONNX) · Post-Training Quantization (PTQ) · Industrial PC

Sources

OpenVINO 官方文档 (Chinese)
openvinotoolkit/openvino (GitHub)