CUDA Deep Neural Network Library (cuDNN)

NVIDIA's GPU-accelerated library of the operators most commonly used in deep learning.

cuDNN is NVIDIA's acceleration library for deep-learning primitives, providing highly optimized implementations, across GPU generations, of common operators (an operator being a basic unit of computation in a neural network) such as convolution, pooling, normalization, and attention. Frameworks like PyTorch and TensorFlow don't write these low-level kernels themselves — they call into cuDNN — so whether it's installed correctly, and whether its version matches the CUDA toolkit and driver, directly determines whether training runs at all and how fast it goes. It isn't a standalone tool but a library installed alongside the CUDA ecosystem; in practice, the most common issues are mismatched environment versions, and the first few batches running slower while cuDNN auto-tunes to pick the fastest convolution algorithm.

ExampleTurning on torch.backends.cudnn.benchmark in PyTorch lets cuDNN search for the fastest convolution implementation for a fixed input size.

Also called

cuDNN

Related

CUDA · PyTorch · NVIDIA TensorRT · Operator / Kernel · GPU Memory (VRAM) · Mixed-Precision Training

Sources

NVIDIA cuDNN
NVIDIA cuDNN Documentation