NVIDIA cuDNN
GPU-accelerated building blocks for deep learning.
The NVIDIA CUDA Deep Neural Network library (cuDNN) provides optimized operations for training and inference on NVIDIA GPUs. Use attention, matrix multiplication, convolution, and multi-operation fusion through your framework or the cuDNN Python and C++ APIs.
Integrations
Use cuDNN through the tools in your AI stack, or build directly on its graph APIs. These paths show how selected frameworks and libraries reach cuDNN. Support depends on the operation, selected backend, library versions, and GPU.
FRAMEWORKS & TRAINING
PyTorch & JAX also connect directly
LLM INFERENCE
cuDNN attention backend
BUILD & EXPERIMENT
AI labs, research & benchmarks
Graph APIs & optimized kernels
Select a framework or library to see how it connects and where to get started.
Developer guide
PyTorch and JAX connect directly to cuDNN. JAX and Megatron Core also connect through Transformer Engine.
Browse samples
Explore the vLLM and SGLang paths through FlashInfer to the cuDNN attention backend.
Attention API
Use cuDNN-backed operations through PyTorch, or integrate the frontend directly with your tensors.
Python samples
JAX has a direct cuDNN attention path and a path through Transformer Engine.
JAX attention API
Transformer Engine connects Megatron training workloads to cuDNN attention kernels.
Megatron Core docs
Explore fused attention support for PyTorch and JAX workloads.
Transformer Engine docs
Use cuDNN through the FlashInfer integration. Execution depends on the selected backend and supported workload.
FlashInfer cuDNN API
Use cuDNN through the FlashInfer integration. Execution depends on the selected backend and supported workload.
FlashInfer cuDNN API
FlashInfer exposes batched prefill attention with paged KV cache backed by cuDNN SDPA.
Explore the API
Use the Python or C++ graph API for custom applications, research, and benchmarks.
Start building
Explore cuDNN within the RTX inference stack. Check the TensorRT for RTX documentation for supported computations.
TensorRT for RTX docs
Tokamax can select the cuDNN implementation of dot-product attention through JAX.
Tokamax
Get started
Set up a compatible NVIDIA GPU, CUDA, and cuDNN backend before using the frontend. Check the supported products and installation guide for your environment.
Python
C++
Install the Python frontend:
Check that Python can load the cuDNN backend:
Follow the Python installation guide, then build and execute your first graph.
For most applications, start with the frontend APIs. Use the cuDNN backend API if you need a C-only interface or legacy fixed-function routines that the frontend does not expose.
Operations
cuDNN supports individual operations and flexible fusion patterns. Describe a computation as an operation graph so cuDNN can select an execution plan for your GPU.
cuDNN also supports softmax and pooling.
Contribute and get help
The cuDNN frontend is open source. Explore the code, contribute, or report an issue. See the release notes for updates and support resources for help.