NVIDIA cuDNN

Getting Started

• Release Notes
  • cuDNN 9.6.0
  • cuDNN 9.5.1
  • cuDNN 9.5.0
  • cuDNN 9.4.0
  • cuDNN 9.3.0
  • cuDNN 9.2.1
  • cuDNN 9.2.0
  • cuDNN 9.1.1
  • cuDNN 9.1.0
  • cuDNN 9.0.0

Installation Guide

• Overview
• Installing cuDNN on Linux
  • Prerequisites
    • Installing NVIDIA Graphics Drivers
    • Installing the CUDA Toolkit for Linux
    • Installing Zlib
  • Installing on Linux
    • Package Manager Installation
      • Package Manager Network Installation
        • Ubuntu/Debian Network Installation
        • RHEL 9/Rocky 9 and RHEL 8/Rocky 8 Network Installation
      • Package Manager Local Installation
        • Ubuntu/Debian Local Installation
        • RHEL 9/Rocky 9 and RHEL 8/Rocky 8 Local Installation
      • Additional Package Manager Capabilities
        • Meta-Packages
        • Base Packages
    • Tarball Installation
      • Redist Archive
    • Python Wheels - Linux Installation
      • Prerequisites
      • Installing cuDNN with Pip
    • Verifying the Install on Linux
    • Upgrading From Older Versions of cuDNN to cuDNN 9.x.y
• Installing cuDNN on Windows
  • Prerequisites
    • Installing NVIDIA Graphic Drivers
    • Installing the CUDA Toolkit for Windows
  • Downloading cuDNN for Windows
  • Installing on Windows
  • Upgrading cuDNN
  • Python Wheels - Windows Installation
    • Prerequisites
    • Installing cuDNN with Pip
• Building and Running a cuDNN Dependent Program
  • Building a cuDNN Dependent Program
  • Running a cuDNN Dependent Program
  • Inter-Library Dependencies
• Cross-Compiling cuDNN Samples
  • Linux arm64-SBSA
    • Installing the CUDA Toolkit for Linux arm64-SBSA
    • Installing cuDNN for Linux arm64-SBSA
    • Cross-Compiling cuDNN Samples for Linux arm64-SBSA
  • Linux aarch64-Jetson
    • Installing the CUDA Toolkit for Linux aarch64-Jetson
    • Installing cuDNN for Linux aarch64-Jetson
    • Cross-Compiling cuDNN Samples for Linux aarch64-Jetson

Backend API

• Backend API Overview
• cudnn_graph Library
• cudnn_ops Library
• cudnn_cnn Library
• cudnn_adv Library

Developer Guide

• Overview
• Core Concepts
  • cuDNN Handle
  • Tensors and Layouts
    • Tensor Descriptor
      • WXYZ Tensor Descriptor
      • 3-D Tensor Descriptor
      • 4-D Tensor Descriptor
      • 5-D Tensor Descriptor
      • Fully-Packed Tensors
      • Partially-Packed Tensors
      • Spatially Packed Tensors
      • Overlapping Tensors
    • Data Layout Formats
      • Example Tensor
      • Convolution Layouts
        • NCHW Memory Layout
        • NHWC Memory Layout
        • NC/32HW32 Memory Layout
      • Matmul Layouts
  • Tensor Core Operations
    • Notes on Tensor Core Precision
• Graph API
  • Key Concepts
    • Operations and Operation Graphs
    • Engines and Engine Configurations
    • Other Runtime Concepts
      • Native CUDA Graph API
    • Heuristics
  • Graph API Example with Operation Fusion
    • Creating Operation and Tensor Descriptors to Specify the Graph Dataflow
    • Finalizing The Operation Graph
    • Configuring An Engine That Can Execute The Operation Graph
  • Supported Graph Patterns
    • Pre-Compiled Single Operation Engines
      • ConvolutionBwdData
      • ConvolutionBwdFilter
      • ConvolutionFwd
      • NormalizationBackward
      • NormalizationForward
    • Generic Runtime Fusion Engines
      • Support Surface
        • Support Surface 90
        • Support Surface 80
        • Support Surface 70
      • Operation Specific Constraints for the Runtime Fusion Engines
        • Matmul
        • Convolutions
        • Pointwise
        • GenStats
        • Reduction
        • ResampleFwd
        • Resampling Index Tensor Dump for Training
        • ResampleBwd
      • Examples of Supported Patterns
        • Single Operation
        • Pointwise Operations After Convolution 1
        • Pointwise Operations After Convolution 2
        • Pointwise Operations Before Matrix Multiplication
        • Convolution Producer Node in Middle of DAG
        • Mixed Input Precision Matmul and Convolution
    • Specialized Runtime Fusion Engines
      • BnAddRelu
      • DReluForkDBn
      • Fused Attention fprop
      • Fused Attention bprop
      • Fused Flash Attention fprop
      • Fused Flash Attention bprop
    • Specialized Pre-Compiled Engines
      • ConvBNfprop
      • ConvBNwgrad
      • ConvBiasAct
      • ConvScaleBiasAct
      • DgradDreluBNBwdWeight
      • FP8 Fused Flash Attention
    • Mapping with Backend Descriptors
• Legacy API
  • Convolution Functions
    • Prerequisites
    • Supported Algorithms
    • Data and Filter Formats
  • RNN Functions
    • Prerequisites
    • Supported Algorithms
    • Data and Filter Formats
    • Features of RNN Functions
  • Tensor Transformations
    • Conversion Between FP32 and FP16
    • Padding
    • Folding
    • Conversion Between NCHW And NHWC
  • Mixed Precision Numerical Accuracy
• Compatibility
  • cuDNN API Compatibility
  • cuDNN Hardware Forward Compatibility
    • Configuring Forward-Compatibility Mode
    • Forward-Compatibility and the Graph API
      • Supported Graph Patterns
    • Forward-Compatibility and the Legacy API
• Odds and Ends
  • cuDNN Library Configuration
    • FULL Configuration
    • GRAPH_JIT_ONLY Configuration
    • Sub-Library Configuration Instructions
  • Thread Safety
  • Reproducibility (Determinism)
  • Scaling Parameters
  • Deprecation Policy
  • GPU And Driver Requirements
  • Conventions And Features For Convolutions
    • Convolution Formulas
    • Grouped Convolutions
    • Best Practices for 3D Convolutions
      • Recommended Settings
      • Limitations
  • Environment Variables
  • SM Carveout
  • Version Checking Against CUDNN_VERSION
  • cuDNN Symbol Server
    • Example: Symbolizing
  • API Usage During Initialization and Termination

Reference

• Support Matrix
  • GPU, CUDA Toolkit, and CUDA Driver Requirements
  • CPU Architecture and OS Requirements
    • Linux
    • Windows
• Troubleshooting
  • Error Reporting And API Logging
  • FAQs
  • Support
• Documentation Archives
• Software License Agreement
  • LICENSE AGREEMENT FOR NVIDIA SOFTWARE DEVELOPMENT KITS
  • License.
    • Grant
    • Distribution Requirements
    • Authorized Users
    • Pre-Release SDK
    • Updates
    • Components Under Other Licenses
    • Reservation of Rights
  • Limitations.
  • Ownership.
  • No Warranties.
  • Limitations of Liability.
  • Termination.
  • General.
  • cuDNN SUPPLEMENT TO SOFTWARE LICENSE AGREEMENT FOR NVIDIA SOFTWARE DEVELOPMENT KITS
• Acknowledgements
  • University of Tennessee
  • University of California, Berkeley
  • Facebook AI Research, New York
  • Institutions and their Contributions
• Notices