NVIDIA cuDNN Frontend

The NVIDIA cuDNN frontend API provides a simplified programming model that is sufficient for most use cases.

Use the NVIDIA cuDNN backend API only if you want to use the legacy fixed-function routines that are not graph-based interfaces and are not exposed by the frontend API layers.

• Overview
• Installation Guide
• Samples
  • Python Samples
    • Source Code
    • Dependencies
  • C++ Samples
    • Source Code
    • Build
• Releases Notes

Frontend API

• Operations
  • Attention
  • Block Scaling
  • Concatenate
  • Convolutions
  • Matmul
  • Normalizations
  • Pointwise and Reduction
  • Resampling
  • Slice
• Utilities
  • CUDA Graphs
  • Custom Execution Plan
  • Dynamic Shapes and Kernel Cache

Frontend Developer Guide

• Overview
  • Building and Running a cuDNN Graph Workflow
  • APIs
    • Creating the Graph
    • Defining Tensors
    • Defining Operations
    • Validating the Graph
    • Building the Backend Graph
    • Creating the Execution Plan
    • Getting the Execution Plan Count
    • Checking Graph Support
    • Building the Execution Plan
    • Filtering Plans (Optional)
    • Autotuning
    • Executing the Graph
    • Miscellaneous APIs
    • Serialization
    • Error Handling
  • Operations
• Core Concepts
  • cuDNN Handle
  • Tensors and Layouts
    • Tensor Descriptor
      • WXYZ Tensor Descriptor
      • 3-D Tensor Descriptor
      • 4-D Tensor Descriptor
      • 5-D Tensor Descriptor
      • Fully-Packed Tensors
      • Partially-Packed Tensors
      • Spatially Packed Tensors
      • Overlapping Tensors
    • Data Layout Formats
      • Example Tensor
      • Convolution Layouts
        • NCHW Memory Layout
        • NHWC Memory Layout
        • NC/32HW32 Memory Layout
      • Matmul Layouts
• Graphs
  • Key Concepts
    • Operations and Operation Graphs
    • Engines and Engine Configurations
    • Other Runtime Concepts
    • Heuristics
  • Supported Graph Patterns
    • Pre-Compiled Single Operation Engines
      • ConvolutionBwdData
      • ConvolutionBwdFilter
      • ConvolutionFwd
      • NormalizationBackward
      • NormalizationForward
    • Generic Runtime Fusion Engines
      • Support Surface
        • Support Surface 90
        • Support Surface 80
        • Support Surface 70
      • Operation Specific Constraints for the Runtime Fusion Engines
        • Matmul
        • Convolutions
        • Pointwise
        • GenStats
        • Reduction
        • ResampleFwd
        • Resampling Index Tensor Dump for Training
        • ResampleBwd
      • Examples of Supported Patterns
        • Single Operation
        • Pointwise Operations After Convolution 1
        • Pointwise Operations After Convolution 2
        • Pointwise Operations Before Matrix Multiplication
        • Convolution Producer Node in Middle of DAG
        • Mixed Input Precision Matmul and Convolution
    • Specialized Runtime Fusion Engines
      • BnAddRelu
      • DReluForkDBn
      • Fused Attention fprop
      • Fused Attention bprop
      • Fused Flash Attention fprop
      • Fused Flash Attention bprop
    • Specialized Pre-Compiled Engines
      • ConvBNfprop
      • ConvBNwgrad
      • ConvBiasAct
      • ConvScaleBiasAct
      • DgradDreluBNBwdWeight
      • FP8 Fused Flash Attention
  • Mapping with Backend Descriptors
• Hardware Forward Compatibility
  • Configuring Forward-Compatibility Mode
  • Forward-Compatibility and the Graph API
    • Supported Graph Patterns
• Odds and Ends
  • cuDNN Library Configuration
    • FULL Configuration
    • GRAPH_JIT_ONLY Configuration
    • Sub-Library Configuration Instructions
  • Thread Safety
  • cuDNN Backend Requirements
  • SM Carveout
  • Native CUDA Graph API
• Debugging
  • Debugging by Using Environment Variables
  • Debugging by Using API Calls

Reference

• Supported Products
• FAQs
• Support
• Software License Agreement
  • LICENSE AGREEMENT FOR NVIDIA SOFTWARE DEVELOPMENT KITS
  • License.
    • Grant
    • Distribution Requirements
    • Authorized Users
    • Pre-Release SDK
    • Updates
    • Components Under Other Licenses
    • Reservation of Rights
  • Limitations.
  • Ownership.
  • No Warranties.
  • Limitations of Liability.
  • Termination.
  • General.
  • cuDNN SUPPLEMENT TO SOFTWARE LICENSE AGREEMENT FOR NVIDIA SOFTWARE DEVELOPMENT KITS
• Acknowledgements
• Notices