> For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.nvidia.com/cudnn/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/cudnn/_mcp/server. # Normalizations ## Batchnorm Forward The batchnorm operation computes: $$ output = scale*{input - mean \over \sqrt{variance + epsilon}} + bias $$ Optionally the operation also computes: $$next\_running\_mean = (1 - momentum)*previous\_running\_mean + momentum*current\_running\_mean$$ $$next\_running\_variance = (1 - momentum)*previous\_running\_variance + momentum*current\_running\_variance$$ ### C++ API ``` std::array, 5> batchnorm(std::shared_ptr& input, std::shared_ptr& scale, std::shared_ptr& bias, Batchnorm_attributes attributes); ``` Where the output array has tensors in order of: `[output, saved_mean, saved_invariance, next_running_mean, next_running_variance]` Batchnorm attributes is a lightweight structure with setters for providing optional input tensors and other operation attributes: ``` Batchnorm_attributes& set_previous_running_stats(std::shared_ptr& previous_running_mean, std::shared_ptr& previous_running_variance, std::shared_ptr& momentum) Batchnorm_attributes& set_name(std::string const&) Batchnorm_attributes& set_compute_data_type(DataType_t value) ``` ### Python API - batchnorm - input - scale - bias - in_running_mean - in_running_var - epsilon - momentum - compute_data_type - name ## Batchnorm Finalize The `bn_finalize` operation calculates the statistics required for the next iteration from the statistics generated by the `genstat` operation. ### C++ API ``` std::array, 6> bn_finalize(std::shared_ptr, std::shared_ptr, std::shared_ptr, std::shared_ptr, std::shared_ptr, std::shared_ptr, BN_finalize_attributes); ``` The outputs are `[EQ_SCALE, EQ_BIAS, MEAN, INV_VARIANCE, NEXT_RUNNING_MEAN, NEXT_RUNNING_VAR]`. ## Batchnorm Backward (DBN) The DBN operation computes data gradient, scale gradient, bias gradient during backpropagation of batchnorm forward operation. ### C++ API ``` std::array, 3> batchnorm_backward(std::shared_ptr loss, std::shared_ptr input, std::shared_ptr scale, Batchnorm_backward_attributes); ``` The output array has tensors in order of: `[input gradient, scale gradient, bias gradient]`. DBN attributes is a lightweight structure with setters: ``` Batchnorm_backward_attributes& set_saved_mean_and_inv_variance(std::shared_ptr saved_mean, std::shared_ptr saved_inverse_variance) Batchnorm_backward_attributes& set_epsilon(std::shared_ptr epsilon) Batchnorm_backward_attributes& set_name(std::string const&) Batchnorm_backward_attributes& set_compute_data_type(DataType_t value) ``` Only setting either (saved mean and inverse_variance) or (epsilon) is necessary. ## Generate Stats The Genstats operation computes the sum and sum of squares per-channel dimension. ### C++ API ``` std::array, 2> cudnn_frontend::graph::genstats(std::shared_ptr, Genstats_attributes); ``` The output array has tensors in order of: `[sum, square_sum]` Genstats attributes is a lightweight structure with setters: ``` Genstats_attributes& set_name(std::string const&) Genstats_attributes& set_compute_data_type(DataType_t value) ``` ### Python API - genstats - input - compute_data_type - name ## Layernorm Forward The layernorm operation computes $$ output = scale*{input - mean \over \sqrt{variance + epsilon}} + bias $$ Normalization happens across features, independently for each sample. ### C++ API ``` std::array, 3> layernorm(std::shared_ptr& input, std::shared_ptr& scale, std::shared_ptr& bias, Layernorm_attributes attributes); ``` The output array has tensors in order of: `[output, mean, variance]` Layernorm_attributes is a lightweight structure with setters for providing optional input tensors and other operation attributes: ``` Layernorm_attributes& set_name(std::string const&) Layernorm_attributes& set_compute_data_type(DataType_t value) ``` ### Python API - layernorm - norm_forward_phase - input - scale - bias - epsilon - compute_data_type - name ### Experimental PyTorch API `cudnn.experimental.ops.layer_norm(input, normalized_shape, weight=None, bias=None, eps=1e-5)` provides a graph-backed PyTorch custom op with autograd and `torch.compile` support. It currently supports normalization over one final dimension on nonempty CUDA tensors with FP16, BF16, or FP32 activations. ## Layernorm Backward (DLN) The DLN operation computes data gradient, scale gradient, and bias gradient during backpropagation of a layernorm forward operation. ### C++ API ``` std::array, 3> layernorm_backward(std::shared_ptr dy, std::shared_ptr x, std::shared_ptr scale, Layernorm_backward_attributes options); ``` The output array has tensors in order of: `[input gradient, scale gradient, bias gradient]`. Layernorm_attributes is a lightweight structure with setters for providing optional input tensors and other operation attributes: ``` Layernorm_attributes& set_name(std::string const&) Layernorm_attributes& set_compute_data_type(DataType_t value) ``` ### Python API - layernorm - input - scale - loss - compute_data_type - name ## RMSNorm PyTorch API `cudnn.experimental.ops.rms_norm(input, weight, bias=None, eps=1e-5)` applies RMS normalization over the final input dimension using a cuDNN graph and supports autograd and `torch.compile`. This experimental signature intentionally includes an optional additive bias and requires a weight tensor. It is therefore not a drop-in replacement for `torch.nn.functional.rms_norm`, whose signature includes `normalized_shape` and no bias argument. Inputs, weights, and optional bias must be CUDA tensors with matching FP16, BF16, or FP32 dtypes, and the input must contain at least one normalization row. ## Adaptive Layernorm Forward The adaptive layernorm operation computes $$ output = scale*{input - mean \over \sqrt{variance + epsilon}} + bias $$ where the scale and bias vary across samples in a batch. Normalization happens across features, independently for each sample. ### C++ API ``` std::array, 3> adalayernorm(std::shared_ptr& input, std::shared_ptr& scale, std::shared_ptr& bias, Layernorm_attributes attributes); ``` The output array has tensors in order of: `[output, mean, variance]` AdaLayernorm_attributes is a lightweight structure with setters for providing optional input tensors and other operation attributes: ``` AdaLayernorm_attributes& set_name(std::string const&) AdaLayernorm_attributes& set_compute_data_type(DataType_t value) ``` ### Python API - adalayernorm - norm_forward_phase - input - scale - bias - epsilon - compute_data_type - name ## Adaptive Layernorm Backward (DADALN) The DADALN operation computes data gradient, scale gradient, and bias gradient during backpropagation of an adaptive layernorm forward operation. ### C++ API ``` std::array, 3> adalayernorm_backward(std::shared_ptr dy, std::shared_ptr x, std::shared_ptr scale, AdaLayernorm_backward_attributes options); ``` The output array has tensors in order of: `[input gradient, scale gradient, bias gradient]`. AdaLayernorm_backward_attributes is a lightweight structure with setters for providing optional input tensors and other operation attributes: ``` AdaLayernorm_backward_attributes& set_name(std::string const&) AdaLayernorm_backward_attributes& set_compute_data_type(DataType_t value) ``` ### Python API - adalayernorm_backward - dy - x - scale - mean - inv_variance - compute_data_type - name ## Instancenorm Forward The instancenorm operation computes $$ output = scale*{input - mean \over \sqrt{variance + epsilon}} + bias $$ Normalization happens across each sample. ### C++ API ``` std::array, 3> instancenorm(std::shared_ptr& input, std::shared_ptr& scale, std::shared_ptr& bias, Instancenorm_attributes attributes); ``` The output array has tensors in order of: `[output, mean, variance]`. Instancenorm_attributes is a lightweight structure with setters for providing optional input tensors and other operation attributes: ``` Instancenorm_attributes& set_name(std::string const&) Instancenorm_attributes& set_compute_data_type(DataType_t value) ``` ### Python API - instancenorm - norm_forward_phase - input - scale - bias - epsilon - compute_data_type - name ## Instancenorm Backward (DIN) The DIN operation computes data gradient, scale gradient, and bias gradient during backpropagation of an instancenorm forward operation. ### C++ API ``` std::array, 3> instancenorm_backward(std::shared_ptr dy, std::shared_ptr x, std::shared_ptr scale, Instancenorm_backward_attributes options); ``` The output array has tensors in order of: `[input gradient, scale gradient, bias gradient]`. Instancenorm_attributes is a lightweight structure with setters for providing optional input tensors and other operation attributes: ``` Instancenorm_attributes& set_name(std::string const&) Instancenorm_attributes& set_compute_data_type(DataType_t value) ``` ### Python API - layernorm - input - scale - loss - compute_data_type - name