cudnn_ops Library #

Data Type References #

These are the data type references in the cudnn_ops library.

Pointer To Opaque Struct Types #

These are the pointers to the opaque struct types in the cudnn_ops library.

cudnnActivationDescriptor_t #

This enumerated type is deprecated and is currently only used by deprecated APIs. Consider using replacements for the deprecated APIs that use this enumerated type.

cudnnActivationDescriptor_t is a pointer to an opaque structure holding the description of an activation operation. cudnnCreateActivationDescriptor() is used to create one instance, and cudnnSetActivationDescriptor() must be used to initialize this instance.

cudnnCTCLossDescriptor_t #

cudnnCTCLossDescriptor_t is a pointer to an opaque structure holding the description of a CTC loss operation. cudnnCreateCTCLossDescriptor() is used to create one instance, cudnnSetCTCLossDescriptor() is used to initialize this instance, and cudnnDestroyCTCLossDescriptor() is used to destroy this instance.

cudnnDropoutDescriptor_t #

cudnnDropoutDescriptor_t is a pointer to an opaque structure holding the description of a dropout operation. cudnnCreateDropoutDescriptor() is used to create one instance, cudnnSetDropoutDescriptor() is used to initialize this instance, cudnnDestroyDropoutDescriptor() is used to destroy this instance, cudnnGetDropoutDescriptor() is used to query fields of a previously initialized instance, cudnnRestoreDropoutDescriptor() is used to restore an instance to a previously saved off state.

cudnnFilterDescriptor_t #

This enumerated type is deprecated and is currently only used by deprecated APIs. Consider using replacements for the deprecated APIs that use this enumerated type.

cudnnFilterDescriptor_t is a pointer to an opaque structure holding the description of a filter dataset. cudnnCreateFilterDescriptor() is used to create one instance, and cudnnSetFilter4dDescriptor() or cudnnSetFilterNdDescriptor() must be used to initialize this instance.

cudnnLRNDescriptor_t #

cudnnLRNDescriptor_t is a pointer to an opaque structure holding the parameters of a local response normalization. cudnnCreateLRNDescriptor() is used to create one instance, and the routine cudnnSetLRNDescriptor() must be used to initialize this instance.

cudnnOpTensorDescriptor_t #

This enumerated type is deprecated and is currently only used by deprecated APIs. Consider using replacements for the deprecated APIs that use this enumerated type.

cudnnOpTensorDescriptor_t is a pointer to an opaque structure holding the description of a Tensor Core operation, used as a parameter to cudnnOpTensor(). cudnnCreateOpTensorDescriptor() is used to create one instance, and cudnnSetOpTensorDescriptor() must be used to initialize this instance.

cudnnPoolingDescriptor_t #

This enumerated type is deprecated and is currently only used by deprecated APIs. Consider using replacements for the deprecated APIs that use this enumerated type.

cudnnPoolingDescriptor_t is a pointer to an opaque structure holding the description of a pooling operation. cudnnCreatePoolingDescriptor() is used to create one instance, and cudnnSetPoolingNdDescriptor() or cudnnSetPooling2dDescriptor() must be used to initialize this instance.

cudnnReduceTensorDescriptor_t #

This enumerated type is deprecated and is currently only used by deprecated APIs. Consider using replacements for the deprecated APIs that use this enumerated type.

cudnnReduceTensorDescriptor_t is a pointer to an opaque structure holding the description of a tensor reduction operation, used as a parameter to cudnnReduceTensor(). cudnnCreateReduceTensorDescriptor() is used to create one instance, and cudnnSetReduceTensorDescriptor() must be used to initialize this instance.

cudnnSpatialTransformerDescriptor_t #

cudnnSpatialTransformerDescriptor_t is a pointer to an opaque structure holding the description of a spatial transformation operation. cudnnCreateSpatialTransformerDescriptor() is used to create one instance, cudnnSetSpatialTransformerNdDescriptor() is used to initialize this instance, and cudnnDestroySpatialTransformerDescriptor() is used to destroy this instance.

cudnnTensorDescriptor_t #

cudnnTensorDescriptor_t is a pointer to an opaque structure holding the description of a generic n-D dataset. cudnnCreateTensorDescriptor() is used to create one instance, and one of the routines cudnnSetTensorNdDescriptor(), cudnnSetTensor4dDescriptor(), or cudnnSetTensor4dDescriptorEx() must be used to initialize this instance.

cudnnTensorTransformDescriptor_t #

This enumerated type is deprecated and is currently only used by deprecated APIs. Consider using replacements for the deprecated APIs that use this enumerated type.

cudnnTensorTransformDescriptor_t is an opaque structure containing the description of the tensor transform. Use the cudnnCreateTensorTransformDescriptor() function to create an instance of this descriptor, and cudnnDestroyTensorTransformDescriptor() function to destroy a previously created instance.

Enumeration Types #

These are the enumeration types in the cudnn_ops library.

cudnnBatchNormMode_t #

This enumerated type is deprecated and is currently only used by deprecated APIs. Consider using replacements for the deprecated APIs that use this enumerated type.

cudnnBatchNormMode_t is an enumerated type used to specify the mode of operation in cudnnBatchNormalizationForwardInference(), cudnnBatchNormalizationForwardTraining(), cudnnBatchNormalizationBackward(), and cudnnDeriveBNTensorDescriptor() routines.

Values

CUDNN_BATCHNORM_PER_ACTIVATION

Normalization is performed per-activation. This mode is intended to be used after the non-convolutional network layers. In this mode, the tensor dimensions of bnBias and bnScale and the parameters used in the cudnnBatchNormalization* functions are 1xCxHxW.

CUDNN_BATCHNORM_SPATIAL

Normalization is performed over N+spatial dimensions. This mode is intended for use after convolutional layers (where spatial invariance is desired). In this mode, the bnBias and bnScale tensor dimensions are 1xCx1x1.

CUDNN_BATCHNORM_SPATIAL_PERSISTENT

This mode is similar to CUDNN_BATCHNORM_SPATIAL but it can be faster for some tasks.

An optimized path may be selected for CUDNN_DATA_FLOAT and CUDNN_DATA_HALF types, compute capability 6.0 or higher for the following two batch normalization API calls: cudnnBatchNormalizationForwardTraining(), and cudnnBatchNormalizationBackward(). In the case of cudnnBatchNormalizationBackward(), the savedMean and savedInvVariance arguments should not be NULL.

NCHW Mode Only This mode may use a scaled atomic integer reduction that is deterministic but imposes more restrictions on the input data range. When a numerical overflow occurs, the algorithm may produce NaN-s or Inf-s (infinity) in output buffers.

When Inf-s/NaN-s are present in the input data, the output in this mode is the same as from a pure floating-point implementation.

For finite but very large input values, the algorithm may encounter overflows more frequently due to a lower dynamic range and emit Inf-s/NaN-s while CUDNN_BATCHNORM_SPATIAL will produce finite results. The user can invoke cudnnQueryRuntimeError() to check if a numerical overflow occurred in this mode.

cudnnBatchNormOps_t #

This enumerated type is deprecated and is currently only used by deprecated APIs. Consider using replacements for the deprecated APIs that use this enumerated type.

cudnnBatchNormOps_t is an enumerated type used to specify the mode of operation in cudnnGetBatchNormalizationForwardTrainingExWorkspaceSize(), cudnnBatchNormalizationForwardTrainingEx(), cudnnGetBatchNormalizationBackwardExWorkspaceSize(), cudnnBatchNormalizationBackwardEx(), and cudnnGetBatchNormalizationTrainingExReserveSpaceSize() functions.

Values

CUDNN_BATCHNORM_OPS_BN: Only batch normalization is performed, per-activation.
CUDNN_BATCHNORM_OPS_BN_ACTIVATION: First, the batch normalization is performed, and then the activation is performed.
CUDNN_BATCHNORM_OPS_BN_ADD_ACTIVATION: Performs the batch normalization, then element-wise addition, followed by the activation operation.

cudnnConvolutionBwdDataAlgo_t #

cudnnConvolutionBwdDataAlgo_t is an enumerated type that exposes the different algorithms available to execute the backward data convolution operation.

Values

CUDNN_CONVOLUTION_BWD_DATA_ALGO_0: This algorithm expresses the convolution as a sum of matrix products without actually explicitly forming the matrix that holds the input tensor data. The sum is done using the atomic add operation, thus the results are non-deterministic.
CUDNN_CONVOLUTION_BWD_DATA_ALGO_1: This algorithm expresses the convolution as a matrix product without actually explicitly forming the matrix that holds the input tensor data. The results are deterministic.
CUDNN_CONVOLUTION_BWD_DATA_ALGO_FFT: This algorithm uses a Fast-Fourier Transform approach to compute the convolution. A significant memory workspace is needed to store intermediate results. The results are deterministic.
CUDNN_CONVOLUTION_BWD_DATA_ALGO_FFT_TILING: This algorithm uses the Fast-Fourier Transform approach but splits the inputs into tiles. A significant memory workspace is needed to store intermediate results but less than CUDNN_CONVOLUTION_BWD_DATA_ALGO_FFT for large size images. The results are deterministic.
CUDNN_CONVOLUTION_BWD_DATA_ALGO_WINOGRAD: This algorithm uses the Winograd Transform approach to compute the convolution. A reasonably sized workspace is needed to store intermediate results. The results are deterministic.

Note

The CUDNN_CONVOLUTION_BWD_DATA_ALGO_WINOGRAD algorithm is not supported on GPUs based on the NVIDIA Hopper GPU architecture and later GPU architectures.
CUDNN_CONVOLUTION_BWD_DATA_ALGO_WINOGRAD_NONFUSED: This algorithm uses the Winograd Transform approach to compute the convolution. A significant workspace may be needed to store intermediate results. The results are deterministic.

cudnnConvolutionBwdFilterAlgo_t #

cudnnConvolutionBwdFilterAlgo_t is an enumerated type that exposes the different algorithms available to execute the backward filter convolution operation.

Values

CUDNN_CONVOLUTION_BWD_FILTER_ALGO_0: This algorithm expresses the convolution as a sum of matrix products without actually explicitly forming the matrix that holds the input tensor data. The sum is done using the atomic add operation, thus the results are non-deterministic.
CUDNN_CONVOLUTION_BWD_FILTER_ALGO_1: This algorithm expresses the convolution as a matrix product without actually explicitly forming the matrix that holds the input tensor data. The results are deterministic.
CUDNN_CONVOLUTION_BWD_FILTER_ALGO_FFT: This algorithm uses the Fast-Fourier Transform approach to compute the convolution. A significant workspace is needed to store intermediate results. The results are deterministic.
CUDNN_CONVOLUTION_BWD_FILTER_ALGO_3: This algorithm is similar to CUDNN_CONVOLUTION_BWD_FILTER_ALGO_0 but uses some small workspace to precompute some indices. The results are also non-deterministic.
CUDNN_CONVOLUTION_BWD_FILTER_WINOGRAD_NONFUSED: This algorithm uses the Winograd Transform approach to compute the convolution. A significant workspace may be needed to store intermediate results. The results are deterministic.
CUDNN_CONVOLUTION_BWD_FILTER_ALGO_FFT_TILING: This algorithm uses the Fast-Fourier Transform approach to compute the convolution but splits the input tensor into tiles. A significant workspace may be needed to store intermediate results. The results are deterministic.

cudnnConvolutionFwdAlgo_t #

cudnnConvolutionFwdAlgo_t is an enumerated type that exposes the different algorithms available to execute the forward convolution operation.

Values

CUDNN_CONVOLUTION_FWD_ALGO_IMPLICIT_GEMM: This algorithm expresses the convolution as a matrix product without actually explicitly forming the matrix that holds the input tensor data.
CUDNN_CONVOLUTION_FWD_ALGO_IMPLICIT_PPRECOMP_GEMM: This algorithm expresses the convolution as a matrix product without actually explicitly forming the matrix that holds the input tensor data, but still needs some memory workspace to precompute some indices in order to facilitate the implicit construction of the matrix that holds the input tensor data.
CUDNN_CONVOLUTION_FWD_ALGO_GEMM: This algorithm expresses convolution as an explicit matrix product. A significant memory workspace is needed to store the matrix that holds the input tensor data.
CUDNN_CONVOLUTION_FWD_ALGO_DIRECT: This algorithm expresses the convolution as a direct convolution (for example, without implicitly or explicitly doing a matrix multiplication).
CUDNN_CONVOLUTION_FWD_ALGO_FFT: This algorithm uses the Fast-Fourier Transform approach to compute the convolution. A significant memory workspace is needed to store intermediate results.
CUDNN_CONVOLUTION_FWD_ALGO_FFT_TILING: This algorithm uses the Fast-Fourier Transform approach but splits the inputs into tiles. A significant memory workspace is needed to store intermediate results but less than CUDNN_CONVOLUTION_FWD_ALGO_FFT for large size images.
CUDNN_CONVOLUTION_FWD_ALGO_WINOGRAD: This algorithm uses the Winograd Transform approach to compute the convolution. A reasonably sized workspace is needed to store intermediate results.

Note

The CUDNN_CONVOLUTION_FWD_ALGO_WINOGRAD algorithm is not supported on GPUs based on the NVIDIA Hopper GPU architecture and later GPU architectures.
CUDNN_CONVOLUTION_FWD_ALGO_WINOGRAD_NONFUSED: This algorithm uses the Winograd Transform approach to compute the convolution. A significant workspace may be needed to store intermediate results.

cudnnCTCLossAlgo_t #

cudnnCTCLossAlgo_t is an enumerated type that exposes the different algorithms available to execute the CTC loss operation.

Values

CUDNN_CTC_LOSS_ALGO_DETERMINISTIC: Results are guaranteed to be reproducible.
CUDNN_CTC_LOSS_ALGO_NON_DETERMINISTIC: Results are not guaranteed to be reproducible.

cudnnDeterminism_t #

cudnnDeterminism_t is an enumerated type used to indicate if the computed results are deterministic (reproducible). For more information, refer to Reproducibility (Determinism).

Values

CUDNN_NON_DETERMINISTIC: Results are not guaranteed to be reproducible.
CUDNN_DETERMINISTIC: Results are guaranteed to be reproducible.

cudnnDivNormMode_t #

cudnnDivNormMode_t is an enumerated type used to specify the mode of operation in cudnnDivisiveNormalizationForward() and cudnnDivisiveNormalizationBackward().

Values

CUDNN_DIVNORM_PRECOMPUTED_MEANS

The means tensor data pointer is expected to contain means or other kernel convolution values precomputed by the user. The means pointer can also be NULL, in that case, it’s considered to be filled with zeroes. This is equivalent to spatial LRN.

In the backward pass, the means are treated as independent inputs and the gradient over means is computed independently. In this mode, to yield a net gradient over the entire LCN computational graph, the destDiffMeans result should be backpropagated through the user’s means layer (which can be implemented using average pooling) and added to the destDiffData tensor produced by cudnnDivisiveNormalizationBackward().

cudnnFoldingDirection_t #

cudnnFoldingDirection_t is an enumerated type used to select the folding direction. For more information, refer to cudnnTensorTransformDescriptor_t.

Data Member

CUDNN_TRANSFORM_FOLD = 0U: Selects folding.
CUDNN_TRANSFORM_UNFOLD = 1U: Selects unfolding.

cudnnIndicesType_t #

This enumerated type is deprecated and is currently only used by deprecated APIs. Consider using replacements for the deprecated APIs that use this enumerated type.

cudnnIndicesType_t is an enumerated type used to indicate the data type for the indices to be computed by the cudnnReduceTensor() routine. This enumerated type is used as a field for the cudnnReduceTensorDescriptor_t descriptor.

Values

CUDNN_32BIT_INDICES: Compute unsigned int indices.
CUDNN_64BIT_INDICES: Compute unsigned long indices.
CUDNN_16BIT_INDICES: Compute unsigned short indices.
CUDNN_8BIT_INDICES: Compute unsigned char indices.

cudnnLRNMode_t #

cudnnLRNMode_t is an enumerated type used to specify the mode of operation in cudnnLRNCrossChannelForward() and cudnnLRNCrossChannelBackward().

Values

CUDNN_LRN_CROSS_CHANNEL_DIM1: LRN computation is performed across the tensor’s dimension dimA[1].

cudnnNormAlgo_t #

This enumerated type is deprecated and is currently only used by deprecated APIs. Consider using replacements for the deprecated APIs that use this enumerated type.

cudnnNormAlgo_t is an enumerated type used to specify the algorithm to execute the normalization operation.

Values

CUDNN_NORM_ALGO_STANDARD

Standard normalization is performed.

CUDNN_NORM_ALGO_PERSIST

This mode is similar to CUDNN_NORM_ALGO_STANDARD, however it only supports CUDNN_NORM_PER_CHANNEL and can be faster for some tasks.

An optimized path may be selected for CUDNN_DATA_FLOAT and CUDNN_DATA_HALF types, compute capability 6.0 or higher for the following two normalization API calls: cudnnNormalizationForwardTraining() and cudnnNormalizationBackward(). In the case of cudnnNormalizationBackward(), the savedMean and savedInvVariance arguments should not be NULL.

NCHW Mode Only This mode may use a scaled atomic integer reduction that is deterministic but imposes more restrictions on the input data range. When a numerical overflow occurs, the algorithm may produce NaN-s or Inf-s (infinity) in output buffers.

When Inf-s/NaN-s are present in the input data, the output in this mode is the same as from a pure floating-point implementation.

For finite but very large input values, the algorithm may encounter overflows more frequently due to a lower dynamic range and emit Inf-s/NaN-s while CUDNN_NORM_ALGO_STANDARD will produce finite results. The user can invoke cudnnQueryRuntimeError() to check if a numerical overflow occurred in this mode.

cudnnNormMode_t #

This enumerated type is deprecated and is currently only used by deprecated APIs. Consider using replacements for the deprecated APIs that use this enumerated type.

cudnnNormMode_t is an enumerated type used to specify the mode of operation in cudnnNormalizationForwardInference(), cudnnNormalizationForwardTraining(), cudnnBatchNormalizationBackward(), cudnnGetNormalizationForwardTrainingWorkspaceSize(), cudnnGetNormalizationBackwardWorkspaceSize(), and cudnnGetNormalizationTrainingReserveSpaceSize() routines.

Values

CUDNN_NORM_PER_ACTIVATION: Normalization is performed per-activation. This mode is intended to be used after the non-convolutional network layers. In this mode, the tensor dimensions of normBias and normScale and the parameters used in the cudnnNormalization* functions are 1xCxHxW.
CUDNN_NORM_PER_CHANNEL: Normalization is performed per-channel over N+spatial dimensions. This mode is intended for use after convolutional layers (where spatial invariance is desired). In this mode, the normBias and normScale tensor dimensions are 1xCx1x1.

cudnnNormOps_t #

This enumerated type is deprecated and is currently only used by deprecated APIs. Consider using replacements for the deprecated APIs that use this enumerated type.

cudnnNormOps_t is an enumerated type used to specify the mode of operation in cudnnGetNormalizationForwardTrainingWorkspaceSize(), cudnnNormalizationForwardTraining(), cudnnGetNormalizationBackwardWorkspaceSize(), cudnnNormalizationBackward(), and cudnnGetNormalizationTrainingReserveSpaceSize() functions.

Values

CUDNN_NORM_OPS_NORM: Only normalization is performed.
CUDNN_NORM_OPS_NORM_ACTIVATION: First, the normalization is performed, then the activation is performed.
CUDNN_NORM_OPS_NORM_ADD_ACTIVATION: Performs the normalization, then element-wise addition, followed by the activation operation.

cudnnOpTensorOp_t #

cudnnOpTensorOp_t is an enumerated type used to indicate the Tensor Core operation to be used by the cudnnOpTensor() routine. This enumerated type is used as a field for the cudnnOpTensorDescriptor_t descriptor.

Values

CUDNN_OP_TENSOR_ADD: The operation to be performed is addition.
CUDNN_OP_TENSOR_MUL: The operation to be performed is multiplication.
CUDNN_OP_TENSOR_MIN: The operation to be performed is a minimum comparison.
CUDNN_OP_TENSOR_MAX: The operation to be performed is a maximum comparison.
CUDNN_OP_TENSOR_SQRT: The operation to be performed is square root, performed on only the A tensor.
CUDNN_OP_TENSOR_NOT: The operation to be performed is negation, performed on only the A tensor.

cudnnPoolingMode_t #

This enumerated type is deprecated and is currently only used by deprecated APIs. Consider using replacements for the deprecated APIs that use this enumerated type.

cudnnPoolingMode_t is an enumerated type passed to cudnnSetPooling2dDescriptor() to select the pooling method to be used by cudnnPoolingForward() and cudnnPoolingBackward().

Values

CUDNN_POOLING_MAX: The maximum value inside the pooling window is used.
CUDNN_POOLING_AVERAGE_COUNT_INCLUDE_PADDING: Values inside the pooling window are averaged. The number of elements used to calculate the average includes spatial locations falling in the padding region.
CUDNN_POOLING_AVERAGE_COUNT_EXCLUDE_PADDING: Values inside the pooling window are averaged. The number of elements used to calculate the average excludes spatial locations falling in the padding region.
CUDNN_POOLING_MAX_DETERMINISTIC: The maximum value inside the pooling window is used. The algorithm used is deterministic.

cudnnReduceTensorIndices_t #

This enumerated type is deprecated and is currently only used by deprecated APIs. Consider using replacements for the deprecated APIs that use this enumerated type.

cudnnReduceTensorIndices_t is an enumerated type used to indicate whether indices are to be computed by the cudnnReduceTensor() routine. This enumerated type is used as a field for the cudnnReduceTensorDescriptor_t descriptor.

Values

CUDNN_REDUCE_TENSOR_NO_INDICES: Do not compute indices.
CUDNN_REDUCE_TENSOR_FLATTENED_INDICES: Compute indices. The resulting indices are relative, and flattened.

cudnnSamplerType_t #

cudnnSamplerType_t is an enumerated type passed to cudnnSetSpatialTransformerNdDescriptor() to select the sampler type to be used by cudnnSpatialTfSamplerForward() and cudnnSpatialTfSamplerBackward().

Values

CUDNN_SAMPLER_BILINEAR: Selects the bilinear sampler.

cudnnSoftmaxAlgorithm_t #

cudnnSoftmaxAlgorithm_t is used to select an implementation of the softmax function used in cudnnSoftmaxForward() and cudnnSoftmaxBackward().

Values

CUDNN_SOFTMAX_FAST: This implementation applies the straightforward softmax operation.
CUDNN_SOFTMAX_ACCURATE: This implementation scales each point of the softmax input domain by its maximum value to avoid potential floating point overflows in the softmax evaluation.
CUDNN_SOFTMAX_LOG: This entry performs the log softmax operation, avoiding overflows by scaling each point in the input domain as in CUDNN_SOFTMAX_ACCURATE.

cudnnSoftmaxMode_t #

cudnnSoftmaxMode_t is used to select over which data the cudnnSoftmaxForward() and cudnnSoftmaxBackward() are computing their results.

Values

CUDNN_SOFTMAX_MODE_INSTANCE The softmax operation is computed per image (N) across the dimensions C,H,W.

CUDNN_SOFTMAX_MODE_CHANNEL The softmax operation is computed per spatial location (H,W) per image (N) across dimension C.

API Functions #

These are the API functions in the cudnn_ops library.

cudnnActivationBackward()#

This function has been deprecated in cuDNN 9.0.

This routine computes the gradient of a neuron activation function.

cudnnStatus_t cudnnActivationBackward(
    cudnnHandle_t                    handle,
    cudnnActivationDescriptor_t      activationDesc,
    const void                      *alpha,
    const cudnnTensorDescriptor_t    yDesc,
    const void                      *y,
    const cudnnTensorDescriptor_t    dyDesc,
    const void                      *dy,
    const cudnnTensorDescriptor_t    xDesc,
    const void                      *x,
    const void                      *beta,
    const cudnnTensorDescriptor_t    dxDesc,
    void                            *dx)

In-place operation is allowed for this routine; meaning dy and dx pointers may be equal. However, this requires the corresponding tensor descriptors to be identical (particularly, the strides of the input and output must match for an in-place operation to be allowed).

All tensor formats are supported for 4 and 5 dimensions, however, the best performance is obtained when the strides of yDesc and xDesc are equal and HW-packed. For more than 5 dimensions the tensors must have their spatial dimensions packed.

Parameters

handle

Input. Handle to a previously created cuDNN context. For more information, refer to cudnnHandle_t.

activationDesc

Input. Activation descriptor. For more information, refer to cudnnActivationDescriptor_t.

alpha, beta

Input. Pointers to scaling factors (in host memory) used to blend the computation result with prior value in the output layer as follows:

dstValue = alpha[0]*result + beta[0]*priorDstValue

For more information, refer to Scaling Parameters.

yDesc

Input. Handle to the previously initialized input tensor descriptor. For more information, refer to cudnnTensorDescriptor_t.

y

Input. Data pointer to GPU memory associated with the tensor descriptor yDesc.

dyDesc

Input. Handle to the previously initialized input differential tensor descriptor.

dy

Input. Data pointer to GPU memory associated with the tensor descriptor dyDesc.

xDesc

Input. Handle to the previously initialized output tensor descriptor.

x

Input. Data pointer to GPU memory associated with the output tensor descriptor xDesc.

dxDesc

Input. Handle to the previously initialized output differential tensor descriptor.

dx

Output. Data pointer to GPU memory associated with the output tensor descriptor dxDesc.

Returns

CUDNN_STATUS_SUCCESS

The function launched successfully.

CUDNN_STATUS_BAD_PARAM

At least one of the following conditions are met:

The strides nStride, cStride, hStride, wStride of the input differential tensor and output differential tensor differ and in-place operation is used.

CUDNN_STATUS_NOT_SUPPORTED

The function does not support the provided configuration. Refer to the following for some examples of non-supported configurations:

The dimensions n, c, h, and w of the input tensor and output tensor differ.

The datatype of the input tensor and output tensor differs.

The strides nStride, cStride, hStride, and wStride of the input tensor and the input differential tensor differ.

The strides nStride, cStride, hStride, and wStride of the output tensor and the output differential tensor differ.

CUDNN_STATUS_EXECUTION_FAILED

The function failed to launch on the GPU.

cudnnActivationForward()#

This function has been deprecated in cuDNN 9.0.

This routine applies a specified neuron activation function element-wise over each input value.

cudnnStatus_t cudnnActivationForward(
    cudnnHandle_t handle,
    cudnnActivationDescriptor_t     activationDesc,
    const void                     *alpha,
    const cudnnTensorDescriptor_t   xDesc,
    const void                     *x,
    const void                     *beta,
    const cudnnTensorDescriptor_t   yDesc,
    void                           *y)

In-place operation is allowed for this routine; meaning, xData and yData pointers may be equal. However, this requires xDesc and yDesc descriptors to be identical (particularly, the strides of the input and output must match for an in-place operation to be allowed).

All tensor formats are supported for 4 and 5 dimensions, however, the best performance is obtained when the strides of xDesc and yDesc are equal and HW-packed. For more than 5 dimensions the tensors must have their spatial dimensions packed.

Parameters

handle

Input. Handle to a previously created cuDNN context. For more information, refer to cudnnHandle_t.

activationDesc

Input. Activation descriptor. For more information, refer to cudnnActivationDescriptor_t.

alpha, beta

Input. Pointers to scaling factors (in host memory) used to blend the computation result with prior value in the output layer as follows:

dstValue = alpha[0]*result + beta[0]*priorDstValue

For more information, refer to Scaling Parameters.

xDesc

Input. Handle to the previously initialized input tensor descriptor. For more information, refer to cudnnTensorDescriptor_t.

x

Input. Data pointer to GPU memory associated with the tensor descriptor xDesc.

yDesc

Input. Handle to the previously initialized input tensor descriptor.

y

Input. Data pointer to GPU memory associated with the tensor descriptor yDesc.

Returns

CUDNN_STATUS_SUCCESS

The function launched successfully.

CUDNN_STATUS_NOT_SUPPORTED

The function does not support the provided configuration.

CUDNN_STATUS_BAD_PARAM

At least one of the following conditions are met:

The parameter mode has an invalid enumerant value.

The dimensions n, c, h, w of the input tensor and output tensor differ.

The datatype of the input tensor and output tensor differs.

The strides nStride, cStride, hStride, wStride of the input tensor and output tensor differ and in-place operation is used (meaning, x and y pointers are equal).

CUDNN_STATUS_EXECUTION_FAILED

The function failed to launch on the GPU.

cudnnAddTensor()#

This function has been deprecated in cuDNN 9.0.

This function adds the scaled values of a bias tensor to another tensor. Each dimension of the bias tensor A must match the corresponding dimension of the destination tensor C or must be equal to 1. In the latter case, the same value from the bias tensor for those dimensions will be used to blend into the C tensor.

cudnnStatus_t cudnnAddTensor(
    cudnnHandle_t                     handle,
    const void                       *alpha,
    const cudnnTensorDescriptor_t     aDesc,
    const void                       *A,
    const void                       *beta,
    const cudnnTensorDescriptor_t     cDesc,
    void                             *C)

Only 4D and 5D tensors are supported. Beyond these dimensions, this routine is not supported.

Parameters

handle

Input. Handle to a previously created cuDNN context. For more information, refer to cudnnHandle_t.

alpha, beta

Input. Pointers to scaling factors (in host memory) used to blend the source value with the prior value in the destination tensor as follows:

dstValue = alpha[0]*srcValue + beta[0]*priorDstValue

For more information, refer to Scaling Parameters.

aDesc

Input. Handle to a previously initialized tensor descriptor. For more information, refer to cudnnTensorDescriptor_t.

A

Input. Pointer to data of the tensor described by the aDesc descriptor.

cDesc

Input. Handle to a previously initialized tensor descriptor.

C

Input/Output. Pointer to data of the tensor described by the cDesc descriptor.

Returns

CUDNN_STATUS_SUCCESS: The function executed successfully.
CUDNN_STATUS_NOT_SUPPORTED: The function does not support the provided configuration.
CUDNN_STATUS_BAD_PARAM: The dimensions of the bias tensor refer to an amount of data that is incompatible with the output tensor dimensions or the dataType of the two tensor descriptors are different.
CUDNN_STATUS_EXECUTION_FAILED: The function failed to launch on the GPU.

cudnnBatchNormalizationBackward()#

This function has been deprecated in cuDNN 9.0.

This function performs the backward batch normalization layer computation. This layer is based on the Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift paper.

cudnnStatus_t cudnnBatchNormalizationBackward(
    cudnnHandle_t                    handle,
    cudnnBatchNormMode_t             mode,
    const void                      *alphaDataDiff,
    const void                      *betaDataDiff,
    const void                      *alphaParamDiff,
    const void                      *betaParamDiff,
    const cudnnTensorDescriptor_t    xDesc,
    const void                      *x,
    const cudnnTensorDescriptor_t    dyDesc,
    const void                      *dy,
    const cudnnTensorDescriptor_t    dxDesc,
    void                            *dx,
    const cudnnTensorDescriptor_t    bnScaleBiasDiffDesc,
    const void                      *bnScale,
    void                            *resultBnScaleDiff,
    void                            *resultBnBiasDiff,
    double                           epsilon,
    const void                      *savedMean,
    const void                      *savedInvVariance)

Only 4D and 5D tensors are supported.

The epsilon value has to be the same during training, backpropagation, and inference.

Higher performance can be obtained when HW-packed tensors are used for all of x, dy, and dx.

For more information, refer to cudnnDeriveBNTensorDescriptor() for the secondary tensor descriptor generation for the parameters used in this function.

Parameters

handle

Input. Handle to a previously created cuDNN library descriptor. For more information, refer to cudnnHandle_t.

mode

Input. Mode of operation (spatial or per-activation). For more information, refer to cudnnBatchNormMode_t.

*alphaDataDiff, *betaDataDiff

Inputs. Pointers to scaling factors (in host memory) used to blend the gradient output dx with a prior value in the destination tensor as follows:

dstValue = alphaDataDiff[0]*resultValue + betaDataDiff[0]*priorDstValue

For more information, refer to Scaling Parameters.

*alphaParamDiff, *betaParamDiff

Inputs. Pointers to scaling factors (in host memory) used to blend the gradient outputs resultBnScaleDiff and resultBnBiasDiff with prior values in the destination tensor as follows:

dstValue = alphaParamDiff[0]*resultValue + betaParamDiff[0]*priorDstValue

For more information, refer to Scaling Parameters.

xDesc, dxDesc, dyDesc

Inputs. Handles to the previously initialized tensor descriptors.

*x

Inputs. Data pointer to GPU memory associated with the tensor descriptor xDesc, for the layer’s x data.

*dy

Inputs. Data pointer to GPU memory associated with the tensor descriptor dyDesc, for the backpropagated differential dy input.

*dx

Inputs/Outputs. Data pointer to GPU memory associated with the tensor descriptor dxDesc, for the resulting differential output with respect to x.

bnScaleBiasDiffDesc

Input. Shared tensor descriptor for the following five tensors: bnScale, resultBnScaleDiff, resultBnBiasDiff, savedMean, and savedInvVariance. The dimensions for this tensor descriptor are dependent on normalization mode. For more information, refer to cudnnDeriveBNTensorDescriptor().

Note

The data type of this tensor descriptor must be float for FP16 and FP32 input tensors, and double for FP64 input tensors.

*bnScale

Input. Pointer in the device memory for the batch normalization scale parameter (in the original paper the quantity scale is referred to as gamma).

Note

The bnBias parameter is not needed for this layer’s computation.

resultBnScaleDiff, resultBnBiasDiff

Outputs. Pointers in device memory for the resulting scale and bias differentials computed by this routine. Note that these scale and bias gradients are weight gradients specific to this batch normalization operation, and by definition are not backpropagated.

epsilon

Input. Epsilon value used in batch normalization formula. Its value should be equal to or greater than the value defined for CUDNN_BN_MIN_EPSILON in cudnn.h. The same epsilon value should be used in forward and backward functions.

*savedMean, *savedInvVariance

Inputs. Optional cache parameters containing saved intermediate results that were computed during the forward pass. For this to work correctly, the layer’s x and bnScale data have to remain unchanged until this backward function is called.

Note

Both these parameters can be NULL but only at the same time. It is recommended to use this cache since the memory overhead is relatively small.

Supported Configurations

This function supports the following combinations of data types for various descriptors.

Supported Configurations for `cudnnBatchNormalizationBackward()`#
Data Type Configurations Supported	`xDesc` Data Type	`bnScaleBiasMeanVarDesc` Data Type	`alpha`, `beta` Data Type	`yDesc` Data Type
`PSEUDO_HALF_CONFIG`	`CUDNN_DATA_HALF`	`CUDNN_DATA_FLOAT`	`CUDNN_DATA_FLOAT`	`CUDNN_DATA_HALF`
`FLOAT_CONFIG`	`CUDNN_DATA_FLOAT`	`CUDNN_DATA_FLOAT`	`CUDNN_DATA_FLOAT`	`CUDNN_DATA_FLOAT`
`DOUBLE_CONFIG`	`CUDNN_DATA_DOUBLE`	`CUDNN_DATA_DOUBLE`	`CUDNN_DATA_DOUBLE`	`CUDNN_DATA_DOUBLE`
`PSEUDO_BFLOAT16_CONFIG`	`CUDNN_DATA_BFLOAT16`	`CUDNN_DATA_FLOAT`	`CUDNN_DATA_FLOAT`	`CUDNN_DATA_BFLOAT16`

Returns

CUDNN_STATUS_SUCCESS

The computation was performed successfully.

CUDNN_STATUS_NOT_SUPPORTED

The function does not support the provided configuration.

CUDNN_STATUS_BAD_PARAM

At least one of the following conditions are met:

Any of the pointers alpha, beta, x, dy, dx, bnScale, resultBnScaleDiff, and resultBnBiasDiff is NULL.

The number of xDesc, yDesc, or dxDesc tensor descriptor dimensions is not within the range of [4,5] (only 4D and 5D tensors are supported).

bnScaleBiasDiffDesc dimensions are not 1xCx1x1 for 4D and 1xCx1x1x1 for 5D for spatial, and are not 1xCxHxW for 4D and 1xCxDxHxW for 5D for per-activation mode.

Exactly one of savedMean, savedInvVariance pointers is NULL.

epsilon value is less than CUDNN_BN_MIN_EPSILON.

Dimensions or data types mismatch for any pair of xDesc, dyDesc, or dxDesc.

cudnnBatchNormalizationBackwardEx()#

This function has been deprecated in cuDNN 9.0.

This function is an extension of the cudnnBatchNormalizationBackward() for performing the backward batch normalization layer computation with a fast NHWC semi-persistent kernel.

cudnnStatus_t cudnnBatchNormalizationBackwardEx (
    cudnnHandle_t                       handle,
    cudnnBatchNormMode_t                mode,
    cudnnBatchNormOps_t                 bnOps,
    const void                          *alphaDataDiff,
    const void                          *betaDataDiff,
    const void                          *alphaParamDiff,
    const void                          *betaParamDiff,
    const cudnnTensorDescriptor_t       xDesc,
    const void                          *xData,
    const cudnnTensorDescriptor_t       yDesc,
    const void                          *yData,
    const cudnnTensorDescriptor_t       dyDesc,
    const void                          *dyData,
    const cudnnTensorDescriptor_t       dzDesc,
    void                                *dzData,
    const cudnnTensorDescriptor_t       dxDesc,
    void                                *dxData,
    const cudnnTensorDescriptor_t       dBnScaleBiasDesc,
    const void                          *bnScaleData,
    const void                          *bnBiasData,
    void                                *dBnScaleData,
    void                                *dBnBiasData,
    double                              epsilon,
    const void                          *savedMean,
    const void                          *savedInvVariance,
    const cudnnActivationDescriptor_t   activationDesc,
    void                                *workspace,
    size_t                              workSpaceSizeInBytes
    void                                *reserveSpace
    size_t                              reserveSpaceSizeInBytes);

This API will trigger the new semi-persistent NHWC kernel when the following conditions are true:

All tensors, namely, x, y, dz, dy, and dx must be NHWC-fully packed, and must be of the type CUDNN_DATA_HALF.

The input parameter mode must be set to CUDNN_BATCHNORM_SPATIAL_PERSISTENT.

Before cuDNN version 8.2.0, the tensor C dimension should always be a multiple of 4. After 8.2.0, the tensor C dimension should be a multiple of 4 only when bnOps is CUDNN_BATCHNORM_OPS_BN_ADD_ACTIVATION.

workspace is not NULL.

workSpaceSizeInBytes is equal to or larger than the amount required by cudnnGetBatchNormalizationBackwardExWorkspaceSize().

reserveSpaceSizeInBytes is equal to or larger than the amount required by cudnnGetBatchNormalizationTrainingExReserveSpaceSize().

The content in reserveSpace stored by cudnnBatchNormalizationForwardTrainingEx() must be preserved.

If workspace is NULL and workSpaceSizeInBytes of zero is passed in, this API will function exactly like the non-extended function cudnnBatchNormalizationBackward().

This workspace is not required to be clean. Moreover, the workspace does not have to remain unchanged between the forward and backward pass, as it is not used for passing any information.

This extended function can accept a *workspace pointer to the GPU workspace, and workSpaceSizeInBytes, the size of the workspace, from the user.

The bnOps input can be used to set this function to perform either only the batch normalization, or batch normalization followed by activation, or batch normalization followed by element-wise addition and then activation.

Only 4D and 5D tensors are supported. The epsilon value has to be the same during the training, the backpropagation, and the inference.

When the tensor layout is NCHW, higher performance can be obtained when HW-packed tensors are used for x, dy, and dx.

Parameters

handle

Input. Handle to a previously created cuDNN library descriptor. For more information, refer to cudnnHandle_t.

mode

Input. Mode of operation (spatial or per-activation). For more information, refer to cudnnBatchNormMode_t.

bnOps

Input. Mode of operation. Currently, CUDNN_BATCHNORM_OPS_BN_ACTIVATION and CUDNN_BATCHNORM_OPS_BN_ADD_ACTIVATION are only supported in the NHWC layout. For more information, refer to cudnnBatchNormOps_t. This input can be used to set this function to perform either only the batch normalization, or batch normalization followed by activation, or batch normalization followed by element-wise addition and then activation.

*alphaDataDiff, *betaDataDiff

Inputs. Pointers to scaling factors (in host memory) used to blend the gradient output dx with a prior value in the destination tensor as follows:

dstValue = alpha[0]*resultValue + beta[0]*priorDstValue

For more information, refer to Scaling Parameters.

*alphaParamDiff, *betaParamDiff

Inputs. Pointers to scaling factors (in host memory) used to blend the gradient outputs dBnScaleData and dBnBiasData with prior values in the destination tensor as follows:

dstValue = alpha[0]*resultValue + beta[0]*priorDstValue

For more information, refer to Scaling Parameters.

xDesc, *x, yDesc, *yData, dyDesc, *dyData

Inputs. Tensor descriptors and pointers in the device memory for the layer’s x data, backpropagated gradient input dy, the original forward output y data. yDesc and yData are not needed if bnOps is set to CUDNN_BATCHNORM_OPS_BN, users may pass NULL. For more information, refer to cudnnTensorDescriptor_t.

dzDesc, dxDesc

Inputs. Tensor descriptors and pointers in the device memory for the computed gradient output dz, and dx. dzDesc is not needed when bnOps is CUDNN_BATCHNORM_OPS_BN or CUDNN_BATCHNORM_OPS_BN_ACTIVATION, users may pass NULL. For more information, refer to cudnnTensorDescriptor_t.

*dzData, *dxData

Outputs. Tensor descriptors and pointers in the device memory for the computed gradient output dz, and dx. *dzData is not needed when bnOps is CUDNN_BATCHNORM_OPS_BN or CUDNN_BATCHNORM_OPS_BN_ACTIVATION, users may pass NULL. For more information, refer to cudnnTensorDescriptor_t.

dBnScaleBiasDesc

Input. Shared tensor descriptor for the following six tensors: bnScaleData, bnBiasData, dBnScaleData, dBnBiasData, savedMean, and savedInvVariance. For more information, refer to cudnnDeriveBNTensorDescriptor().

The dimensions for this tensor descriptor are dependent on normalization mode.

Note

The data type of this tensor descriptor must be float for FP16 and FP32 input tensors and double for FP64 input tensors. For more information, refer to cudnnTensorDescriptor_t.

*bnScaleData

Input. Pointer in the device memory for the batch normalization scale parameter (in the Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift paper, the quantity scale is referred to as gamma).

*bnBiasData

Input. Pointers in the device memory for the batch normalization bias parameter (in the Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift paper, bias is referred to as beta). This parameter is used only when activation should be performed.

*dBnScaleData, *dBnBiasData

Outputs. Pointers in the device memory for the gradients of bnScaleData and bnBiasData, respectively.

epsilon

Input. Epsilon value used in batch normalization formula. Its value should be equal to or greater than the value defined for CUDNN_BN_MIN_EPSILON in cudnn.h. The same epsilon value should be used in forward and backward functions.

*savedMean, *savedInvVariance

Inputs. Optional cache parameters containing saved intermediate results computed during the forward pass. For this to work correctly, the layer’s x and bnScaleData, bnBiasData data has to remain unchanged until this backward function is called. Note that both these parameters can be NULL but only at the same time. It is recommended to use this cache since the memory overhead is relatively small.

activationDesc

Input. Descriptor for the activation operation. When the bnOps input is set to either CUDNN_BATCHNORM_OPS_BN_ACTIVATION or CUDNN_BATCHNORM_OPS_BN_ADD_ACTIVATION then this activation is used, otherwise the user may pass NULL.

workspace

Input. Pointer to the GPU workspace. If workspace is NULL and workSpaceSizeInBytes of zero is passed in, then this API will function exactly like the non-extended function cudnnBatchNormalizationBackward().

workSpaceSizeInBytes

Input. The size of the workspace. It must be large enough to trigger the fast NHWC semi-persistent kernel by this function.

*reserveSpace

Input. Pointer to the GPU workspace for the reserveSpace.

reserveSpaceSizeInBytes

Input. The size of the reserveSpace. It must be equal or larger than the amount required by cudnnGetBatchNormalizationTrainingExReserveSpaceSize().

Supported Configurations

This function supports the following combinations of data types for various descriptors.

Supported Configurations for `cudnnBatchNormalizationBackwardEx()`#
Data Type Configurations Supported	`xDesc` Data Type	`bnScaleBiasMeanVarDesc` Data Type	`alpha`, `beta` Data Type	`yDesc` Data Type
`PSEUDO_HALF_CONFIG`	`CUDNN_DATA_HALF`	`CUDNN_DATA_FLOAT`	`CUDNN_DATA_FLOAT`	`CUDNN_DATA_HALF`
`FLOAT_CONFIG`	`CUDNN_DATA_FLOAT`	`CUDNN_DATA_FLOAT`	`CUDNN_DATA_FLOAT`	`CUDNN_DATA_FLOAT`
`DOUBLE_CONFIG`	`CUDNN_DATA_DOUBLE`	`CUDNN_DATA_DOUBLE`	`CUDNN_DATA_DOUBLE`	`CUDNN_DATA_DOUBLE`
`PSEUDO_BFLOAT16_CONFIG`	`CUDNN_DATA_BFLOAT16`	`CUDNN_DATA_FLOAT`	`CUDNN_DATA_FLOAT`	`CUDNN_DATA_BFLOAT16`

Returns

CUDNN_STATUS_SUCCESS

The computation was performed successfully.

CUDNN_STATUS_NOT_SUPPORTED

The function does not support the provided configuration.

CUDNN_STATUS_BAD_PARAM

At least one of the following conditions are met:

Any of the pointers alphaDataDiff, betaDataDiff, alphaParamDiff, betaParamDiff, x, dy, dx, bnScale, resultBnScaleDiff, and resultBnBiasDiff is NULL.

The number of xDesc, yDesc, or dxDesc tensor descriptor dimensions is not within the range of [4,5] (only 4D and 5D tensors are supported).

dBnScaleBiasDesc dimensions not 1xCx1x1 for 4D and 1xCx1x1x1 for 5D for spatial, and are not 1xCxHxW for 4D and 1xCxDxHxW for 5D for per-activation mode.

Exactly one of savedMean, savedInvVariance pointers is NULL.

epsilon value is less than CUDNN_BN_MIN_EPSILON.

Dimensions or data types mismatch for any pair of xDesc, dyDesc, and dxDesc.

cudnnBatchNormalizationForwardInference()#

This function has been deprecated in cuDNN 9.0.

This function performs the forward batch normalization layer computation for the inference phase. This layer is based on the Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift paper.

cudnnStatus_t cudnnBatchNormalizationForwardInference(
    cudnnHandle_t                    handle,
    cudnnBatchNormMode_t             mode,
    const void                      *alpha,
    const void                      *beta,
    const cudnnTensorDescriptor_t    xDesc,
    const void                      *x,
    const cudnnTensorDescriptor_t    yDesc,
    void                            *y,
    const cudnnTensorDescriptor_t    bnScaleBiasMeanVarDesc,
    const void                      *bnScale,
    const void                      *bnBias,
    const void                      *estimatedMean,
    const void                      *estimatedVariance,
    double                           epsilon)

Only 4D and 5D tensors are supported.

The input transformation performed by this function is defined as:

y = beta*y + alpha *[bnBias + (bnScale * (x-estimatedMean)/sqrt(epsilon + estimatedVariance)]

The epsilon value has to be the same during training, backpropagation and inference.

For the training phase, refer to cudnnBatchNormalizationForwardTraining().

Higher performance can be obtained when HW-packed tensors are used for all of x and dx.

For more information, refer to cudnnDeriveBNTensorDescriptor() for the secondary tensor descriptor generation for the parameters used in this function.

Parameters

handle

Input. Handle to a previously created cuDNN library descriptor. For more information, refer to cudnnHandle_t.

mode

Input. Mode of operation (spatial or per-activation). For more information, refer to cudnnBatchNormMode_t.

alpha, beta

Inputs. Pointers to scaling factors (in host memory) used to blend the layer output value with prior value in the destination tensor as follows:

dstValue = alpha[0]*resultValue + beta[0]*priorDstValue

For more information, refer to Scaling Parameters.

xDesc, yDesc

Input. Handles to the previously initialized tensor descriptors.

*x

Input. Data pointer to GPU memory associated with the tensor descriptor xDesc, for the layer’s x input data.

*y

Input/Output. Data pointer to GPU memory associated with the tensor descriptor yDesc, for the y output of the batch normalization layer.

bnScaleBiasMeanVarDesc, bnScale, bnBias

Inputs. Tensor descriptors and pointers in device memory for the batch normalization scale and bias parameters (in the Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift paper, bias is referred to as beta and scale as gamma).

estimatedMean, estimatedVariance

Inputs. Mean and variance tensors (these have the same descriptor as the bias and scale). The resultRunningMean and resultRunningVariance, accumulated during the training phase from the cudnnBatchNormalizationForwardTraining() call, should be passed as inputs here.

epsilon

Input. Epsilon value used in batch normalization formula. Its value should be equal to or greater than the value defined for CUDNN_BN_MIN_EPSILON in cudnn.h. The same epsilon value should be used in forward and backward functions.

Supported Configurations

This function supports the following combinations of data types for various descriptors.

Supported Configurations for `cudnnBatchNormalizationForwardInference()`#
Data Type Configurations Supported	`xDesc` Data Type	`bnScaleBiasMeanVarDesc` Data Type	`alpha`, `beta` Data Type	`yDesc` Data Type
`INT8_CONFIG`	`CUDNN_DATA_INT8`	`CUDNN_DATA_FLOAT`	`CUDNN_DATA_FLOAT`	`CUDNN_DATA_INT8`
`PSEUDO_HALF_CONFIG`	`CUDNN_DATA_HALF`	`CUDNN_DATA_FLOAT`	`CUDNN_DATA_FLOAT`	`CUDNN_DATA_HALF`
`FLOAT_CONFIG`	`CUDNN_DATA_FLOAT`	`CUDNN_DATA_FLOAT`	`CUDNN_DATA_FLOAT`	`CUDNN_DATA_FLOAT`
`DOUBLE_CONFIG`	`CUDNN_DATA_DOUBLE`	`CUDNN_DATA_DOUBLE`	`CUDNN_DATA_DOUBLE`	`CUDNN_DATA_DOUBLE`
`BFLOAT16_CONFIG`	`CUDNN_DATA_BFLOAT16`	`CUDNN_DATA_BFLOAT16`	`CUDNN_DATA_FLOAT`	`CUDNN_DATA_BFLOAT16`

Returns

CUDNN_STATUS_SUCCESS

The computation was performed successfully.

CUDNN_STATUS_NOT_SUPPORTED

The function does not support the provided configuration.

CUDNN_STATUS_BAD_PARAM

At least one of the following conditions are met:

One of the pointers alpha, beta, x, y, bnScale, bnBias, estimatedMean, and estimatedInvVariance is NULL.

The number of xDesc or yDesc tensor descriptor dimensions is not within the range of [4,5] (only 4D and 5D tensors are supported.)

bnScaleBiasMeanVarDesc dimensions are not 1xCx1x1 for 4D and 1xCx1x1x1 for 5D for spatial, and are not 1xCxHxW for 4D and 1xCxDxHxW for 5D for per-activation mode.

epsilon value is less than CUDNN_BN_MIN_EPSILON.

Dimensions or data types mismatch for xDesc, yDesc.

cudnnBatchNormalizationForwardTraining()#

This function has been deprecated in cuDNN 9.0.

This function performs the forward batch normalization layer computation for the training phase. This layer is based on the Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift paper.

cudnnStatus_t cudnnBatchNormalizationForwardTraining(
    cudnnHandle_t                    handle,
    cudnnBatchNormMode_t             mode,
    const void                      *alpha,
    const void                      *beta,
    const cudnnTensorDescriptor_t    xDesc,
    const void                      *x,
    const cudnnTensorDescriptor_t    yDesc,
    void                            *y,
    const cudnnTensorDescriptor_t    bnScaleBiasMeanVarDesc,
    const void                      *bnScale,
    const void                      *bnBias,
    double                           exponentialAverageFactor,
    void                            *resultRunningMean,
    void                            *resultRunningVariance,
    double                           epsilon,
    void                            *resultSaveMean,
    void                            *resultSaveInvVariance)

Only 4D and 5D tensors are supported.

The epsilon value has to be the same during training, backpropagation, and inference.

For the inference phase, use cudnnBatchNormalizationForwardInference().

Higher performance can be obtained when HW-packed tensors are used for both x and y.

Refer to cudnnDeriveBNTensorDescriptor() for the secondary tensor descriptor generation for the parameters used in this function.

Parameters

handle

Input. Handle to a previously created cuDNN library descriptor. For more information, refer to cudnnHandle_t.

mode

Input. Mode of operation (spatial or per-activation). For more information, refer to cudnnBatchNormMode_t.

alpha, beta

Inputs. Pointers to scaling factors (in host memory) used to blend the layer output value with prior value in the destination tensor as follows:

dstValue = alpha[0]*resultValue + beta[0]*priorDstValue

For more information, refer to Scaling Parameters.

xDesc, yDesc

Input. Tensor descriptors and pointers in device memory for the layer’s x and y data. For more information, refer to cudnnTensorDescriptor_t.

*x

Input. Data pointer to GPU memory associated with the tensor descriptor xDesc, for the layer’s x input data.

*y

Input. Data pointer to GPU memory associated with the tensor descriptor yDesc, for the y output of the batch normalization layer.

bnScaleBiasMeanVarDesc

Input. Shared tensor descriptor desc for the secondary tensor that was derived by cudnnDeriveBNTensorDescriptor(). The dimensions for this tensor descriptor are dependent on the normalization mode.

bnScale, bnBias

Inputs. Pointers in device memory for the batch normalization scale and bias parameters (in the Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift paper, bias is referred to as beta and scale as gamma). Note that bnBias parameter can replace the previous layer’s bias parameter for improved efficiency.

exponentialAverageFactor

Input. Factor used in the moving average computation as follows:

runningMean = runningMean*(1-factor) + newMean*factor

Use a factor=1/(1+n) at N-th call to the function to get the Cumulative Moving Average (CMA) behavior, for example:

CMA[n] = (x[1]+...+x[n])/n

For example:

CMA[n+1] = (n*CMA[n]+x[n+1])/(n+1)
= ((n+1)*CMA[n]-CMA[n])/(n+1) + x[n+1]/(n+1)
= CMA[n]*(1-1/(n+1))+x[n+1]*1/(n+1)
= CMA[n]*(1-factor) + x(n+1)*factor

resultRunningMean, resultRunningVariance

Inputs/Outputs. Running mean and variance tensors (these have the same descriptor as the bias and scale). Both of these pointers can be NULL but only at the same time. The value stored in resultRunningVariance (or passed as an input in inference mode) is the sample variance and is the moving average of variance[x] where the variance is computed either over batch or spatial+batch dimensions depending on the mode. If these pointers are not NULL, the tensors should be initialized to some reasonable values or to 0.

epsilon

Input. Epsilon value used in the batch normalization formula. Its value should be equal to or greater than the value defined for CUDNN_BN_MIN_EPSILON in cudnn.h. The same epsilon value should be used in forward and backward functions.

resultSaveMean, resultSaveInvVariance

Outputs. Optional cache to save intermediate results computed during the forward pass. These buffers can be used to speed up the backward pass when supplied to the cudnnBatchNormalizationBackward() function. The intermediate results stored in resultSaveMean and resultSaveInvVariance buffers should not be used directly by the user. Depending on the batch normalization mode, the results stored in resultSaveInvVariance may vary. For the cache to work correctly, the input layer data must remain unchanged until the backward function is called. Note that both parameters can be NULL but only at the same time. In such a case, intermediate statistics will not be saved, and cudnnBatchNormalizationBackward() will have to re-compute them. It is recommended to use this cache as the memory overhead is relatively small because these tensors have a much lower product of dimensions than the data tensors.

Supported Configurations

This function supports the following combinations of data types for various descriptors.

Supported Configurations for `cudnnBatchNormalizationForwardTraining()`#
Data Type Configurations Supported	`xDesc` Data Type	`bnScaleBiasMeanVarDesc` Data Type	`alpha`, `beta` Data Type	`yDesc` Data Type
`PSEUDO_HALF_CONFIG`	`CUDNN_DATA_HALF`	`CUDNN_DATA_FLOAT`	`CUDNN_DATA_FLOAT`	`CUDNN_DATA_HALF`
`FLOAT_CONFIG`	`CUDNN_DATA_FLOAT`	`CUDNN_DATA_FLOAT`	`CUDNN_DATA_FLOAT`	`CUDNN_DATA_FLOAT`
`DOUBLE_CONFIG`	`CUDNN_DATA_DOUBLE`	`CUDNN_DATA_DOUBLE`	`CUDNN_DATA_DOUBLE`	`CUDNN_DATA_DOUBLE`
`PSEUDO_BFLOAT16_CONFIG`	`CUDNN_DATA_BFLOAT16`	`CUDNN_DATA_FLOAT`	`CUDNN_DATA_FLOAT`	`CUDNN_DATA_BFLOAT16`

Returns

CUDNN_STATUS_SUCCESS

The computation was performed successfully.

CUDNN_STATUS_NOT_SUPPORTED

The function does not support the provided configuration.

CUDNN_STATUS_BAD_PARAM

At least one of the following conditions are met:

One of the pointers alpha, beta, x, y, bnScale, and bnBias is NULL.

The number of xDesc or yDesc tensor descriptor dimensions is not within the range of [4,5] (only 4D and 5D tensors are supported.)

bnScaleBiasMeanVarDesc dimensions are not 1xCx1x1 for 4D and 1xCx1x1x1 for 5D for spatial, and are not 1xCxHxW for 4D and 1xCxDxHxW for 5D for per-activation mode.

Exactly one of resultSaveMean, resultSaveInvVariance pointers are NULL.

Exactly one of resultRunningMean, resultRunningInvVariance pointers are NULL.

epsilon value is less than CUDNN_BN_MIN_EPSILON.

Dimensions or data types mismatch for xDesc, yDesc.

cudnnBatchNormalizationForwardTrainingEx()#

This function has been deprecated in cuDNN 9.0.

This function is an extension of the cudnnBatchNormalizationForwardTraining() for performing the forward batch normalization layer computation.

cudnnStatus_t cudnnBatchNormalizationForwardTrainingEx(
    cudnnHandle_t                       handle,
    cudnnBatchNormMode_t                mode,
    cudnnBatchNormOps_t                 bnOps,
    const void                          *alpha,
    const void                          *beta,
    const cudnnTensorDescriptor_t       xDesc,
    const void                          *xData,
    const cudnnTensorDescriptor_t       zDesc,
    const void                          *zData,
    const cudnnTensorDescriptor_t       yDesc,
    void                                *yData,
    const cudnnTensorDescriptor_t       bnScaleBiasMeanVarDesc,
    const void                          *bnScaleData,
    const void                          *bnBiasData,
    double                              exponentialAverageFactor,
    void                                *resultRunningMeanData,
    void                                *resultRunningVarianceData,
    double                              epsilon,
    void                                *saveMean,
    void                                *saveInvVariance,
    const cudnnActivationDescriptor_t   activationDesc,
    void                                *workspace,
    size_t                              workSpaceSizeInBytes
    void                                *reserveSpace
    size_t                              reserveSpaceSizeInBytes);

This API will trigger the new semi-persistent NHWC kernel when the following conditions are true:

All tensors, namely, x, y, dz, dy, and dx must be NHWC-fully packed and must be of the type CUDNN_DATA_HALF.

The input parameter mode must be set to CUDNN_BATCHNORM_SPATIAL_PERSISTENT.

workspace is not NULL.

Before cuDNN version 8.2.0, the tensor C dimension should always be a multiple of 4. After 8.2.0, the tensor C dimension should be a multiple of 4 only when bnOps is CUDNN_BATCHNORM_OPS_BN_ADD_ACTIVATION.

workSpaceSizeInBytes is equal to or larger than the amount required by cudnnGetBatchNormalizationForwardTrainingExWorkspaceSize().

reserveSpaceSizeInBytes is equal to or larger than the amount required by cudnnGetBatchNormalizationTrainingExReserveSpaceSize().

The content in reserveSpace stored by cudnnBatchNormalizationForwardTrainingEx() must be preserved.

If workspace is NULL and workSpaceSizeInBytes of zero is passed in, this API will function exactly like the non-extended function cudnnBatchNormalizationForwardTraining().

This workspace is not required to be clean. Moreover, the workspace does not have to remain unchanged between the forward and backward pass, as it is not used for passing any information.

This extended function can accept a *workspace pointer to the GPU workspace, and workSpaceSizeInBytes, the size of the workspace, from the user.

The bnOps input can be used to set this function to perform either only the batch normalization, or batch normalization followed by activation, or batch normalization followed by element-wise addition and then activation.

Only 4D and 5D tensors are supported. The epsilon value has to be the same during the training, the backpropagation, and the inference.

When the tensor layout is NCHW, higher performance can be obtained when HW-packed tensors are used for x, dy, and dx.

Parameters

handle

Input. Handle to a previously created cuDNN library descriptor. For more information, refer to cudnnHandle_t.

mode

Input. Mode of operation (spatial or per-activation). For more information, refer to cudnnBatchNormMode_t.

bnOps

Input. Mode of operation for the fast NHWC kernel. For more information, refer to cudnnBatchNormOps_t. This input can be used to set this function to perform either only the batch normalization, or batch normalization followed by activation, or batch normalization followed by element-wise addition and then activation.

*alpha, *beta

Inputs. Pointers to scaling factors (in host memory) used to blend the layer output value with prior value in the destination tensor as follows:

dstValue = alpha[0]*resultValue + beta[0]*priorDstValue

For more information, refer to Scaling Parameters.

xDesc, *xData, zDesc, *zData, yDesc, *yData

Inputs. Tensor descriptors and pointers in device memory for the layer’s input x and output y, and for the optional z tensor input for residual addition to the result of the batch normalization operation, prior to the activation. The optional zDesc and *zData descriptors are only used when bnOps is CUDNN_BATCHNORM_OPS_BN_ADD_ACTIVATION, otherwise users may pass NULL. When in use, z should have exactly the same dimension as x and the final output y. For more information, refer to cudnnTensorDescriptor_t.

bnScaleBiasMeanVarDesc

Input. Shared tensor descriptor desc for the secondary tensor that was derived by cudnnDeriveBNTensorDescriptor(). The dimensions for this tensor descriptor are dependent on the normalization mode.

*bnScaleData, *bnBiasData

Inputs. Pointers in the device memory for the batch normalization scale and bias data. In the Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift paper, bias is referred to as beta and scale as gamma. Note that bnBiasData parameter can replace the previous operations bias parameter for improved efficiency.

exponentialAverageFactor

Input. Factor used in the moving average computation as follows:

runningMean = runningMean*(1-factor) + newMean*factor

Use a factor=1/(1+n) at N-th call to the function to get the Cumulative Moving Average (CMA) behavior, for example:

CMA[n] = (x[1]+...+x[n])/n

For example:

CMA[n+1] = (n*CMA[n]+x[n+1])/(n+1)
= ((n+1)*CMA[n]-CMA[n])/(n+1) + x[n+1]/(n+1)
= CMA[n]*(1-1/(n+1))+x[n+1]*1/(n+1)
= CMA[n]*(1-factor) + x(n+1)*factor

*resultRunningMeanData, *resultRunningVarianceData

Inputs/Outputs. Pointers to the running mean and running variance data. Both these pointers can be NULL but only at the same time. The value stored in resultRunningVarianceData (or passed as an input in inference mode) is the sample variance and is the moving average of variance[x] where the variance is computed either over batch or spatial+batch dimensions depending on the mode. If these pointers are not NULL, the tensors should be initialized to some reasonable values or to 0.

epsilon

Input. Epsilon value used in the batch normalization formula. Its value should be equal to or greater than the value defined for CUDNN_BN_MIN_EPSILON in cudnn.h. The same epsilon value should be used in forward and backward functions.

*saveMean, *saveInvVariance

Outputs. Optional cache parameters containing saved intermediate results computed during the forward pass. For this to work correctly, the layer’s x and bnScaleData, bnBiasData data has to remain unchanged until this backward function is called. Note that both these parameters can be NULL but only at the same time. It is recommended to use this cache since the memory overhead is relatively small.

activationDesc

Input. The tensor descriptor for the activation operation. When the bnOps input is set to either CUDNN_BATCHNORM_OPS_BN_ACTIVATION or CUDNN_BATCHNORM_OPS_BN_ADD_ACTIVATION then this activation is used, otherwise user may pass NULL.

*workspace, workSpaceSizeInBytes

Inputs. *workspace is a pointer to the GPU workspace, and workSpaceSizeInBytes is the size of the workspace. When *workspace is not NULL and *workSpaceSizeInBytes is large enough, and the tensor layout is NHWC and the data type configuration is supported, then this function will trigger a new semi-persistent NHWC kernel for batch normalization. The workspace is not required to be clean. Also, the workspace does not need to remain unchanged between the forward and backward passes.

*reserveSpace

Input. Pointer to the GPU workspace for the reserveSpace.

reserveSpaceSizeInBytes

Input. The size of the reserveSpace. Must be equal or larger than the amount required by cudnnGetBatchNormalizationTrainingExReserveSpaceSize().

Supported Configurations

This function supports the following combinations of data types for various descriptors.

Supported Configurations for `cudnnBatchNormalizationForwardTrainingEx()`#
Data Type Configurations Supported	`xDesc` Data Type	`bnScaleBiasMeanVarDesc` Data Type	`alpha`, `beta` Data Type	`yDesc` Data Type
`PSEUDO_HALF_CONFIG`	`CUDNN_DATA_HALF`	`CUDNN_DATA_FLOAT`	`CUDNN_DATA_FLOAT`	`CUDNN_DATA_HALF`
`FLOAT_CONFIG`	`CUDNN_DATA_FLOAT`	`CUDNN_DATA_FLOAT`	`CUDNN_DATA_FLOAT`	`CUDNN_DATA_FLOAT`
`DOUBLE_CONFIG`	`CUDNN_DATA_DOUBLE`	`CUDNN_DATA_DOUBLE`	`CUDNN_DATA_DOUBLE`	`CUDNN_DATA_DOUBLE`
`PSEUDO_BFLOAT16_CONFIG`	`CUDNN_DATA_BFLOAT16`	`CUDNN_DATA_FLOAT`	`CUDNN_DATA_FLOAT`	`CUDNN_DATA_BFLOAT16`

Returns

CUDNN_STATUS_SUCCESS

The computation was performed successfully.

CUDNN_STATUS_NOT_SUPPORTED

The function does not support the provided configuration.

CUDNN_STATUS_BAD_PARAM

At least one of the following conditions are met:

One of the pointers alpha, beta, x, y, bnScaleData, and bnBiasData is NULL.

The number of xDesc or yDesc tensor descriptor dimensions is not within the [4,5] range (only 4D and 5D tensors are supported.)

bnScaleBiasMeanVarDesc dimensions are not 1xCx1x1 for 4D and 1xCx1x1x1 for 5D for spatial, and are not 1xCxHxW for 4D and 1xCxDxHxW for 5D for per-activation mode.

Exactly one of saveMean, saveInvVariance pointers are NULL.

Exactly one of resultRunningMeanData, resultRunningInvVarianceData pointers are NULL.

epsilon value is less than CUDNN_BN_MIN_EPSILON.

Dimensions or data types mismatch for xDesc and yDesc.

cudnnCreateActivationDescriptor()#

This function creates an activation descriptor object by allocating the memory needed to hold its opaque structure. For more information, refer to cudnnActivationDescriptor_t.

cudnnStatus_t cudnnCreateActivationDescriptor(
        cudnnActivationDescriptor_t   *activationDesc)

Returns

CUDNN_STATUS_SUCCESS: The object was created successfully.
CUDNN_STATUS_ALLOC_FAILED: The resources could not be allocated.

cudnnCreateDropoutDescriptor()#

This function creates a generic dropout descriptor object by allocating the memory needed to hold its opaque structure. For more information, refer to cudnnDropoutDescriptor_t.

cudnnStatus_t cudnnCreateDropoutDescriptor(
    cudnnDropoutDescriptor_t    *dropoutDesc)

Returns

CUDNN_STATUS_SUCCESS: The object was created successfully.
CUDNN_STATUS_ALLOC_FAILED: The resources could not be allocated.

cudnnCreateFilterDescriptor()#

This function has been deprecated in cuDNN 9.0.

This function creates a filter descriptor object by allocating the memory needed to hold its opaque structure. For more information, refer to cudnnFilterDescriptor_t.

cudnnStatus_t cudnnCreateFilterDescriptor(
    cudnnFilterDescriptor_t *filterDesc)

Returns

CUDNN_STATUS_SUCCESS: The object was created successfully.
CUDNN_STATUS_ALLOC_FAILED: The resources could not be allocated.

cudnnCreateLRNDescriptor()#

This function allocates the memory needed to hold the data needed for LRN and DivisiveNormalization layers operation and returns a descriptor used with subsequent layer forward and backward calls.

cudnnStatus_t cudnnCreateLRNDescriptor(
            cudnnLRNDescriptor_t    *poolingDesc)

Returns

CUDNN_STATUS_SUCCESS: The object was created successfully.
CUDNN_STATUS_ALLOC_FAILED: The resources could not be allocated.

cudnnCreateOpTensorDescriptor()#

This function creates a tensor pointwise math descriptor. For more information, refer to cudnnOpTensorDescriptor_t.

cudnnStatus_t cudnnCreateOpTensorDescriptor(
    cudnnOpTensorDescriptor_t*  opTensorDesc)

Parameters

opTensorDesc: Output. Pointer to the structure holding the description of the tensor pointwise math such as add, multiply, and more.

Returns

CUDNN_STATUS_SUCCESS: The function returned successfully.
CUDNN_STATUS_BAD_PARAM: Tensor pointwise math descriptor passed to the function is invalid.
CUDNN_STATUS_ALLOC_FAILED: Memory allocation for this tensor pointwise math descriptor failed.

cudnnCreatePoolingDescriptor()#

This function creates a pooling descriptor object by allocating the memory needed to hold its opaque structure.

cudnnStatus_t cudnnCreatePoolingDescriptor(
    cudnnPoolingDescriptor_t    *poolingDesc)

Returns

CUDNN_STATUS_SUCCESS: The object was created successfully.
CUDNN_STATUS_ALLOC_FAILED: The resources could not be allocated.

cudnnCreateReduceTensorDescriptor()#

This function creates a reduced tensor descriptor object by allocating the memory needed to hold its opaque structure.

cudnnStatus_t cudnnCreateReduceTensorDescriptor(
    cudnnReduceTensorDescriptor_t*      reduceTensorDesc)

Returns

CUDNN_STATUS_SUCCESS: The object was created successfully.
CUDNN_STATUS_BAD_PARAM: reduceTensorDesc is a NULL pointer.
CUDNN_STATUS_ALLOC_FAILED: The resources could not be allocated.

cudnnCreateSpatialTransformerDescriptor()#

This function creates a generic spatial transformer descriptor object by allocating the memory needed to hold its opaque structure.

cudnnStatus_t cudnnCreateSpatialTransformerDescriptor(
    cudnnSpatialTransformerDescriptor_t *stDesc)

Returns

CUDNN_STATUS_SUCCESS: The object was created successfully.
CUDNN_STATUS_ALLOC_FAILED: The resources could not be allocated.

cudnnCreateTensorDescriptor()#

This function creates a generic tensor descriptor object by allocating the memory needed to hold its opaque structure. The data is initialized to all zeros.

cudnnStatus_t cudnnCreateTensorDescriptor(
    cudnnTensorDescriptor_t *tensorDesc)

Parameters

tensorDesc: Input. Pointer to pointer where the address to the allocated tensor descriptor object should be stored.

Returns

CUDNN_STATUS_BAD_PARAM: Invalid input argument.
CUDNN_STATUS_ALLOC_FAILED: The resources could not be allocated.
CUDNN_STATUS_SUCCESS: The object was created successfully.

cudnnCreateTensorTransformDescriptor()#

This function has been deprecated in cuDNN 9.0.

This function creates a tensor transform descriptor object by allocating the memory needed to hold its opaque structure. The tensor data is initialized to all zeros. Use the cudnnSetTensorTransformDescriptor() function to initialize the descriptor created by this function.

cudnnStatus_t cudnnCreateTensorTransformDescriptor(
    cudnnTensorTransformDescriptor_t *transformDesc);

Parameters

transformDesc: Output. A pointer to an uninitialized tensor transform descriptor.

Returns

CUDNN_STATUS_SUCCESS: The descriptor object was created successfully.
CUDNN_STATUS_BAD_PARAM: The transformDesc is NULL.
CUDNN_STATUS_ALLOC_FAILED: The memory allocation failed.

cudnnDeriveBNTensorDescriptor()#

This function derives a secondary tensor descriptor for the batch normalization scale, invVariance, bnBias, and bnScale subtensors from the layer’s x data descriptor.

cudnnStatus_t cudnnDeriveBNTensorDescriptor(
    cudnnTensorDescriptor_t         derivedBnDesc,
    const cudnnTensorDescriptor_t   xDesc,
    cudnnBatchNormMode_t            mode)

Use the tensor descriptor produced by this function as the bnScaleBiasMeanVarDesc parameter for the cudnnBatchNormalizationForwardInference() and cudnnBatchNormalizationForwardTraining() functions, and as the bnScaleBiasDiffDesc parameter in the cudnnBatchNormalizationBackward() function.

The resulting dimensions will be:

1xCx1x1 for 4D and 1xCx1x1x1 for 5D for BATCHNORM_MODE_SPATIAL

1xCxHxW for 4D and 1xCxDxHxW for 5D for BATCHNORM_MODE_PER_ACTIVATION mode

For HALF input data type the resulting tensor descriptor will have a FLOAT type. For other data types, it will have the same type as the input data.

Note

Only 4D and 5D tensors are supported.

The derivedBnDesc should be first created using cudnnCreateTensorDescriptor().

xDesc is the descriptor for the layer’s x data and has to be set up with proper dimensions prior to calling this function.

Parameters

derivedBnDesc: Output. Handle to a previously created tensor descriptor.
xDesc: Input. Handle to a previously created and initialized layer’s x data descriptor.
mode: Input. Batch normalization layer mode of operation.

Returns

CUDNN_STATUS_SUCCESS: The computation was performed successfully.
CUDNN_STATUS_BAD_PARAM: Invalid batch normalization mode.

cudnnDeriveNormTensorDescriptor()#

This function derives tensor descriptors for the normalization mean, invariance, normBias, and normScale subtensors from the layer’s x data descriptor and norm mode. Normalization mean and invariance share the same descriptor while bias and scale share the same descriptor.

cudnnStatus_t CUDNNWINAPI
cudnnDeriveNormTensorDescriptor(cudnnTensorDescriptor_t derivedNormScaleBiasDesc,
                                cudnnTensorDescriptor_t derivedNormMeanVarDesc,
                                const cudnnTensorDescriptor_t xDesc,
                            cudnnNormMode_t mode,
                                int groupCnt)

Use the tensor descriptor produced by this function as the normScaleBiasDesc or normMeanVarDesc parameter for the cudnnNormalizationForwardInference() and cudnnNormalizationForwardTraining() functions, and as the dNormScaleBiasDesc and normMeanVarDesc parameters in the cudnnNormalizationBackward() function.

The resulting dimensions will be:

1xCx1x1 for 4D and 1xCx1x1x1 for 5D for CUDNN_NORM_PER_ACTIVATION

1xCxHxW for 4D and 1xCxDxHxW for 5D for CUDNN_NORM_PER_CHANNEL mode

For HALF input data type the resulting tensor descriptor will have a FLOAT type. For other data types, it will have the same type as the input data.

Note

Only 4D and 5D tensors are supported.

The derivedNormScaleBiasDesc and derivedNormMeanVarDesc should be first created using cudnnCreateTensorDescriptor().

xDesc is the descriptor for the layer’s x data and has to be set up with proper dimensions prior to calling this function.

Parameters

derivedNormScaleBiasDesc: Output. Handle to a previously created tensor descriptor.
derivedNormMeanVarDesc: Output. Handle to a previously created tensor descriptor.
xDesc: Input. Handle to a previously created and initialized layer’s x data descriptor.
mode: Input. Batch normalization layer mode of operation.

Returns

CUDNN_STATUS_SUCCESS: The computation was performed successfully.
CUDNN_STATUS_BAD_PARAM: Invalid batch normalization mode.

cudnnDestroyActivationDescriptor()#

This function destroys a previously created activation descriptor object.

cudnnStatus_t cudnnDestroyActivationDescriptor(
        cudnnActivationDescriptor_t activationDesc)

Returns

CUDNN_STATUS_SUCCESS: The object was destroyed successfully.

cudnnDestroyDropoutDescriptor()#

This function destroys a previously created dropout descriptor object.

cudnnStatus_t cudnnDestroyDropoutDescriptor(
    cudnnDropoutDescriptor_t dropoutDesc)

Returns

CUDNN_STATUS_SUCCESS: The object was destroyed successfully.

cudnnDestroyFilterDescriptor()#

This function has been deprecated in cuDNN 9.0.

This function destroys a filter object.

cudnnStatus_t cudnnDestroyFilterDescriptor(
    cudnnFilterDescriptor_t filterDesc)

Returns

CUDNN_STATUS_SUCCESS: The object was destroyed successfully.

cudnnDestroyLRNDescriptor()#

This function destroys a previously created LRN descriptor object.

cudnnStatus_t cudnnDestroyLRNDescriptor(
    cudnnLRNDescriptor_t lrnDesc)

Returns

CUDNN_STATUS_SUCCESS: The object was destroyed successfully.

cudnnDestroyOpTensorDescriptor()#

This function deletes a tensor pointwise math descriptor object.

cudnnStatus_t cudnnDestroyOpTensorDescriptor(
    cudnnOpTensorDescriptor_t   opTensorDesc)

Parameters

opTensorDesc: Input. Pointer to the structure holding the description of the tensor pointwise math to be deleted.

Returns

CUDNN_STATUS_SUCCESS: The function returned successfully.

cudnnDestroyPoolingDescriptor()#

This function destroys a previously created pooling descriptor object.

cudnnStatus_t cudnnDestroyPoolingDescriptor(
    cudnnPoolingDescriptor_t poolingDesc)

Returns

CUDNN_STATUS_SUCCESS: The object was destroyed successfully.

cudnnDestroyReduceTensorDescriptor()#

This function destroys a previously created reduce tensor descriptor object. When the input pointer is NULL, this function performs no destroy operation.

cudnnStatus_t cudnnDestroyReduceTensorDescriptor(
    cudnnReduceTensorDescriptor_t   tensorDesc)

Parameters

tensorDesc: Input. Pointer to the reduce tensor descriptor object to be destroyed.

Returns

CUDNN_STATUS_SUCCESS: The object was destroyed successfully.

cudnnDestroySpatialTransformerDescriptor()#

This function destroys a previously created spatial transformer descriptor object.

cudnnStatus_t cudnnDestroySpatialTransformerDescriptor(
    cudnnSpatialTransformerDescriptor_t stDesc)

Returns

CUDNN_STATUS_SUCCESS: The object was destroyed successfully.

cudnnDestroyTensorDescriptor()#

This function destroys a previously created tensor descriptor object. When the input pointer is NULL, this function performs no destroy operation.

cudnnStatus_t cudnnDestroyTensorDescriptor(cudnnTensorDescriptor_t tensorDesc)

Parameters

tensorDesc: Input. Pointer to the tensor descriptor object to be destroyed.

Returns

CUDNN_STATUS_SUCCESS: The object was destroyed successfully.

cudnnDestroyTensorTransformDescriptor()#

This function has been deprecated in cuDNN 9.0.

Destroys a previously created tensor transform descriptor.

cudnnStatus_t cudnnDestroyTensorTransformDescriptor(
    cudnnTensorTransformDescriptor_t transformDesc);

Parameters

transformDesc: Input. The tensor transform descriptor to be destroyed.

Returns

CUDNN_STATUS_SUCCESS: The object was destroyed successfully.

cudnnDivisiveNormalizationBackward()#

This function performs the backward DivisiveNormalization layer computation.

cudnnStatus_t cudnnDivisiveNormalizationBackward(
    cudnnHandle_t                    handle,
    cudnnLRNDescriptor_t             normDesc,
    cudnnDivNormMode_t               mode,
    const void                      *alpha,
    const cudnnTensorDescriptor_t    xDesc,
    const void                      *x,
    const void                      *means,
    const void                      *dy,
    void                            *temp,
    void                            *temp2,
    const void                      *beta,
    const cudnnTensorDescriptor_t    dxDesc,
    void                            *dx,
    void                            *dMeans)

Supported tensor formats are NCHW for 4D and NCDHW for 5D with any non-overlapping non-negative strides. Only 4D and 5D tensors are supported.

Parameters

handle

Input. Handle to a previously created cuDNN library descriptor.

normDesc

Input. Handle to a previously initialized LRN parameter descriptor (this descriptor is used for both LRN and DivisiveNormalization layers).

mode

Input. DivisiveNormalization layer mode of operation. Currently only CUDNN_DIVNORM_PRECOMPUTED_MEANS is implemented. Normalization is performed using the means input tensor that is expected to be precomputed by the user.

alpha, beta

Inputs. Pointers to scaling factors (in host memory) used to blend the layer output value with prior value in the destination tensor as follows:

dstValue = alpha[0]*resultValue + beta[0]*priorDstValue.

For more information, refer to Scaling Parameters.

xDesc, x, means

Inputs. Tensor descriptor and pointers in device memory for the layer’s x and means data. Note that the means tensor is expected to be precomputed by the user. It can also contain any valid values (not required to be actual means, and can be for instance a result of a convolution with a Gaussian kernel).

dy

Input. Tensor pointer in device memory for the layer’s dy cumulative loss differential data (error backpropagation).

temp, temp2

Workspace. Temporary tensors in device memory. These are used for computing intermediate values during the backward pass. These tensors do not have to be preserved from forward to backward pass. Both use xDesc as a descriptor.

dxDesc

Input. Tensor descriptor for dx and dMeans.

dx, dMeans

Outputs. Tensor pointers (in device memory) for the layers resulting in cumulative gradients dx and dMeans (dLoss/dx and dLoss/dMeans). Both share the same descriptor.

Returns

CUDNN_STATUS_SUCCESS

The computation was performed successfully.

CUDNN_STATUS_BAD_PARAM

At least one of the following conditions are met:

One of the tensor pointers x, dx, temp, tmep2, and dy is NULL.

Number of any of the input or output tensor dimensions is not within the [4,5] range.

Either alpha or beta pointer is NULL.

A mismatch in dimensions between xDesc and dxDesc.

LRN descriptor parameters are outside of their valid ranges.

Any of the tensor strides is negative.

CUDNN_STATUS_UNSUPPORTED

The function does not support the provided configuration, for example, any of the input and output tensor strides mismatch (for the same dimension) is a non-supported configuration.

cudnnDivisiveNormalizationForward()#

This function performs the forward spatial DivisiveNormalization layer computation. It divides every value in a layer by the standard deviation of its spatial neighbors. Note that DivisiveNormalization only implements the x/max(c, sigma_x) portion of the computation, where sigma_x is the variance over the spatial neighborhood of x.

cudnnStatus_t cudnnDivisiveNormalizationForward(
    cudnnHandle_t                    handle,
    cudnnLRNDescriptor_t             normDesc,
    cudnnDivNormMode_t               mode,
    const void                      *alpha,
    const cudnnTensorDescriptor_t    xDesc,
    const void                      *x,
    const void                      *means,
    void                            *temp,
    void                            *temp2,
    const void                      *beta,
    const cudnnTensorDescriptor_t    yDesc,
    void                            *y)

The full LCN (Local Contrastive Normalization) computation can be implemented as a two-step process:

x_m = x-mean(x);
y = x_m/max(c, sigma(x_m));

The x-mean(x) which is often referred to as “subtractive normalization” portion of the computation can be implemented using cuDNN average pooling layer followed by a call to addTensor.

Supported tensor formats are NCHW for 4D and NCDHW for 5D with any non-overlapping non-negative strides. Only 4D and 5D tensors are supported.

Parameters

handle

Input. Handle to a previously created cuDNN library descriptor.

normDesc

Input. Handle to a previously initialized LRN parameter descriptor. This descriptor is used for both LRN and DivisiveNormalization layers.

divNormMode

Input. DivisiveNormalization layer mode of operation. Currently only CUDNN_DIVNORM_PRECOMPUTED_MEANS is implemented. Normalization is performed using the means input tensor that is expected to be precomputed by the user.

alpha, beta

Input. Pointers to scaling factors (in host memory) used to blend the layer output value with prior value in the destination tensor as follows:

dstValue = alpha[0]*resultValue + beta[0]*priorDstValue

For more information, refer to Scaling Parameters.

xDesc, yDesc

Input. Tensor descriptor objects for the input and output tensors. Note that xDesc is shared between x, means, temp, and temp2 tensors.

x

Input. Input tensor data pointer in device memory.

means

Input. Input means tensor data pointer in device memory. Note that this tensor can be NULL (in that case its values are assumed to be zero during the computation). This tensor also doesn’t have to contain means, these can be any values, a frequently used variation is a result of convolution with a normalized positive kernel (such as Gaussian).

temp, temp2

Workspace. Temporary tensors in device memory. These are used for computing intermediate values during the forward pass. These tensors do not have to be preserved as inputs from forward to the backward pass. Both use xDesc as their descriptor.

y

Output. Pointer in device memory to a tensor for the result of the forward DivisiveNormalization computation.

Returns

CUDNN_STATUS_SUCCESS

The computation was performed successfully.

CUDNN_STATUS_BAD_PARAM

At least one of the following conditions are met:

One of the tensor pointers x, y, temp, and temp2 is NULL.

Number of input tensor or output tensor dimensions is outside of [4,5] range.

A mismatch in dimensions between any two of the input or output tensors.

For in-place computation when pointers x == y, a mismatch in strides between the input data and output data tensors.

alpha or beta pointer is NULL.

LRN descriptor parameters are outside of their valid ranges.

Any of the tensor strides are negative.

CUDNN_STATUS_UNSUPPORTED

The function does not support the provided configuration, for example, any of the input and output tensor strides mismatch (for the same dimension) is a non-supported configuration.

cudnnDropoutBackward()#

This function performs backward dropout operation over dy returning results in dx. If during forward dropout operation value from x was propagated to y then during backward operation value from dy will be propagated to dx, otherwise, dx value will be set to 0.

cudnnStatus_t cudnnDropoutBackward(
    cudnnHandle_t                   handle,
    const cudnnDropoutDescriptor_t  dropoutDesc,
    const cudnnTensorDescriptor_t   dydesc,
    const void                     *dy,
    const cudnnTensorDescriptor_t   dxdesc,
    void                           *dx,
    void                           *reserveSpace,
    size_t                          reserveSpaceSizeInBytes)

Better performance is obtained for fully packed tensors.

Parameters

handle: Input. Handle to a previously created cuDNN context.
dropoutDesc: Input. Previously created dropout descriptor object.
dyDesc: Input. Handle to a previously initialized tensor descriptor.
dy: Input. Pointer to data of the tensor described by the dyDesc descriptor.
dxDesc: Input. Handle to a previously initialized tensor descriptor.
dx: Output. Pointer to data of the tensor described by the dxDesc descriptor.
reserveSpace: Input. Pointer to user-allocated GPU memory used by this function. It is expected that reserveSpace was populated during a call to cudnnDropoutForward and has not been changed.
reserveSpaceSizeInBytes: Input. Specifies the size in bytes of the provided memory for the reserve space

Returns

CUDNN_STATUS_SUCCESS

The call was successful.

CUDNN_STATUS_NOT_SUPPORTED

The function does not support the provided configuration.

CUDNN_STATUS_BAD_PARAM

At least one of the following conditions are met:

The number of elements of input tensor and output tensors differ.

The datatype of the input tensor and output tensors differs.

The strides of the input tensor and output tensors differ and in-place operation is used (meaning, x and y pointers are equal).

The provided reserveSpaceSizeInBytes is less than the value returned by cudnnDropoutGetReserveSpaceSize().

cudnnSetDropoutDescriptor() has not been called on dropoutDesc with non-NULL states argument.

CUDNN_STATUS_EXECUTION_FAILED

The function failed to launch on the GPU.

cudnnDropoutForward()#

This function performs forward dropout operation over x returning results in y. If dropout was used as a parameter to cudnnSetDropoutDescriptor(), the approximate dropout fraction of x values will be replaced by a 0, and the rest will be scaled by 1/(1-dropout). This function should not be running concurrently with another cudnnDropoutForward() function using the same states.

cudnnStatus_t cudnnDropoutForward(
    cudnnHandle_t                       handle,
    const cudnnDropoutDescriptor_t      dropoutDesc,
    const cudnnTensorDescriptor_t       xdesc,
    const void                         *x,
    const cudnnTensorDescriptor_t       ydesc,
    void                               *y,
    void                               *reserveSpace,
    size_t                              reserveSpaceSizeInBytes)

Note

Better performance is obtained for fully packed tensors.
This function should not be called during inference.

Parameters

handle: Input. Handle to a previously created cuDNN context.
dropoutDesc: Input. Previously created dropout descriptor object.
xDesc: Input. Handle to a previously initialized tensor descriptor.
x: Input. Pointer to data of the tensor described by the xDesc descriptor.
yDesc: Input. Handle to a previously initialized tensor descriptor.
y: Output. Pointer to data of the tensor described by the yDesc descriptor.
reserveSpace: Output. Pointer to user-allocated GPU memory used by this function. It is expected that the contents of reserveSpace does not change between cudnnDropoutForward() and cudnnDropoutBackward() calls.
reserveSpaceSizeInBytes: Input. Specifies the size in bytes of the provided memory for the reserve space.

Returns

CUDNN_STATUS_SUCCESS

The call was successful.

CUDNN_STATUS_NOT_SUPPORTED

The function does not support the provided configuration.

CUDNN_STATUS_BAD_PARAM

At least one of the following conditions are met:

The number of elements of input tensor and output tensors differ.

The datatype of the input tensor and output tensors differs.

The strides of the input tensor and output tensors differ and in-place operation is used (meaning, x and y pointers are equal).

The provided reserveSpaceSizeInBytes is less than the value returned by cudnnDropoutGetReserveSpaceSize().

cudnnSetDropoutDescriptor() has not been called on dropoutDesc with non-NULL states argument.

CUDNN_STATUS_EXECUTION_FAILED

The function failed to launch on the GPU.

cudnnDropoutGetReserveSpaceSize()#

This function is used to query the amount of reserve needed to run dropout with the input dimensions given by xDesc. The same reserve space is expected to be passed to cudnnDropoutForward() and cudnnDropoutBackward(), and its contents is expected to remain unchanged between cudnnDropoutForward() and cudnnDropoutBackward() calls.

cudnnStatus_t cudnnDropoutGetReserveSpaceSize(
    cudnnTensorDescriptor_t     xDesc,
    size_t                     *sizeInBytes)

Parameters

xDesc: Input. Handle to a previously initialized tensor descriptor, describing input to a dropout operation.
sizeInBytes: Output. Amount of GPU memory needed as reserve space to be able to run dropout with an input tensor descriptor specified by xDesc.

Returns

CUDNN_STATUS_SUCCESS: The query was successful.

cudnnDropoutGetStatesSize()#

This function is used to query the amount of space required to store the states of the random number generators used by the cudnnDropoutForward() function.

cudnnStatus_t cudnnDropoutGetStatesSize(
    cudnnHandle_t       handle,
    size_t             *sizeInBytes)

Parameters

handle: Input. Handle to a previously created cuDNN context.
sizeInBytes: Output. Amount of GPU memory needed to store random generator states.

Returns

CUDNN_STATUS_SUCCESS: The query was successful.

cudnnGetActivationDescriptor()#

This function queries a previously initialized generic activation descriptor object.

cudnnStatus_t cudnnGetActivationDescriptor(
        const cudnnActivationDescriptor_t   activationDesc,
        cudnnActivationMode_t              *mode,
        cudnnNanPropagation_t              *reluNanOpt,
        double                             *coef)

Parameters

activationDesc: Input. Handle to a previously created activation descriptor.
mode: Output. Enumerant to specify the activation mode.
reluNanOpt: Output. Enumerant to specify the Nan propagation mode.
coef: Output. Floating point number to specify the clipping threshold when the activation mode is set to CUDNN_ACTIVATION_CLIPPED_RELU or to specify the alpha coefficient when the activation mode is set to CUDNN_ACTIVATION_ELU.

Returns

CUDNN_STATUS_SUCCESS: The object was queried successfully.

cudnnGetActivationDescriptorSwishBeta()#

This function queries the current beta parameter set for SWISH activation.

cudnnStatus_t
cudnnGetActivationDescriptorSwishBeta(cudnnActivationDescriptor_t
activationDesc, double* swish_beta)

Parameters

activationDesc: Input. Handle to a previously created activation descriptor.
swish_beta: Output. Pointer to a double value that will receive the currently configured SWISH beta parameter.

Returns

CUDNN_STATUS_SUCCESS: The beta parameter was queried successfully.
CUDNN_STATUS_BAD_PARAM: At least one of activationDesc or swish_beta were NULL.

cudnnGetBatchNormalizationBackwardExWorkspaceSize()#

This function returns the amount of GPU memory workspace the user should allocate to be able to call cudnnGetBatchNormalizationBackwardExWorkspaceSize() function for the specified bnOps input setting. The workspace allocated will then be passed to the function cudnnGetBatchNormalizationBackwardExWorkspaceSize().

cudnnStatus_t cudnnGetBatchNormalizationBackwardExWorkspaceSize(
    cudnnHandle_t                       handle,
    cudnnBatchNormMode_t                mode,
    cudnnBatchNormOps_t                 bnOps,
    const cudnnTensorDescriptor_t       xDesc,
    const cudnnTensorDescriptor_t       yDesc,
    const cudnnTensorDescriptor_t       dyDesc,
    const cudnnTensorDescriptor_t       dzDesc,
    const cudnnTensorDescriptor_t       dxDesc,
    const cudnnTensorDescriptor_t       dBnScaleBiasDesc,
    const cudnnActivationDescriptor_t   activationDesc,
    size_t                              *sizeInBytes);

Parameters

handle: Input. Handle to a previously created cuDNN library descriptor. For more information, refer to cudnnHandle_t.
mode: Input. Mode of operation (spatial or per-activation). For more information, refer to cudnnBatchNormMode_t.
bnOps: Input. Mode of operation for the fast NHWC kernel. For more information, refer to cudnnBatchNormOps_t. This input can be used to set this function to perform either only the batch normalization, or batch normalization followed by activation, or batch normalization followed by element-wise addition and then activation.
xDesc, yDesc, dyDesc, dzDesc, dxDesc: Inputs. Tensor descriptors and pointers in the device memory for the layer’s x data, back propagated differential dy (inputs), the optional y input data, the optional dz output, and the dx output, which is the resulting differential with respect to x. For more information, refer to cudnnTensorDescriptor_t.
dBnScaleBiasDesc: Input. Shared tensor descriptor for the following six tensors: bnScaleData, bnBiasData, dBnScaleData, dBnBiasData, savedMean, and savedInvVariance. This is the shared tensor descriptor desc for the secondary tensor that was derived by cudnnDeriveBNTensorDescriptor(). The dimensions for this tensor descriptor are dependent on normalization mode. Note that the data type of this tensor descriptor must be float for FP16 and FP32 input tensors, and double for FP64 input tensors.
activationDesc: Input. Descriptor for the activation operation. When the bnOps input is set to either CUDNN_BATCHNORM_OPS_BN_ACTIVATION or CUDNN_BATCHNORM_OPS_BN_ADD_ACTIVATION, then this activation is used, otherwise user may pass NULL.
*sizeInBytes: Output. Amount of GPU memory required for the workspace, as determined by this function, to be able to execute the cudnnGetBatchNormalizationBackwardExWorkspaceSize() function with the specified bnOps input setting.

Returns

CUDNN_STATUS_SUCCESS

The computation was performed successfully.

CUDNN_STATUS_NOT_SUPPORTED

The function does not support the provided configuration.

CUDNN_STATUS_BAD_PARAM

At least one of the following conditions are met:

Number of xDesc, yDesc, or dxDesc tensor descriptor dimensions is not within the range of [4,5] (only 4D and 5D tensors are supported).

dBnScaleBiasDesc dimensions not 1xCx1x1 for 4D and 1xCx1x1x1 for 5D for spatial, and are not 1xCxHxW for 4D and 1xCxDxHxW for 5D for per-activation mode.

Dimensions or data types mismatch for any pair of xDesc, dyDesc, or dxDesc.

cudnnGetBatchNormalizationForwardTrainingExWorkspaceSize()#

This function has been deprecated in cuDNN 9.0.

This function returns the amount of GPU memory workspace the user should allocate to be able to call cudnnGetBatchNormalizationForwardTrainingExWorkspaceSize() function for the specified bnOps input setting. The workspace allocated should then be passed by the user to the function cudnnGetBatchNormalizationForwardTrainingExWorkspaceSize().

cudnnStatus_t cudnnGetBatchNormalizationForwardTrainingExWorkspaceSize(
    cudnnHandle_t                           handle,
    cudnnBatchNormMode_t                    mode,
    cudnnBatchNormOps_t                     bnOps,
    const cudnnTensorDescriptor_t           xDesc,
    const cudnnTensorDescriptor_t           zDesc,
    const cudnnTensorDescriptor_t           yDesc,
    const cudnnTensorDescriptor_t           bnScaleBiasMeanVarDesc,
    const cudnnActivationDescriptor_t       activationDesc,
    size_t                                  *sizeInBytes);

Parameters

handle: Input. Handle to a previously created cuDNN library descriptor. For more information, refer to cudnnHandle_t.
mode: Input. Mode of operation (spatial or per-activation). For more information, refer to cudnnBatchNormMode_t.
bnOps: Input. Mode of operation for the fast NHWC kernel. For more information, refer to cudnnBatchNormOps_t. This input can be used to set this function to perform either only the batch normalization, or batch normalization followed by activation, or batch normalization followed by element-wise addition and then activation.
xDesc, zDesc, yDesc: Inputs. Tensor descriptors and pointers in the device memory for the layer’s x data, the optional z input data, and the y output. zDesc is only needed when bnOps is CUDNN_BATCHNORM_OPS_BN_ADD_ACTIVATION, otherwise the user may pass NULL. For more information, refer to cudnnTensorDescriptor_t.
bnScaleBiasMeanVarDesc: Input. Shared tensor descriptor for the following six tensors: bnScaleData, bnBiasData, dBnScaleData, dBnBiasData, savedMean, and savedInvVariance. This is the shared tensor descriptor desc for the secondary tensor that was derived by cudnnDeriveBNTensorDescriptor(). The dimensions for this tensor descriptor are dependent on normalization mode. Note that the data type of this tensor descriptor must be float for FP16 and FP32 input tensors, and double for FP64 input tensors.
activationDesc: Input. Descriptor for the activation operation. When the bnOps input is set to either CUDNN_BATCHNORM_OPS_BN_ACTIVATION or CUDNN_BATCHNORM_OPS_BN_ADD_ACTIVATION, then this activation is used, otherwise user may pass NULL.
*sizeInBytes: Output. Amount of GPU memory required for the workspace, as determined by this function, to be able to execute the cudnnGetBatchNormalizationBackwardExWorkspaceSize() function with the specified bnOps input setting.

Returns

CUDNN_STATUS_SUCCESS

The computation was performed successfully.

CUDNN_STATUS_NOT_SUPPORTED

The function does not support the provided configuration.

CUDNN_STATUS_BAD_PARAM

At least one of the following conditions are met:

Number of xDesc, yDesc, or dxDesc tensor descriptor dimensions is not within the range of [4,5] (only 4D and 5D tensors are supported).

dBnScaleBiasDesc dimensions not 1xCx1x1 for 4D and 1xCx1x1x1 for 5D for spatial, and are not 1xCxHxW for 4D and 1xCxDxHxW for 5D for per-activation mode.

Dimensions or data types mismatch for any pair of xDesc or dyDesc.

cudnnGetBatchNormalizationTrainingExReserveSpaceSize()#

This function has been deprecated in cuDNN 9.0.

This function returns the amount of reserve GPU memory workspace the user should allocate for the batch normalization operation, for the specified bnOps input setting. In contrast to the workspace, the reserved space should be preserved between the forward and backward calls, and the data should not be altered.

cudnnStatus_t cudnnGetBatchNormalizationTrainingExReserveSpaceSize(
    cudnnHandle_t                       handle,
    cudnnBatchNormMode_t                mode,
    cudnnBatchNormOps_t                 bnOps,
    const cudnnActivationDescriptor_t   activationDesc,
    const cudnnTensorDescriptor_t       xDesc,
    size_t                              *sizeInBytes);

Parameters

handle: Input. Handle to a previously created cuDNN library descriptor. For more information, refer to cudnnHandle_t.
mode: Input. Mode of operation (spatial or per-activation). For more information, refer to cudnnBatchNormMode_t.
bnOps: Input. Mode of operation for the fast NHWC kernel. For more information, refer to cudnnBatchNormOps_t. This input can be used to set this function to perform either only the batch normalization, or batch normalization followed by activation, or batch normalization followed by element-wise addition and then activation.
xDesc: Input. Tensor descriptors and pointers in the device memory for the layer’s x data. For more information, refer to cudnnTensorDescriptor_t.
activationDesc: Input. Descriptor for the activation operation. When the bnOps input is set to either CUDNN_BATCHNORM_OPS_BN_ACTIVATION or CUDNN_BATCHNORM_OPS_BN_ADD_ACTIVATION, then this activation is used, otherwise user may pass NULL.
*sizeInBytes: Output. Amount of GPU memory reserved.

Returns

CUDNN_STATUS_SUCCESS

The computation was performed successfully.

CUDNN_STATUS_NOT_SUPPORTED

The function does not support the provided configuration.

CUDNN_STATUS_BAD_PARAM

At least one of the following conditions are met:

The xDesc tensor descriptor dimension is not within the [4,5] range (only 4D and 5D tensors are supported).

cudnnGetDropoutDescriptor()#

This function queries the fields of a previously initialized dropout descriptor.

cudnnStatus_t cudnnGetDropoutDescriptor(
    cudnnDropoutDescriptor_t    dropoutDesc,
    cudnnHandle_t               handle,
    float                      *dropout,
    void                       **states,
    unsigned long long         *seed)

Parameters

dropoutDesc: Input. Previously initialized dropout descriptor.
handle: Input. Handle to a previously created cuDNN context.
dropout: Output. The probability with which the value from input is set to 0 during the dropout layer.
states: Output. Pointer to user-allocated GPU memory that holds random number generator states.
seed: Output. Seed used to initialize random number generator states.

Returns

CUDNN_STATUS_SUCCESS: The call was successful.
CUDNN_STATUS_BAD_PARAM: One or more of the arguments was an invalid pointer.

cudnnGetFilter4dDescriptor()#

This function has been deprecated in cuDNN 9.0.

This function queries the parameters of the previously initialized Filter4d descriptor object.

cudnnStatus_t cudnnGetFilter4dDescriptor(
    const cudnnFilterDescriptor_t     filterDesc,
    cudnnDataType_t            *dataType,
    cudnnTensorFormat_t        *format,
    int                        *k,
    int                        *c,
    int                        *h,
    int                        *w)

Parameters

filterDesc: Input. Handle to a previously created filter descriptor.
datatype: Output. Data type.
format: Output. Type of format.
k: Output. Number of output feature maps.
c: Output. Number of input feature maps.
h: Output. Height of each filter.
w: Output. Width of each filter.

Returns

CUDNN_STATUS_SUCCESS: The object was set successfully.

cudnnGetFilterNdDescriptor()#

This function has been deprecated in cuDNN 9.0.

This function queries a previously initialized FilterNd descriptor object.

cudnnStatus_t cudnnGetFilterNdDescriptor(
    const cudnnFilterDescriptor_t   wDesc,
    int                             nbDimsRequested,
    cudnnDataType_t                *dataType,
    cudnnTensorFormat_t            *format,
    int                            *nbDims,
    int                             filterDimA[])

Parameters

wDesc: Input. Handle to a previously initialized filter descriptor.
nbDimsRequested: Input. Dimension of the expected filter descriptor. It is also the minimum size of the arrays filterDimA in order to be able to hold the results
datatype: Output. Data type.
format: Output. Type of format.
nbDims: Output. Actual dimension of the filter.
filterDimA: Output. Array of dimensions of at least nbDimsRequested that will be filled with the filter parameters from the provided filter descriptor.

Returns

CUDNN_STATUS_SUCCESS: The object was set successfully.
CUDNN_STATUS_BAD_PARAM: The parameter nbDimsRequested is negative.

cudnnGetFilterSizeInBytes()#

This function has been deprecated in cuDNN 9.0.

This function returns the size of the filter tensor in memory with respect to the given descriptor. It can be used to know the amount of GPU memory to be allocated to hold that filter tensor.

cudnnStatus_t
cudnnGetFilterSizeInBytes(const cudnnFilterDescriptor_t filterDesc, size_t *size);

Parameters

filterDesc: Input. handle to a previously initialized filter descriptor.
size: Output. size in bytes needed to hold the tensor in GPU memory.

Returns

CUDNN_STATUS_SUCCESS: filterDesc is valid.
CUDNN_STATUS_BAD_PARAM: filerDesc is invalid.

cudnnGetLRNDescriptor()#

This function retrieves values stored in the previously initialized LRN descriptor object.

cudnnStatus_t cudnnGetLRNDescriptor(
    cudnnLRNDescriptor_t    normDesc,
    unsigned               *lrnN,
    double                 *lrnAlpha,
    double                 *lrnBeta,
    double                 *lrnK)

Parameters

normDesc: Output. Handle to a previously created LRN descriptor.

lrnN, lrnAlpha, lrnBeta, lrnK Outputs. Pointers to receive values of parameters stored in the descriptor object. For more information, refer to cudnnSetLRNDescriptor(). Any of these pointers can be NULL (no value is returned for the corresponding parameter).

Returns

CUDNN_STATUS_SUCCESS: Function completed successfully.

cudnnGetNormalizationBackwardWorkspaceSize()#

This function returns the amount of GPU memory workspace the user should allocate to be able to call cudnnNormalizationBackward() function for the specified normOps and algo input setting. The workspace allocated will then be passed to the function cudnnNormalizationBackward().

cudnnStatus_t
cudnnGetNormalizationBackwardWorkspaceSize(cudnnHandle_t handle,
                                        cudnnNormMode_t mode,
                                        cudnnNormOps_t normOps,
                                        cudnnNormAlgo_t algo,
                                        const cudnnTensorDescriptor_t xDesc,
                                        const cudnnTensorDescriptor_t yDesc,
                                        const cudnnTensorDescriptor_t dyDesc,
                                            const cudnnTensorDescriptor_t dzDesc,
                                        const cudnnTensorDescriptor_t dxDesc,
                                        const cudnnTensorDescriptor_t dNormScaleBiasDesc,
                                    const cudnnActivationDescriptor_t activationDesc,
                                        const cudnnTensorDescriptor_t normMeanVarDesc,
                                        size_t *sizeInBytes,
                                        int groupCnt);

Parameters

handle: Input. Handle to a previously created cuDNN library descriptor. For more information, refer to cudnnHandle_t.
mode: Input. Mode of operation (per-channel or per-activation). For more information, refer to cudnnNormMode_t.
normOps: Input. Mode of post-operative. Currently CUDNN_NORM_OPS_NORM_ACTIVATION and CUDNN_NORM_OPS_NORM_ADD_ACTIVATION are only supported in the NHWC layout. For more information, refer to cudnnNormOps_t. This input can be used to set this function to perform either only the normalization, or normalization followed by activation, or normalization followed by element-wise addition and then activation.
algo: Input. Algorithm to be performed. For more information, refer to cudnnNormAlgo_t.
xDesc, yDesc, dyDesc, dzDesc, dxDesc: Inputs. Tensor descriptors and pointers in the device memory for the layer’s x data, back propagated differential dy (inputs), the optional y input data, the optional dz output, and the dx output, which is the resulting differential with respect to x. For more information, refer to cudnnTensorDescriptor_t.
dNormScaleBiasDesc: Input. Shared tensor descriptor for the following four tensors: normScaleData, normBiasData, dNormScaleData, dNormBiasData. The dimensions for this tensor descriptor are dependent on normalization mode. Note that the data type of this tensor descriptor must be float for FP16 and FP32 input tensors, and double for FP64 input tensors.
activationDesc: Input. Descriptor for the activation operation. When the normOps input is set to either CUDNN_NORM_OPS_NORM_ACTIVATION or CUDNN_NORM_OPS_NORM_ADD_ACTIVATION, then this activation is used, otherwise the user may pass NULL.
normMeanVarDesc: Input. Shared tensor descriptor for the following tensors: savedMean and savedInvVariance. The dimensions for this tensor descriptor are dependent on normalization mode. Note that the data type of this tensor descriptor must be float for FP16 and FP32 input tensors, and double for FP64 input tensors.
*sizeInBytes: Output. Amount of GPU memory required for the workspace, as determined by this function, to be able to execute the cudnnGetNormalizationForwardTrainingWorkspaceSize() function with the specified normOps input setting.
groutCnt: Input. Only support 1 for now.

Returns

CUDNN_STATUS_SUCCESS

The computation was performed successfully.

CUDNN_STATUS_NOT_SUPPORTED

The function does not support the provided configuration.

CUDNN_STATUS_BAD_PARAM

At least one of the following conditions are met:

Number of xDesc, yDesc, or dxDesc tensor descriptor dimensions is not within the range of [4,5] (only 4D and 5D tensors are supported).

dNormScaleBiasDesc dimensions not 1xCx1x1 for 4D and 1xCx1x1x1 for 5D for per-channel, and are not 1xCxHxW for 4D and 1xCxDxHxW for 5D for per-activation mode.

Dimensions or data types mismatch for any pair of xDesc, dyDesc, or dxDesc.

cudnnGetNormalizationForwardTrainingWorkspaceSize()#

This function returns the amount of GPU memory workspace the user should allocate to be able to call cudnnNormalizationForwardTraining() function for the specified normOps and algo input setting. The workspace allocated should then be passed by the user to the function cudnnNormalizationForwardTraining().

cudnnStatus_t
cudnnGetNormalizationForwardTrainingWorkspaceSize(cudnnHandle_t handle,
                                                cudnnNormMode_t mode,
                                                cudnnNormOps_t normOps,
                                                cudnnNormAlgo_t algo,
                                                const cudnnTensorDescriptor_t xDesc,
                                                const cudnnTensorDescriptor_t zDesc,
                                                const cudnnTensorDescriptor_t yDesc,
                                                    const cudnnTensorDescriptor_t normScaleBiasDesc,
                                                const cudnnActivationDescriptor_t activationDesc,
                                                const cudnnTensorDescriptor_t normMeanVarDesc,
                                                size_t *sizeInBytes,
                                                int groupCnt);

Parameters

handle: Input. Handle to a previously created cuDNN library descriptor. For more information, refer to cudnnHandle_t.
mode: Input. Mode of operation (per-channel or per-activation). For more information, refer to cudnnNormMode_t.
normOps: Input. Mode of post-operative. Currently CUDNN_NORM_OPS_NORM_ACTIVATION and CUDNN_NORM_OPS_NORM_ADD_ACTIVATION are only supported in the NHWC layout. For more information, refer to cudnnNormOps_t. This input can be used to set this function to perform either only the normalization, or normalization followed by activation, or normalization followed by element-wise addition and then activation.
algo: Input. Algorithm to be performed. For more information, refer to cudnnNormAlgo_t.
xDesc, zDesc, yDesc: Inputs. Tensor descriptors and pointers in the device memory for the layer’s x data, back propagated differential dy (inputs), the optional y input data, the optional dz output, and the dx output, which is the resulting differential with respect to x. For more information, refer to cudnnTensorDescriptor_t.
normScaleBiasDesc: Input. Shared tensor descriptor for the following four tensors: normScaleData and normBiasData. The dimensions for this tensor descriptor are dependent on normalization mode. Note that the data type of this tensor descriptor must be float for FP16 and FP32 input tensors, and double for FP64 input tensors.
activationDesc: Input. Descriptor for the activation operation. When the normOps input is set to either CUDNN_NORM_OPS_NORM_ACTIVATION or CUDNN_NORM_OPS_NORM_ADD_ACTIVATION, then this activation is used, otherwise the user may pass NULL.
normMeanVarDesc: Input. Shared tensor descriptor for the following tensors: savedMean and savedInvVariance. The dimensions for this tensor descriptor are dependent on normalization mode. Note that the data type of this tensor descriptor must be float for FP16 and FP32 input tensors, and double for FP64 input tensors.
*sizeInBytes: Output. Amount of GPU memory required for the workspace, as determined by this function, to be able to execute the cudnnGetNormalizationForwardTrainingWorkspaceSize() function with the specified normOps input setting.
groutCnt: Input. Only support 1 for now.

Returns

CUDNN_STATUS_SUCCESS

The computation was performed successfully.

CUDNN_STATUS_NOT_SUPPORTED

The function does not support the provided configuration.

CUDNN_STATUS_BAD_PARAM

At least one of the following conditions are met:

Number of xDesc, yDesc, or zDesc tensor descriptor dimensions is not within the range of [4,5] (only 4D and 5D tensors are supported).

normScaleBiasDesc dimensions not 1xCx1x1 for 4D and 1xCx1x1x1 for 5D for per-channel, and are not 1xCxHxW for 4D and 1xCxDxHxW for 5D for per-activation mode.

Dimensions or data types mismatch for any pair of xDesc or yDesc.

cudnnGetNormalizationTrainingReserveSpaceSize()#

This function returns the amount of reserve GPU memory workspace the user should allocate for the normalization operation, for the specified normOps input setting. In contrast to the workspace, the reserved space should be preserved between the forward and backward calls, and the data should not be altered.

cudnnStatus_t
cudnnGetNormalizationTrainingReserveSpaceSize(cudnnHandle_t handle,
                                            cudnnNormMode_t mode,
                                            cudnnNormOps_t normOps,
                                            cudnnNormAlgo_t algo,
                                            const cudnnActivationDescriptor_t activationDesc,
                                            const cudnnTensorDescriptor_t xDesc,
                                            size_t *sizeInBytes,
                                            int groupCnt);

Parameters

handle: Input. Handle to a previously created cuDNN library descriptor. For more information, refer to cudnnHandle_t.
mode: Input. Mode of operation (per-channel or per-activation). For more information, refer to cudnnNormMode_t.
normOps: Input. Mode of post-operative. Currently CUDNN_NORM_OPS_NORM_ACTIVATION and CUDNN_NORM_OPS_NORM_ADD_ACTIVATION are only supported in the NHWC layout. For more information, refer to cudnnNormOps_t. This input can be used to set this function to perform either only the normalization, or normalization followed by activation, or normalization followed by element-wise addition and then activation.
algo: Input. Algorithm to be performed. For more information, refer to cudnnNormAlgo_t.
xDesc: Input. Tensor descriptors for the layer’s x data. For more information, refer to cudnnTensorDescriptor_t.
activationDesc: Input. Descriptor for the activation operation. When the normOps input is set to either CUDNN_NORM_OPS_NORM_ACTIVATION or CUDNN_NORM_OPS_NORM_ADD_ACTIVATION, then this activation is used, otherwise the user may pass NULL.
*sizeInBytes: Output. Amount of GPU memory reserved.
groutCnt: Input. Only support 1 for now.

Returns

CUDNN_STATUS_SUCCESS

The computation was performed successfully.

CUDNN_STATUS_NOT_SUPPORTED

The function does not support the provided configuration.

CUDNN_STATUS_BAD_PARAM

At least one of the following conditions are met:

The xDesc tensor descriptor dimension is not within the [4,5] range (only 4D and 5D tensors are supported).

cudnnGetOpTensorDescriptor()#

This function returns the configuration of the passed tensor pointwise math descriptor.

cudnnStatus_t cudnnGetOpTensorDescriptor(
    const cudnnOpTensorDescriptor_t opTensorDesc,
    cudnnOpTensorOp_t               *opTensorOp,
    cudnnDataType_t                 *opTensorCompType,
    cudnnNanPropagation_t           *opTensorNanOpt)

Parameters

opTensorDesc: Input. Tensor pointwise math descriptor passed to get the configuration from.
opTensorOp: Output. Pointer to the tensor pointwise math operation type, associated with this tensor pointwise math descriptor.
opTensorCompType: Output. Pointer to the cuDNN data-type associated with this tensor pointwise math descriptor.
opTensorNanOpt: Output. Pointer to the NAN propagation option associated with this tensor pointwise math descriptor.

Returns

CUDNN_STATUS_SUCCESS: The function returned successfully.
CUDNN_STATUS_BAD_PARAM: Input tensor pointwise math descriptor passed is invalid.

cudnnGetPooling2dDescriptor()#

This function queries a previously created Pooling2d descriptor object.

cudnnStatus_t cudnnGetPooling2dDescriptor(
    const cudnnPoolingDescriptor_t      poolingDesc,
    cudnnPoolingMode_t                 *mode,
    cudnnNanPropagation_t              *maxpoolingNanOpt,
    int                                *windowHeight,
    int                                *windowWidth,
    int                                *verticalPadding,
    int                                *horizontalPadding,
    int                                *verticalStride,
    int                                *horizontalStride)

Parameters

poolingDesc: Input. Handle to a previously created pooling descriptor.
mode: Output. Enumerant to specify the pooling mode.
maxpoolingNanOpt: Output. Enumerant to specify the Nan propagation mode.
windowHeight: Output. Height of the pooling window.
windowWidth: Output. Width of the pooling window.
verticalPadding: Output. Size of vertical padding.
horizontalPadding: Output. Size of horizontal padding.
verticalStride: Output. Pooling vertical stride.
horizontalStride: Output. Pooling horizontal stride.

Returns

CUDNN_STATUS_SUCCESS: The object was set successfully.

cudnnGetPooling2dForwardOutputDim()#

This function provides the output dimensions of a tensor after Pooling2d has been applied.

cudnnStatus_t cudnnGetPooling2dForwardOutputDim(
    const cudnnPoolingDescriptor_t      poolingDesc,
    const cudnnTensorDescriptor_t       inputDesc,
    int                                *outN,
    int                                *outC,
    int                                *outH,
    int                                *outW)

Each dimension h and w of the output images is computed as follows:

outputDim = 1 + (inputDim + 2*padding - windowDim)/poolingStride;

Parameters

poolingDesc: Input. Handle to a previously initialized pooling descriptor.
inputDesc: Input. Handle to the previously initialized input tensor descriptor.
N: Output. Number of images in the output.
C: Output. Number of channels in the output.
H: Output. Height of images in the output.
W: Output. Width of images in the output.

Returns

CUDNN_STATUS_SUCCESS

The function launched successfully.

CUDNN_STATUS_BAD_PARAM

At least one of the following conditions are met:

poolingDesc has not been initialized.
poolingDesc or inputDesc has an invalid number of dimensions (2 and 4 respectively are required).

cudnnGetPoolingNdDescriptor()#

This function queries a previously initialized generic PoolingNd descriptor object.

cudnnStatus_t cudnnGetPoolingNdDescriptor(
const cudnnPoolingDescriptor_t      poolingDesc,
int                                 nbDimsRequested,
cudnnPoolingMode_t                 *mode,
cudnnNanPropagation_t              *maxpoolingNanOpt,
int                                *nbDims,
int                                 windowDimA[],
int                                 paddingA[],
int                                 strideA[])

Parameters

poolingDesc: Input. Handle to a previously created pooling descriptor.
nbDimsRequested: Input. Dimension of the expected pooling descriptor. It is also the minimum size of the arrays windowDimA, paddingA, and strideA in order to be able to hold the results.
mode: Output. Enumerant to specify the pooling mode.
maxpoolingNanOpt: Input. Enumerant to specify the Nan propagation mode.
nbDims: Output. Actual dimension of the pooling descriptor.
windowDimA: Output. Array of dimension of at least nbDimsRequested that will be filled with the window parameters from the provided pooling descriptor.
paddingA: Output. Array of dimension of at least nbDimsRequested that will be filled with the padding parameters from the provided pooling descriptor.
strideA: Output. Array of dimension at least nbDimsRequested that will be filled with the stride parameters from the provided pooling descriptor.

Returns

CUDNN_STATUS_SUCCESS: The object was queried successfully.
CUDNN_STATUS_NOT_SUPPORTED: The parameter nbDimsRequested is greater than CUDNN_DIM_MAX.

cudnnGetPoolingNdForwardOutputDim()#

This function provides the output dimensions of a tensor after PoolingNd has been applied.

cudnnStatus_t cudnnGetPoolingNdForwardOutputDim(
    const cudnnPoolingDescriptor_t  poolingDesc,
    const cudnnTensorDescriptor_t   inputDesc,
    int                             nbDims,
    int                             outDimA[])

Each dimension of the (nbDims-2)-D images of the output tensor is computed as follows:

outputDim = 1 + (inputDim + 2*padding - windowDim)/poolingStride;

Parameters

poolingDesc: Input. Handle to a previously initialized pooling descriptor.
inputDesc: Input. Handle to the previously initialized input tensor descriptor.
nbDims: Input. Number of dimensions in which pooling is to be applied.
outDimA: Output. Array of nbDims output dimensions.

Returns

CUDNN_STATUS_SUCCESS

The function launched successfully.

CUDNN_STATUS_BAD_PARAM

At least one of the following conditions are met:

poolingDesc has not been initialized.

The value of nbDims is inconsistent with the dimensionality of poolingDesc and inputDesc.

cudnnGetReduceTensorDescriptor()#

This function queries a previously initialized reduce tensor descriptor object.

cudnnStatus_t cudnnGetReduceTensorDescriptor(
    const cudnnReduceTensorDescriptor_t reduceTensorDesc,
    cudnnReduceTensorOp_t               *reduceTensorOp,
    cudnnDataType_t                     *reduceTensorCompType,
    cudnnNanPropagation_t               *reduceTensorNanOpt,
    cudnnReduceTensorIndices_t          *reduceTensorIndices,
    cudnnIndicesType_t                  *reduceTensorIndicesType)

Parameters

reduceTensorDesc: Input. Pointer to a previously initialized reduce tensor descriptor object.
reduceTensorOp: Output. Enumerant to specify the reduced tensor operation.
reduceTensorCompType: Output. Enumerant to specify the computation datatype of the reduction.
reduceTensorNanOpt: Input. Enumerant to specify the Nan propagation mode.
reduceTensorIndices: Output. Enumerant to specify the reduced tensor indices.
reduceTensorIndicesType: Output. Enumerant to specify the reduced tensor indices type.

Returns

CUDNN_STATUS_SUCCESS: The object was queried successfully.
CUDNN_STATUS_BAD_PARAM: reduceTensorDesc is NULL.

cudnnGetReductionIndicesSize()#

This is a helper function to return the minimum size of the index space to be passed to the reduction given the input and output tensors.

cudnnStatus_t cudnnGetReductionIndicesSize(
    cudnnHandle_t                       handle,
    const cudnnReduceTensorDescriptor_t reduceDesc,
    const cudnnTensorDescriptor_t       aDesc,
    const cudnnTensorDescriptor_t       cDesc,
    size_t                              *sizeInBytes)

Parameters

handle: Input. Handle to a previously created cuDNN library descriptor.
reduceDesc: Input. Pointer to a previously initialized reduce tensor descriptor object.
aDesc: Input. Pointer to the input tensor descriptor.
cDesc: Input. Pointer to the output tensor descriptor.
sizeInBytes: Output. Minimum size of the index space to be passed to the reduction.

Returns

CUDNN_STATUS_SUCCESS: The index space size is returned successfully.

cudnnGetReductionWorkspaceSize()#

This is a helper function to return the minimum size of the workspace to be passed to the reduction given the input and output tensors.

cudnnStatus_t cudnnGetReductionWorkspaceSize(
    cudnnHandle_t                       handle,
    const cudnnReduceTensorDescriptor_t reduceDesc,
    const cudnnTensorDescriptor_t       aDesc,
    const cudnnTensorDescriptor_t       cDesc,
    size_t                              *sizeInBytes)

Parameters

handle: Input. Handle to a previously created cuDNN library descriptor.
reduceDesc: Input. Pointer to a previously initialized reduce tensor descriptor object.
aDesc: Input. Pointer to the input tensor descriptor.
cDesc: Input. Pointer to the output tensor descriptor.
sizeInBytes: Output. Minimum size of the index space to be passed to the reduction.

Returns

CUDNN_STATUS_SUCCESS: The workspace size is returned successfully.

cudnnGetTensor4dDescriptor()#

This function queries the parameters of the previously initialized Tensor4d descriptor object.

cudnnStatus_t cudnnGetTensor4dDescriptor(
    const cudnnTensorDescriptor_t  tensorDesc,
    cudnnDataType_t         *dataType,
    int                     *n,
    int                     *c,
    int                     *h,
    int                     *w,
    int                     *nStride,
    int                     *cStride,
    int                     *hStride,
    int                     *wStride)

Parameters

tensorDesc: Input. Handle to a previously initialized tensor descriptor.
datatype: Output. Data type.
n: Output. Number of images.
c: Output. Number of feature maps per image.
h: Output. Height of each feature map.
w: Output. Width of each feature map.
nStride: Output. Stride between two consecutive images.
cStride: Output. Stride between two consecutive feature maps.
hStride: Output. Stride between two consecutive rows.
wStride: Output. Stride between two consecutive columns.

Returns

CUDNN_STATUS_SUCCESS: The operation succeeded.

cudnnGetTensorNdDescriptor()#

This function retrieves values stored in a previously initialized TensorNd descriptor object.

cudnnStatus_t cudnnGetTensorNdDescriptor(
    const cudnnTensorDescriptor_t   tensorDesc,
    int                             nbDimsRequested,
    cudnnDataType_t                *dataType,
    int                            *nbDims,
    int                             dimA[],
    int                             strideA[])

Parameters

tensorDesc: Input. Handle to a previously initialized tensor descriptor.
nbDimsRequested: Input. Number of dimensions to extract from a given tensor descriptor. It is also the minimum size of the arrays dimA and strideA. If this number is greater than the resulting nbDims[0], only nbDims[0] dimensions will be returned.
datatype: Output. Data type.
nbDims: Output. Actual number of dimensions of the tensor will be returned in nbDims[0].
dimA: Output. Array of dimensions of at least nbDimsRequested that will be filled with the dimensions from the provided tensor descriptor.
strideA: Output. Array of dimensions of at least nbDimsRequested that will be filled with the strides from the provided tensor descriptor.

Returns

CUDNN_STATUS_SUCCESS: The results were returned successfully.
CUDNN_STATUS_BAD_PARAM: Either tensorDesc or nbDims pointer is NULL.

cudnnGetTensorSizeInBytes()#

This function returns the size of the tensor in memory in respect to the given descriptor. This function can be used to know the amount of GPU memory to be allocated to hold that tensor.

cudnnStatus_t cudnnGetTensorSizeInBytes(
    const cudnnTensorDescriptor_t   tensorDesc,
    size_t                         *size)

Parameters

tensorDesc: Input. Handle to a previously initialized tensor descriptor.
size: Output. Size in bytes needed to hold the tensor in GPU memory.

Returns

CUDNN_STATUS_SUCCESS: The results were returned successfully.

cudnnGetTensorTransformDescriptor()#

This function has been deprecated in cuDNN 9.0.

This function returns the values stored in a previously initialized tensor transform descriptor.

cudnnStatus_t cudnnGetTensorTransformDescriptor(
    cudnnTensorTransformDescriptor_t transformDesc,
    uint32_t nbDimsRequested,
    cudnnTensorFormat_t *destFormat,
    int32_t padBeforeA[],
    int32_t padAfterA[],
    uint32_t foldA[],
    cudnnFoldingDirection_t *direction);

Parameters

transformDesc: Input. A previously initialized tensor transform descriptor.
nbDimsRequested: Input. The number of dimensions to consider. For more information, refer to Tensor Descriptor.
destFormat: Output. The transform format that will be returned.
padBeforeA[]: Output. An array filled with the amount of padding to add before each dimension. The dimension of this padBeforeA[] parameter is equal to nbDimsRequested.
padAfterA[]: Output. An array filled with the amount of padding to add after each dimension. The dimension of this padBeforeA[] parameter is equal to nbDimsRequested.
foldA[]: Output. An array that was filled with the folding parameters for each spatial dimension. The dimension of this foldA[] array is nbDimsRequested-2.
direction: Output. The setting that selects folding or unfolding. For more information, refer to cudnnFoldingDirection_t.

Returns

CUDNN_STATUS_SUCCESS: The results were obtained successfully.
CUDNN_STATUS_BAD_PARAM: If transformDesc is NULL or if nbDimsRequested is less than 3 or greater than CUDNN_DIM_MAX.

cudnnInitTransformDest()#

This function has been deprecated in cuDNN 9.0.

This function initializes and returns a destination tensor descriptor destDesc for tensor transform operations. The initialization is done with the desired parameters described in the transform descriptor cudnnTensorDescriptor_t.

cudnnStatus_t cudnnInitTransformDest(
    const cudnnTensorTransformDescriptor_t transformDesc,
    const cudnnTensorDescriptor_t srcDesc,
    cudnnTensorDescriptor_t destDesc,
    size_t *destSizeInBytes);

The returned Tensor descriptor will be packed.

Parameters

transformDesc: Input. Handle to a previously initialized tensor transform descriptor.
srcDesc: Input. Handle to a previously initialized tensor descriptor.
destDesc: Output. Handle of the tensor descriptor that will be initialized and returned.
destSizeInBytes: Output. A pointer to hold the size, in bytes, of the new tensor.

Returns

CUDNN_STATUS_SUCCESS: The tensor descriptor was initialized successfully.
CUDNN_STATUS_BAD_PARAM: If either srcDesc or destDesc is NULL, or if the tensor descriptors nbDims is incorrect. For more information, refer to Tensor Descriptor..
CUDNN_STATUS_NOT_SUPPORTED: If the provided configuration is not 4D.
CUDNN_STATUS_EXECUTION_FAILED: Function failed to launch on the GPU.

cudnnLRNCrossChannelBackward()#

This function performs the backward LRN layer computation.

cudnnStatus_t cudnnLRNCrossChannelBackward(
    cudnnHandle_t                    handle,
    cudnnLRNDescriptor_t             normDesc,
    cudnnLRNMode_t                   lrnMode,
    const void                      *alpha,
    const cudnnTensorDescriptor_t    yDesc,
    const void                      *y,
    const cudnnTensorDescriptor_t    dyDesc,
    const void                      *dy,
    const cudnnTensorDescriptor_t    xDesc,
    const void                      *x,
    const void                      *beta,
    const cudnnTensorDescriptor_t    dxDesc,
    void                            *dx)

Supported formats are: positive-strided, NCHW and NHWC for 4D x and y, and only NCDHW DHW-packed for 5D (for both x and y). Only non-overlapping 4D and 5D tensors are supported. NCHW layout is preferred for performance.

Parameters

handle

Input. Handle to a previously created cuDNN library descriptor.

normDesc

Input. Handle to a previously initialized LRN parameter descriptor.

lrnMode

Input. LRN layer mode of operation. Currently only CUDNN_LRN_CROSS_CHANNEL_DIM1 is implemented. Normalization is performed along the tensor’s dimA[1].

alpha, beta

Inputs. Pointers to scaling factors (in host memory) used to blend the layer output value with prior value in the destination tensor as follows:

dstValue = alpha[0]*resultValue + beta[0]*priorDstValue

For more information, refer to Scaling Parameters.

yDesc, y

Inputs. Tensor descriptor and pointer in device memory for the layer’s y data.

dyDesc, dy

Inputs. Tensor descriptor and pointer in device memory for the layer’s input cumulative loss differential data dy (including error backpropagation).

xDesc, x

Inputs. Tensor descriptor and pointer in device memory for the layer’s x data. Note that these values are not modified during backpropagation.

dxDesc, dx

Outputs. Tensor descriptor and pointer in device memory for the layer’s resulting cumulative loss differential data dx (including error backpropagation).

Returns

CUDNN_STATUS_SUCCESS

The computation was performed successfully.

CUDNN_STATUS_BAD_PARAM

At least one of the following conditions are met:

One of the tensor pointers x, y is NULL.

Number of input tensor dimensions is 2 or less.

LRN descriptor parameters are outside of their valid ranges.

One of the tensor parameters is 5D but is not in NCDHW DHW-packed format.

CUDNN_STATUS_NOT_SUPPORTED

The function does not support the provided configuration. Some examples include:

Any of the input tensor datatypes is not the same as any of the output tensor datatype.

Any pairwise tensor dimensions mismatch for x, y, dx, or dy.

Any tensor parameters strides are negative.

cudnnLRNCrossChannelForward()#

This function performs the forward LRN layer computation.

cudnnStatus_t cudnnLRNCrossChannelForward(
    cudnnHandle_t                    handle,
    cudnnLRNDescriptor_t             normDesc,
    cudnnLRNMode_t                   lrnMode,
    const void                      *alpha,
    const cudnnTensorDescriptor_t    xDesc,
    const void                      *x,
    const void                      *beta,
    const cudnnTensorDescriptor_t    yDesc,
    void                            *y)

Supported formats are: positive-strided, NCHW and NHWC for 4D x and y, and only NCDHW DHW-packed for 5D (for both x and y). Only non-overlapping 4D and 5D tensors are supported. NCHW layout is preferred for performance.

Parameters

handle

Input. Handle to a previously created cuDNN library descriptor.

normDesc

Input. Handle to a previously initialized LRN parameter descriptor.

lrnMode

Input. LRN layer mode of operation. Currently only CUDNN_LRN_CROSS_CHANNEL_DIM1 is implemented. Normalization is performed along the tensor’s dimA[1].

alpha, beta

Inputs. Pointers to scaling factors (in host memory) used to blend the layer output value with prior value in the destination tensor as follows:

dstValue = alpha[0]*resultValue + beta[0]*priorDstValue

For more information, refer to Scaling Parameters.

xDesc, yDesc

Inputs. Tensor descriptor objects for the input and output tensors.

x

Input. Input tensor data pointer in device memory.

y

Output. Output tensor data pointer in device memory.

Returns

CUDNN_STATUS_SUCCESS

The computation was performed successfully.

CUDNN_STATUS_BAD_PARAM

At least one of the following conditions are met:

One of the tensor pointers x, y is NULL.

Number of input tensor dimensions is 2 or less.

LRN descriptor parameters are outside of their valid ranges.

One of the tensor parameters is 5D but is not in NCDHW DHW-packed format.

CUDNN_STATUS_NOT_SUPPORTED

The function does not support the provided configuration. Some examples include:

Any of the input tensor datatypes is not the same as any of the output tensor datatype.

x and y tensor dimensions mismatch.

Any tensor parameters strides are negative.

cudnnNormalizationBackward()#

This function has been deprecated in cuDNN 9.0.

This function performs backward normalization layer computation that is specified by mode. Per-channel normalization layer is based on the Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift paper.

cudnnStatus_t
cudnnNormalizationBackward(cudnnHandle_t handle,
                        cudnnNormMode_t mode,
                        cudnnNormOps_t normOps,
                        cudnnNormAlgo_t algo,
                        const void *alphaDataDiff,
                            const void *betaDataDiff,
                        const void *alphaParamDiff,
                        const void *betaParamDiff,
                        const cudnnTensorDescriptor_t xDesc,
                        const void *xData,
                        const cudnnTensorDescriptor_t yDesc,
                        const void *yData,
                        const cudnnTensorDescriptor_t dyDesc,
                        const void *dyData,
                        const cudnnTensorDescriptor_t dzDesc,
                        void *dzData,
                        const cudnnTensorDescriptor_t dxDesc,
                        void *dxData,
                        const cudnnTensorDescriptor_t dNormScaleBiasDesc,
                        const void *normScaleData,
                        const void *normBiasData,
                        void *dNormScaleData,
                        void *dNormBiasData,
                        double epsilon,
                        const cudnnTensorDescriptor_t normMeanVarDesc,
                        const void *savedMean,
                        const void *savedInvVariance,
                        cudnnActivationDescriptor_t activationDesc,
                        void *workSpace,
                        size_t workSpaceSizeInBytes,
                        void *reserveSpace,
                        size_t reserveSpaceSizeInBytes,
                        int groupCnt)

Only 4D and 5D tensors are supported.

The epsilon value has to be the same during training, backpropagation, and inference. This workspace is not required to be clean. Moreover, the workspace does not have to remain unchanged between the forward and backward pass, as it is not used for passing any information.

This function can accept a *workspace pointer to the GPU workspace, and workSpaceSizeInBytes, the size of the workspace, from the user.

The normOps input can be used to set this function to perform either only the normalization, or normalization followed by activation, or normalization followed by element-wise addition and then activation.

When the tensor layout is NCHW, higher performance can be obtained when HW-packed tensors are used for x, dy, dx.

Higher performance for CUDNN_NORM_PER_CHANNEL mode can be obtained when the following conditions are true:

All tensors, namely, x, y, dz, dy, and dx must be NHWC-fully packed, and must be of the type CUDNN_DATA_HALF.

The tensor C dimension should be a multiple of 4.

The input parameter mode must be set to CUDNN_NORM_PER_CHANNEL.

The input parameter algo must be set to CUDNN_NORM_ALGO_PERSIST.

Workspace is not NULL.

workSpaceSizeInBytes is equal to or larger than the amount required by cudnnGetNormalizationBackwardWorkspaceSize().

reserveSpaceSizeInBytes is equal to or larger than the amount required by cudnnGetNormalizationTrainingReserveSpaceSize().

The content in reserveSpace stored by cudnnNormalizationForwardTraining() must be preserved.

Parameters

handle

Input. Handle to a previously created cuDNN library descriptor. For more information, refer to cudnnHandle_t.

mode

Input. Mode of operation (per-channel or per-activation). For more information, refer to cudnnNormMode_t.

normOps

Input. Mode of post-operative. Currently CUDNN_NORM_OPS_NORM_ACTIVATION and CUDNN_NORM_OPS_NORM_ADD_ACTIVATION are only supported in the NHWC layout. For more information, refer to cudnnNormOps_t. This input can be used to set this function to perform either only the normalization, or normalization followed by activation, or normalization followed by element-wise addition and then activation.

algo

Input. Algorithm to be performed. For more information, refer to cudnnNormAlgo_t.

*alphaDataDiff, *betaDataDiff

Inputs. Pointers to scaling factors (in host memory) used to blend the gradient output dx with a prior value in the destination tensor as follows:

dstValue = alpha[0]*resultValue + beta[0]*priorDstValue

For more information, refer to Scaling Parameters.

*alphaParamDiff, *betaParamDiff

Inputs. Pointers to scaling factors (in host memory) used to blend the gradient outputs dNormScaleData and dNormBiasData with prior values in the destination tensor as follows:

dstValue = alpha[0]*resultValue + beta[0]*priorDstValue

For more information, refer to Scaling Parameters.

xDesc, *xData, yDesc, *yData, dyDesc, *dyData

Inputs. Tensor descriptors and pointers in the device memory for the layer’s x data, backpropagated gradient input dy, the original forward output y data. yDesc and yData are not needed if normOps is set to CUDNN_NORM_OPS_NORM, users may pass NULL. For more information, refer to cudnnTensorDescriptor_t.

dzDesc, dxDesc

Inputs. Tensor descriptors and pointers in the device memory for the computed gradient output dz and dx. dzDesc is not needed when normOps is CUDNN_NORM_OPS_NORM or CUDNN_NORM_OPS_NORM_ACTIVATION, users may pass NULL. For more information, refer to cudnnTensorDescriptor_t.

*dzData, *dxData

Outputs. Tensor descriptors and pointers in the device memory for the computed gradient output dz and dx. *dzData is not needed when normOps is CUDNN_NORM_OPS_NORM or CUDNN_NORM_OPS_NORM_ACTIVATION, users may pass NULL. For more information, refer to cudnnTensorDescriptor_t.

dNormScaleBiasDesc

Input. Shared tensor descriptor for the following six tensors: normScaleData, normBiasData, dNormScaleData, and dNormBiasData. The dimensions for this tensor descriptor are dependent on normalization mode.

The data type of this tensor descriptor must be float for FP16 and FP32 input tensors and double for FP64 input tensors. For more information, refer to cudnnTensorDescriptor_t.

*normScaleData

Input. Pointer in the device memory for the normalization scale parameter (in the Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift paper, the quantity scale is referred to as gamma).

*normBiasData

Input. Pointers in the device memory for the normalization bias parameter (in the Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift paper, bias is referred to as beta). This parameter is used only when activation should be performed.

*dNormScaleData, *dNormBiasData

Outputs. Pointers in the device memory for the gradients of normScaleData and normBiasData, respectively.

epsilon

Input. Epsilon value used in normalization formula. Its value should be equal to or greater than zero. The same epsilon value should be used in forward and backward functions.

normMeanVarDesc

Input. Shared tensor descriptor for the following tensors: savedMean and savedInvVariance. The dimensions for this tensor descriptor are dependent on normalization mode.

The data type of this tensor descriptor must be float for FP16 and FP32 input tensors and double for FP64 input tensors. For more information, refer to cudnnTensorDescriptor_t.

*savedMean, *savedInvVariance

Inputs. Optional cache parameters containing saved intermediate results computed during the forward pass. For this to work correctly, the layer’s x and normScaleData, normBiasData data has to remain unchanged until this backward function is called. Note that both these parameters can be NULL but only at the same time. It is recommended to use this cache since the memory overhead is relatively small.

activationDesc

Input. Descriptor for the activation operation. When the normOps input is set to either CUDNN_NORM_OPS_NORM_ACTIVATION or CUDNN_NORM_OPS_NORM_ADD_ACTIVATION then this activation is used, otherwise the user may pass NULL.

workspace

Input. Pointer to the GPU workspace.

workSpaceSizeInBytes

Input. The size of the workspace. It must be large enough to trigger the fast NHWC semi-persistent kernel by this function.

*reserveSpace

Input. Pointer to the GPU workspace for the reserveSpace.

reserveSpaceSizeInBytes

Input. The size of the reserveSpace. It must be equal or larger than the amount required by cudnnGetNormalizationTrainingReserveSpaceSize().

groutCnt

Input. Only support 1 for now.

Returns

CUDNN_STATUS_SUCCESS

The computation was performed successfully.

CUDNN_STATUS_NOT_SUPPORTED

The function does not support the provided configuration.

CUDNN_STATUS_BAD_PARAM

At least one of the following conditions are met:

Any of the pointers alphaDataDiff, betaDataDiff, alphaParamDiff, betaParamDiff, xData, dyData, dxData, normScaleData, dNormScaleData, and dNormBiasData is NULL.

The number of xDesc, yDesc, or dxDesc tensor descriptor dimensions is not within the range of [4,5] (only 4D and 5D tensors are supported).

dNormScaleBiasDesc dimensions not 1xCx1x1 for 4D and 1xCx1x1x1 for 5D for per-channel, and are not 1xCxHxW for 4D and 1xCxDxHxW for 5D for per-activation mode.

Exactly one of savedMean, savedInvVariance pointers is NULL.

epsilon value is less than zero.

Dimensions or data types mismatch for any pair of xDesc, dyDesc, dxDesc, dNormScaleBiasDesc, or normMeanVarDesc.

cudnnNormalizationForwardInference()#

This function has been deprecated in cuDNN 9.0.

This function performs the forward normalization layer computation for the inference phase. Per-channel normalization layer is based on the Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift paper.

cudnnStatus_t
cudnnNormalizationForwardInference(cudnnHandle_t handle,
                                cudnnNormMode_t mode,
                                cudnnNormOps_t normOps,
                                cudnnNormAlgo_t algo,
                                const void *alpha,
                                const void *beta,
                                const cudnnTensorDescriptor_t xDesc,
                                const void *x,
                                const cudnnTensorDescriptor_t normScaleBiasDesc,
                                const void *normScale,
                                const void *normBias,
                                const cudnnTensorDescriptor_t normMeanVarDesc,
                                const void *estimatedMean,
                                const void *estimatedVariance,
                                const cudnnTensorDescriptor_t zDesc,
                                const void *z,
                                cudnnActivationDescriptor_t activationDesc,
                                const cudnnTensorDescriptor_t yDesc,
                                void *y,
                                double epsilon,
                                int groupCnt);

Only 4D and 5D tensors are supported.

The input transformation performed by this function is defined as:

y = beta*y + alpha *[normBias + (normScale * (x-estimatedMean)/sqrt(epsilon + estimatedVariance)]

The epsilon value has to be the same during training, backpropagation, and inference.

For the training phase, refer to cudnnNormalizationForwardTraining().

Higher performance can be obtained when HW-packed tensors are used for all of x and y.

Parameters

handle

Input. Handle to a previously created cuDNN library descriptor. For more information, refer to cudnnHandle_t.

mode

Input. Mode of operation (per-channel or per-activation). For more information, refer to cudnnNormMode_t.

normOps

Input. Mode of post-operative. Currently CUDNN_NORM_OPS_NORM_ACTIVATION and CUDNN_NORM_OPS_NORM_ADD_ACTIVATION are only supported in the NHWC layout. For more information, refer to cudnnNormOps_t. This input can be used to set this function to perform either only the normalization, or normalization followed by activation, or normalization followed by element-wise addition and then activation.

algo

Input. Algorithm to be performed. For more information, refer to cudnnNormAlgo_t.

*alpha, *beta

Inputs. Pointers to scaling factors (in host memory) used to blend the layer output value with prior value in the destination tensor as follows:

dstValue = alpha[0]*resultValue + beta[0]*priorDstValue

For more information, refer to Scaling Parameters.

xDesc, yDesc

Inputs. Handles to the previously initialized tensor descriptors.

*x

Input. Data pointer to GPU memory associated with the tensor descriptor xDesc, for the layer’s x input data.

*y

Output. Data pointer to GPU memory associated with the tensor descriptor yDesc, for the y output of the normalization layer.

zDesc, *z

Inputs. Tensor descriptors and pointers in device memory for residual addition to the result of the normalization operation, prior to the activation. zDesc and *z are optional and are only used when normOps is CUDNN_NORM_OPS_NORM_ADD_ACTIVATION, otherwise users may pass NULL. When in use, z should have exactly the same dimension as x and the final output y. For more information, refer to cudnnTensorDescriptor_t.

Since normOps is only supported for CUDNN_NORM_OPS_NORM, we can set these to NULL for now.

normScaleBiasDesc, normScale, normBias

Inputs. Tensor descriptors and pointers in device memory for the normalization scale and bias parameters (in the Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift paper, bias is referred to as beta and scale as gamma).

normMeanVarDesc, estimatedMean, estimatedVariance

Inputs. Mean and variance tensors and their tensor descriptors. The estimatedMean and estimatedVariance inputs, accumulated during the training phase from the cudnnNormalizationForwardTraining() call, should be passed as inputs here.

activationDesc

Input. Descriptor for the activation operation. When the normOps input is set to either CUDNN_NORM_OPS_NORM_ACTIVATION or CUDNN_NORM_OPS_NORM_ADD_ACTIVATION then this activation is used, otherwise the user may pass NULL. Since normOps is only supported for CUDNN_NORM_OPS_NORM, we can set these to NULL for now.

epsilon

Input. Epsilon value used in the normalization formula. Its value should be equal to or greater than zero.

groutCnt

Input. Only support 1 for now.

Returns

CUDNN_STATUS_SUCCESS

The computation was performed successfully.

CUDNN_STATUS_NOT_SUPPORTED

A compute or data type other than what is supported was chosen, or an unknown algorithm type was chosen.

CUDNN_STATUS_BAD_PARAM

At least one of the following conditions are met:

One of the pointers alpha, beta, x, y, normScale, normBias, estimatedMean, and estimatedInvVariance is NULL.

The number of xDesc or yDesc tensor descriptor dimensions is not within the range of [4,5] (only 4D and 5D tensors are supported).

normScaleBiasDesc and normMeanVarDesc dimensions are not 1xCx1x1 for 4D and 1xCx1x1x1 for 5D for per-channel, and are not 1xCxHxW for 4D and 1xCxDxHxW for 5D for per-activation mode.

epsilon value is less than zero.

Dimensions or data types mismatch for xDesc and yDesc.

cudnnNormalizationForwardTraining()#

This function has been deprecated in cuDNN 9.0.

This function performs the forward normalization layer computation for the training phase. Depending on mode, different normalization operations will be performed. Per-channel layer is based on the Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift paper.

cudnnStatus_t
cudnnNormalizationForwardTraining(cudnnHandle_t handle,
                                cudnnNormMode_t mode,
                                cudnnNormOps_t normOps,
                                cudnnNormAlgo_t algo,
                                const void *alpha,
                                const void *beta,
                                const cudnnTensorDescriptor_t xDesc,
                                    const void *xData,
                                const cudnnTensorDescriptor_t normScaleBiasDesc,
                                const void *normScale,
                                const void *normBias,
                                double exponentialAverageFactor,
                                const cudnnTensorDescriptor_t normMeanVarDesc,
                                void *resultRunningMean,
                                void *resultRunningVariance,
                                double epsilon,
                                void *resultSaveMean,
                                void *resultSaveInvVariance,
                                cudnnActivationDescriptor_t activationDesc,
                            const cudnnTensorDescriptor_t zDesc,
                                const void *zData,
                                const cudnnTensorDescriptor_t yDesc,
                                void *yData,
                                void *workspace,
                                size_t workSpaceSizeInBytes,
                                void *reserveSpace,
                                size_t reserveSpaceSizeInBytes,
                                int groupCnt);

Only 4D and 5D tensors are supported.

The epsilon value has to be the same during training, back propagation, and inference.

For the inference phase, refer to cudnnNormalizationForwardInference().

Higher performance can be obtained when HW-packed tensors are used for both x and y.

This API will trigger the new semi-persistent NHWC kernel when the following conditions are true:

All tensors, namely, xData, yData must be NHWC-fully packed and must be of the type CUDNN_DATA_HALF.

The tensor C dimension should be a multiple of 4.

The input parameter mode must be set to CUDNN_NORM_PER_CHANNEL.

The input parameter algo must be set to CUDNN_NORM_ALGO_PERSIST.

workspace is not NULL.

workSpaceSizeInBytes is equal to or larger than the amount required by cudnnGetNormalizationForwardTrainingWorkspaceSize().

reserveSpaceSizeInBytes is equal to or larger than the amount required by cudnnGetNormalizationTrainingReserveSpaceSize().

The content in reserveSpace stored by cudnnNormalizationForwardTraining() must be preserved.

This workspace is not required to be clean. Moreover, the workspace does not have to remain unchanged between the forward and backward pass, as it is not used for passing any information. This extended function can accept a *workspace pointer to the GPU workspace, and workSpaceSizeInBytes, the size of the workspace, from the user.

The normOps input can be used to set this function to perform either only the normalization, or normalization followed by activation, or normalization followed by element-wise addition and then activation.

Only 4D and 5D tensors are supported. The epsilon value has to be the same during the training, the backpropagation, and the inference.

When the tensor layout is NCHW, higher performance can be obtained when HW-packed tensors are used for xData, yData.

Parameters

handle

Input. Handle to a previously created cuDNN library descriptor. For more information, refer to cudnnHandle_t.

mode

Input. Mode of operation (per-channel or per-activation). For more information, refer to cudnnNormMode_t.

normOps

Input. Mode of post-operative. Currently CUDNN_NORM_OPS_NORM_ACTIVATION and CUDNN_NORM_OPS_NORM_ADD_ACTIVATION are only supported in the NHWC layout. For more information, refer to cudnnNormOps_t. This input can be used to set this function to perform either only the normalization, or normalization followed by activation, or normalization followed by element-wise addition and then activation.

algo

Input. Algorithm to be performed. For more information, refer to cudnnNormAlgo_t.

*alpha, *beta

Inputs. Pointers to scaling factors (in host memory) used to blend the layer output value with prior value in the destination tensor as follows:

dstValue = alpha[0]*resultValue + beta[0]*priorDstValue

For more information, refer to Scaling Parameters.

xDesc, yDesc

Inputs. Handles to the previously initialized tensor descriptors.

*xData

Input. Data pointer to GPU memory associated with the tensor descriptor xDesc, for the layer’s x input data.

*yData

Output. Data pointer to GPU memory associated with the tensor descriptor yDesc, for the y output of the normalization layer.

zDesc, *zData

Inputs. Tensor descriptors and pointers in device memory for residual addition to the result of the normalization operation, prior to the activation. zDesc and *zData are optional and are only used when normOps is CUDNN_NORM_OPS_NORM_ADD_ACTIVATION, otherwise the user may pass NULL. When in use, z should have exactly the same dimension as xData and the final output yData. For more information, refer to cudnnTensorDescriptor_t.

normScaleBiasDesc, normScale, normBias

Inputs. Tensor descriptors and pointers in device memory for the normalization scale and bias parameters (in the Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift paper, bias is referred to as beta and scale as gamma). The dimensions for the tensor descriptor are dependent on the normalization mode.

exponentialAverageFactor

Input. Factor used in the moving average computation as follows:

runningMean = runningMean*(1-factor) + newMean*factor

Use a factor=1/(1+n) at N-th call to the function to get Cumulative Moving Average (CMA) behavior such that:

CMA[n] = (x[1]+...+x[n])/n

This is proved below:

Writing
CMA[n+1] = (n*CMA[n]+x[n+1])/(n+1)
= ((n+1)*CMA[n]-CMA[n])/(n+1) + x[n+1]/(n+1)
= CMA[n]*(1-1/(n+1))+x[n+1]*1/(n+1)
= CMA[n]*(1-factor) + x(n+1)*factor

normMeanVarDesc

Inputs. Tensor descriptor used for following tensors: resultRunningMean, resultRunningVariance, resultSaveMean, resultSaveInvVariance.

*resultRunningMean, *resultRunningVariance

Inputs/Outputs. Pointers to the running mean and running variance data. Both these pointers can be NULL but only at the same time. The value stored in resultRunningVariance (or passed as an input in inference mode) is the sample variance and is the moving average of variance[x] where the variance is computed either over batch or spatial+batch dimensions depending on the mode. If these pointers are not NULL, the tensors should be initialized to some reasonable values or to 0.

epsilon

Input. Epsilon value used in the normalization formula. Its value should be equal to or greater than zero.

*resultSaveMean, *resultSaveInvVariance

Outputs. Optional cache parameters containing saved intermediate results computed during the forward pass. For this to work correctly, the layer’s x and normScale, normBias data has to remain unchanged until this backward function is called. Note that both these parameters can be NULL but only at the same time. It is recommended to use this cache since the memory overhead is relatively small.

activationDesc

Input. The tensor descriptor for the activation operation. When the normOps input is set to either CUDNN_NORM_OPS_NORM_ACTIVATION or CUDNN_NORM_OPS_NORM_ADD_ACTIVATION then this activation is used, otherwise the user may pass NULL.

*workspace, workSpaceSizeInBytes

Inputs. *workspace is a pointer to the GPU workspace, and workSpaceSizeInBytes is the size of the workspace. When *workspace is not NULL and *workSpaceSizeInBytes is large enough, and the tensor layout is NHWC and the data type configuration is supported, then this function will trigger a semi-persistent NHWC kernel for normalization. The workspace is not required to be clean. Also, the workspace does not need to remain unchanged between the forward and backward passes.

*reserveSpace

Input. Pointer to the GPU workspace for the reserveSpace.

reserveSpaceSizeInBytes

Input. The size of the reserveSpace. Must be equal or larger than the amount required by cudnnGetNormalizationTrainingReserveSpaceSize().

groutCnt

Input. Only support 1 for now.

Returns

CUDNN_STATUS_SUCCESS

The computation was performed successfully.

CUDNN_STATUS_NOT_SUPPORTED

The function does not support the provided configuration.

CUDNN_STATUS_BAD_PARAM

At least one of the following conditions are met:

One of the pointers alpha, beta, xData, yData, normScale, and normBias is NULL.

The number of xDesc or yDesc tensor descriptor dimensions is not within the [4,5] range (only 4D and 5D tensors are supported).

normScaleBiasDesc dimensions are not 1xCx1x1 for 4D and 1xCx1x1x1 for 5D for per-channel mode, and are not 1xCxHxW for 4D and 1xCxDxHxW for 5D for per-activation mode.

Exactly one of resultSaveMean, resultSaveInvVariance pointers are NULL.

Exactly one of resultRunningMean, resultRunningInvVariance pointers are NULL.

epsilon value is less than zero.

Dimensions or data types mismatch for xDesc or yDesc.

cudnnOpTensor()#

This function has been deprecated in cuDNN 9.0.

This function implements the equation C = op(alpha1[0] * A, alpha2[0] * B) + beta[0] * C, given the tensors A, B, and C and the scaling factors alpha1, alpha2, and beta. The op to use is indicated by the descriptor cudnnOpTensorDescriptor_t, meaning, the type of opTensorDesc. Currently-supported ops are listed by the cudnnOpTensorOp_t enum.

cudnnStatus_t cudnnOpTensor(
    cudnnHandle_t                     handle,
    const cudnnOpTensorDescriptor_t   opTensorDesc,
    const void                       *alpha1,
    const cudnnTensorDescriptor_t     aDesc,
    const void                       *A,
    const void                       *alpha2,
    const cudnnTensorDescriptor_t     bDesc,
    const void                       *B,
    const void                       *beta,
    const cudnnTensorDescriptor_t     cDesc,
    void                             *C)

The following restrictions on the input and destination tensors apply:

Each dimension of the input tensor A must match the corresponding dimension of the destination tensor C, and each dimension of the input tensor B must match the corresponding dimension of the destination tensor C or must be equal to 1. In the latter case, the same value from the input tensor B for those dimensions will be used to blend into the C tensor.

Data Types of the Input Tensors A and B, and the Destination Tensor C, Satisfaction Requirements#
`opTensorCompType` in `opTensorDesc`	Tensor A	Tensor B	Destination Tensor C
`FLOAT`	`FLOAT`	`FLOAT`	`FLOAT`
`FLOAT`	`INT8`	`INT8`	`FLOAT`
`FLOAT`	`HALF`	`HALF`	`FLOAT`
`FLOAT`	`BFLOAT16`	`BFLOAT16`	`FLOAT`
`DOUBLE`	`DOUBLE`	`DOUBLE`	`DOUBLE`
`FLOAT`	`FLOAT`	`FLOAT`	`HALF`
`FLOAT`	`HALF`	`HALF`	`HALF`
`FLOAT`	`INT8`	`INT8`	`INT8`
`FLOAT`	`FLOAT`	`FLOAT`	`INT8`
`FLOAT`	`FLOAT`	`FLOAT`	`BFLOAT16`
`FLOAT`	`BFLOAT16`	`BFLOAT16`	`BFLOAT16`

CUDNN_TENSOR_NCHW_VECT_C is not supported as input tensor format. All tensors up to dimension five (5) are supported. This routine does not support tensor formats beyond these dimensions.

Parameters

handle

Input. Handle to a previously created cuDNN context.

opTensorDesc

Input. Handle to a previously initialized op tensor descriptor.

alpha1, alpha2, beta

Inputs. Pointers to scaling factors (in host memory) used to blend the source value with prior value in the destination tensor as follows:

dstValue = alpha[0]*resultValue + beta[0]*priorDstValue

For more information, refer to Scaling Parameters.

aDesc, bDesc, cDesc

Inputs. Handle to a previously initialized tensor descriptor.

A, B

Inputs. Pointer to data of the tensors described by the aDesc and bDesc descriptors, respectively.

C

Input/Output. Pointer to data of the tensor described by the cDesc descriptor.

Returns

CUDNN_STATUS_SUCCESS

The function executed successfully.

CUDNN_STATUS_NOT_SUPPORTED

The function does not support the provided configuration. Some examples include:

The dimensions of the bias tensor and the output tensor dimensions are above 5.

opTensorCompType is not set as stated above.

CUDNN_STATUS_BAD_PARAM

The data type of the destination tensor C is unrecognized, or the restrictions on the input and destination tensors, stated above, are not met.

CUDNN_STATUS_EXECUTION_FAILED

The function failed to launch on the GPU.

cudnnOpsVersionCheck()#

Cross-library version checker. Each sublibrary has a version checker that checks whether its own version matches that of its dependencies.

cudnnStatus_t cudnnOpsVersionCheck(void)

Returns

CUDNN_STATUS_SUCCESS: The version check passed.
CUDNN_STATUS_SUBLIBRARY_VERSION_MISMATCH: The versions are inconsistent.

cudnnPoolingBackward()#

This function has been deprecated in cuDNN 9.0.

This function computes the gradient of a pooling operation.

cudnnStatus_t cudnnPoolingBackward(
    cudnnHandle_t                       handle,
    const cudnnPoolingDescriptor_t      poolingDesc,
    const void                         *alpha,
    const cudnnTensorDescriptor_t       yDesc,
    const void                         *y,
    const cudnnTensorDescriptor_t       dyDesc,
    const void                         *dy,
    const cudnnTensorDescriptor_t       xDesc,
    const void                         *xData,
    const void                         *beta,
    const cudnnTensorDescriptor_t       dxDesc,
    void                               *dx)

As of cuDNN version 6.0, a deterministic algorithm is implemented for max backwards pooling. This algorithm can be chosen via the pooling mode enum of poolingDesc. The deterministic algorithm has been measured to be up to 50% slower than the legacy max backwards pooling algorithm, or up to 20% faster, depending upon the use case.

Tensor vectorization is not supported for any tensor descriptor arguments in this function. Best performance is expected when using HW-packed tensors. Only 2 and 3 spatial dimensions are supported.

Parameters

handle

Input. Handle to a previously created cuDNN context.

poolingDesc

Input. Handle to the previously initialized pooling descriptor.

alpha, beta

Inputs. Pointers to scaling factors (in host memory) used to blend the computation result with prior value in the output layer as follows:

dstValue = alpha[0]*result + beta[0]*priorDstValue

For more information, refer to Scaling Parameters.

yDesc

Input. Handle to the previously initialized input tensor descriptor. Can be NULL for avg pooling.

y

Input. Data pointer to GPU memory associated with the tensor descriptor yDesc. Can be NULL for avg pooling.

dyDesc

Input. Handle to the previously initialized input differential tensor descriptor. Must be of type FLOAT, DOUBLE, HALF, or BFLOAT16. For more information, refer to cudnnDataType_t.

dy

Input. Data pointer to GPU memory associated with the tensor descriptor dyData.

xDesc

Input. Handle to the previously initialized output tensor descriptor. Can be NULL for avg pooling.

x

Input. Data pointer to GPU memory associated with the output tensor descriptor xDesc. Can be NULL for avg pooling.

dxDesc

Input. Handle to the previously initialized output differential tensor descriptor. Must be of type FLOAT, DOUBLE, HALF, or BFLOAT16. For more information, refer to cudnnDataType_t.

dx

Output. Data pointer to GPU memory associated with the output tensor descriptor dxDesc.

Returns

CUDNN_STATUS_SUCCESS

The function launched successfully.

CUDNN_STATUS_BAD_PARAM

At least one of the following conditions are met:

The dimensions n, c, h, w of the yDesc and dyDesc tensors differ.

The strides nStride, cStride, hStride, wStride of the yDesc and dyDesc tensors differ.

The dimensions n, c, h, w of the dxDesc and dxDesc tensors differ.

The strides nStride, cStride, hStride, wStride of the xDesc and dxDesc tensors differ.

The datatype of the four tensors differ.

CUDNN_STATUS_NOT_SUPPORTED

The function does not support the provided configuration. Some examples includes:

The wStride of input tensor or output tensor is not 1.

CUDNN_STATUS_EXECUTION_FAILED

The function failed to launch on the GPU.

cudnnPoolingForward()#

This function has been deprecated in cuDNN 9.0.

This function computes pooling of input values (meaning, the maximum or average of several adjacent values) to produce an output with smaller height and/or width.

cudnnStatus_t cudnnPoolingForward(
    cudnnHandle_t                    handle,
    const cudnnPoolingDescriptor_t   poolingDesc,
    const void                      *alpha,
    const cudnnTensorDescriptor_t    xDesc,
    const void                      *x,
    const void                      *beta,
    const cudnnTensorDescriptor_t    yDesc,
    void                            *y)

All tensor formats are supported, best performance is expected when using HW-packed tensors. Only 2 and 3 spatial dimensions are allowed. Vectorized tensors are only supported if they have 2 spatial dimensions.

The dimensions of the output tensor yDesc can be smaller or bigger than the dimensions advised by the routine cudnnGetPooling2dForwardOutputDim() or cudnnGetPoolingNdForwardOutputDim().

For average pooling, the compute type is float even for integer input and output data type. Output round is nearest-even and clamp to the most negative or most positive value of type if out of range.

Parameters

handle

Input. Handle to a previously created cuDNN context.

poolingDesc

Input. Handle to a previously initialized pooling descriptor.

alpha, beta

Inputs. Pointers to scaling factors (in host memory) used to blend the computation result with prior value in the output layer as follows:

dstValue = alpha[0]*result + beta[0]*priorDstValue

For more information, refer to Scaling Parameters.

xDesc

Input. Handle to the previously initialized input tensor descriptor. Must be of type FLOAT, DOUBLE, HALF, INT8, INT8x4, INT8x32, or BFLOAT16. For more information, refer to cudnnDataType_t.

x

Input. Data pointer to GPU memory associated with the tensor descriptor xDesc.

yDesc

Input. Handle to the previously initialized output tensor descriptor. Must be of type FLOAT, DOUBLE, HALF, INT8, INT8x4, INT8x32, or BFLOAT16. For more information, refer to cudnnDataType_t.

y

Output. Data pointer to GPU memory associated with the output tensor descriptor yDesc.

Returns

CUDNN_STATUS_SUCCESS

The function launched successfully.

CUDNN_STATUS_BAD_PARAM

At least one of the following conditions are met:

The dimensions n, c of the input tensor and output tensors differ.

The datatype of the input tensor and output tensors differs.

CUDNN_STATUS_NOT_SUPPORTED

The function does not support the provided configuration.

CUDNN_STATUS_EXECUTION_FAILED

The function failed to launch on the GPU.

cudnnReduceTensor()#

This function has been deprecated in cuDNN 9.0.

This function reduces tensor A by implementing the equation C = alpha * reduce op ( A ) + beta * C, given tensors A and C and scaling factors alpha and beta. The reduction op to use is indicated by the descriptor reduceTensorDesc. Currently-supported ops are listed by the cudnnReduceTensorOp_t enum.

cudnnStatus_t cudnnReduceTensor(
    cudnnHandle_t                           handle,
    const cudnnReduceTensorDescriptor_t     reduceTensorDesc,
    void                                   *indices,
    size_t                                  indicesSizeInBytes,
    void                                   *workspace,
    size_t                                  workspaceSizeInBytes,
    const void                             *alpha,
    const cudnnTensorDescriptor_t           aDesc,
    const void                             *A,
    const void                             *beta,
    const cudnnTensorDescriptor_t           cDesc,
    void                                   *C)

Each dimension of the output tensor C must match the corresponding dimension of the input tensor A or must be equal to 1. The dimensions equal to 1 indicate the dimensions of A to be reduced.

The implementation will generate indices for the min and max ops only, as indicated by the cudnnReduceTensorIndices_t enum of the reduceTensorDesc. Requesting indices for the other reduction ops results in an error. The data type of the indices is indicated by the cudnnIndicesType_t enum; currently only the 32-bit (unsigned int) type is supported.

The indices returned by the implementation are not absolute indices but relative to the dimensions being reduced. The indices are also flattened, meaning, not coordinate tuples.

The data types of the tensors A and C must match if of type double. In this case, alpha and beta and the computation enum of reduceTensorDesc are all assumed to be of type double.

The HALF and INT8 data types may be mixed with the FLOAT data types. In these cases, the computation enum of reduceTensorDesc is required to be of type FLOAT.

Up to dimension 8, all tensor formats are supported. Beyond those dimensions, this routine is not supported.

Parameters

handle

Input. Handle to a previously created cuDNN context.

reduceTensorDesc

Input. Handle to a previously initialized reduce tensor descriptor.

indices

Output. Handle to a previously allocated space for writing indices.

indicesSizeInBytes

Input. Size of the above previously allocated space.

workspace

Input. Handle to a previously allocated space for the reduction implementation.

workspaceSizeInBytes

Input. Size of the above previously allocated space.

alpha, beta

Inputs. Pointers to scaling factors (in host memory) used to blend the source value with prior value in the destination tensor as follows:

dstValue = alpha[0]*resultValue + beta[0]*priorDstValue

For more information, refer to Scaling Parameters.

aDesc, cDesc

Inputs. Handle to a previously initialized tensor descriptor.

A

Input. Pointer to data of the tensor described by the aDesc descriptor.

C

Input/Output. Pointer to data of the tensor described by the cDesc descriptor.

Returns

CUDNN_STATUS_SUCCESS

The function executed successfully.

CUDNN_STATUS_NOT_SUPPORTED

The function does not support the provided configuration. Some examples include:

The dimensions of the input tensor and the output tensor are above 8.
reduceTensorCompType is not set as stated above.

CUDNN_STATUS_BAD_PARAM

The corresponding dimensions of the input and output tensors all match, or the conditions in the above paragraphs are unmet.

CUDNN_INVALID_VALUE

The allocations for the indices or workspace are insufficient.

CUDNN_STATUS_EXECUTION_FAILED

The function failed to launch on the GPU.

cudnnRestoreDropoutDescriptor()#

This function restores a dropout descriptor to a previously saved-off state.

cudnnStatus_t cudnnRestoreDropoutDescriptor(
    cudnnDropoutDescriptor_t dropoutDesc,
    cudnnHandle_t            handle,
    float                    dropout,
    void                    *states,
    size_t                   stateSizeInBytes,
    unsigned long long       seed)

Parameters

dropoutDesc: Input/Output. Previously created dropout descriptor.
handle: Input. Handle to a previously created cuDNN context.
dropout: Input. Probability with which the value from an input tensor is set to 0 when performing dropout.
states: Input. Pointer to GPU memory that holds random number generator states initialized by a prior call to cudnnSetDropoutDescriptor().
stateSizeInBytes: Input. Size in bytes of buffer holding random number generator states.
seed: Input. Seed used in prior calls to cudnnSetDropoutDescriptor() that initialized states buffer. Using a different seed from this has no effect. A change of seed, and subsequent update to random number generator states can be achieved by calling cudnnSetDropoutDescriptor().

Returns

CUDNN_STATUS_SUCCESS: The call was successful.
CUDNN_STATUS_INVALID_VALUE: The states buffer size (as indicated in stateSizeInBytes) is too small.

cudnnScaleTensor()#

This function has been deprecated in cuDNN 9.0.

This function scales all the elements of a tensor by a given factor.

cudnnStatus_t cudnnScaleTensor(
    cudnnHandle_t                   handle,
    const cudnnTensorDescriptor_t   yDesc,
    void                           *y,
    const void                     *alpha)

Parameters

handle: Input. Handle to a previously created cuDNN context.
yDesc: Input. Handle to a previously initialized tensor descriptor.
y: Input/Output. Pointer to data of the tensor described by the yDesc descriptor.
alpha: Input. Pointer in Host memory to a single value that all elements of the tensor will be scaled with. For more information, refer to Scaling Parameters.

Returns

CUDNN_STATUS_SUCCESS: The function launched successfully.
CUDNN_STATUS_NOT_SUPPORTED: The function does not support the provided configuration.
CUDNN_STATUS_BAD_PARAM: One of the provided pointers is NIL.
CUDNN_STATUS_EXECUTION_FAILED: The function failed to launch on the GPU.

cudnnSetActivationDescriptor()#

This function initializes a previously created generic activation descriptor object.

cudnnStatus_t cudnnSetActivationDescriptor(
    cudnnActivationDescriptor_t         activationDesc,
    cudnnActivationMode_t               mode,
    cudnnNanPropagation_t               reluNanOpt,
    double                              coef)

Parameters

activationDesc: Input/Output. Handle to a previously created activation descriptor.
mode: Input. Enumerant to specify the activation mode.
reluNanOpt: Input. Enumerant to specify the Nan propagation mode.
coef: Input. Floating point number. When the activation mode (refer to cudnnActivationMode_t) is set to CUDNN_ACTIVATION_CLIPPED_RELU, this input specifies the clipping threshold; and when the activation mode is set to CUDNN_ACTIVATION_RELU, this input specifies the upper bound.

Returns

CUDNN_STATUS_SUCCESS: The object was set successfully.
CUDNN_STATUS_BAD_PARAM: mode or reluNanOpt has an invalid enumerant value.

cudnnSetActivationDescriptorSwishBeta()#

This function sets the beta parameter of the SWISH activation function to swish_beta.

cudnnStatus_t cudnnSetActivationDescriptorSwishBeta(cudnnActivationDescriptor_t activationDesc, double swish_beta)

Parameters

activationDesc: Input/Output. Handle to a previously created activation descriptor.
swish_beta: Input. The value to set the SWISH activations’ beta parameter to.

Returns

CUDNN_STATUS_SUCCESS: The value was set successfully.
CUDNN_STATUS_BAD_PARAM: The activation descriptor is a NULL pointer.

cudnnSetDropoutDescriptor()#

This function initializes a previously created dropout descriptor object. If the states argument is equal to NULL, then the random number generator states won’t be initialized, and only the dropout value will be set. The user is expected not to change the memory pointed at by states for the duration of the computation.

cudnnStatus_t cudnnSetDropoutDescriptor(
    cudnnDropoutDescriptor_t    dropoutDesc,
    cudnnHandle_t               handle,
    float                       dropout,
    void                       *states,
    size_t                      stateSizeInBytes,
    unsigned long long          seed)

When the states argument is not NULL, a cuRAND initialization kernel is invoked by cudnnSetDropoutDescriptor(). This kernel requires a substantial amount of GPU memory for the stack. Memory is released when the kernel finishes. The CUDNN_STATUS_ALLOC_FAILED status is returned when no sufficient free memory is available for the GPU stack.

Parameters

dropoutDesc: Input/Output. Previously created dropout descriptor object.
handle: Input. Handle to a previously created cuDNN context.
dropout: Input. The probability with which the value from input is set to zero during the dropout layer.
states: Output. Pointer to user-allocated GPU memory that will hold random number generator states.
stateSizeInBytes: Input. Specifies the size in bytes of the provided memory for the states.
seed: Input. Seed used to initialize random number generator states.

Returns

CUDNN_STATUS_SUCCESS: The call was successful.
CUDNN_STATUS_INVALID_VALUE: The sizeInBytes argument is less than the value returned by cudnnDropoutGetStatesSize().
CUDNN_STATUS_ALLOC_FAILED: The function failed to temporarily extend the GPU stack.
CUDNN_STATUS_EXECUTION_FAILED: The function failed to launch on the GPU.
CUDNN_STATUS_INTERNAL_ERROR: Internally used CUDA functions returned an error status.

cudnnSetFilter4dDescriptor()#

This function has been deprecated in cuDNN 9.0.

This function initializes a previously created filter descriptor object into a 4D filter. The layout of the filters must be contiguous in memory.

cudnnStatus_t cudnnSetFilter4dDescriptor(
    cudnnFilterDescriptor_t    filterDesc,
    cudnnDataType_t            dataType,
    cudnnTensorFormat_t        format,
    int                        k,
    int                        c,
    int                        h,
    int                        w)

Tensor format CUDNN_TENSOR_NHWC has limited support in cudnnConvolutionForward(), cudnnConvolutionBackwardData(), and cudnnConvolutionBackwardFilter().

Parameters

filterDesc

Input/Output. Handle to a previously created filter descriptor.

datatype

Input. Data type.

format

Input. Type of the filter layout format. If this input is set to CUDNN_TENSOR_NCHW, which is one of the enumerant values allowed by cudnnTensorFormat_t descriptor, then the layout of the filter is in the form of KCRS, where:

K represents the number of output feature maps

C is the number of input feature maps

R is the number of rows per filter

S is the number of columns per filter

If this input is set to CUDNN_TENSOR_NHWC, then the layout of the filter is in the form of KRSC.

k

Input. Number of output feature maps.

c

Input. Number of input feature maps.

h

Input. Height of each filter.

w

Input. Width of each filter.

Returns

CUDNN_STATUS_SUCCESS: The object was set successfully.
CUDNN_STATUS_BAD_PARAM: At least one of the parameters k, c, h, w is negative or dataType or format has an invalid enumerant value.

cudnnSetFilterNdDescriptor()#

This function has been deprecated in cuDNN 9.0.

This function initializes a previously created filter descriptor object. The layout of the filters must be contiguous in memory.

cudnnStatus_t cudnnSetFilterNdDescriptor(
    cudnnFilterDescriptor_t filterDesc,
    cudnnDataType_t         dataType,
    cudnnTensorFormat_t     format,
    int                     nbDims,
    const int               filterDimA[])

The tensor format CUDNN_TENSOR_NHWC has limited support in cudnnConvolutionForward(), cudnnConvolutionBackwardData(), and cudnnConvolutionBackwardFilter().

Parameters

filterDesc

Input/Output. Handle to a previously created filter descriptor.

datatype

Input. Data type.

format

Input.Type of the filter layout format. If this input is set to CUDNN_TENSOR_NCHW, which is one of the enumerant values allowed by cudnnTensorFormat_t descriptor, then the layout of the filter is as follows:

For N=4, a 4D filter descriptor, the filter layout is in the form of KCRS:

K represents the number of output feature maps

C is the number of input feature maps

R is the number of rows per filter

S is the number of columns per filter

For N=3, a 3D filter descriptor, the number S (number of columns per filter) is omitted.

For N=5 and greater, the layout of the higher dimensions immediately follows RS.

On the other hand, if this input is set to CUDNN_TENSOR_NHWC, then the layout of the filter is as follows:

For N=4, a 4D filter descriptor, the filter layout is in the form of KRSC.

For N=3, a 3D filter descriptor, the number S (number of columns per filter) is omitted, and the layout of C immediately follows R.

For N=5 and greater, the layout of the higher dimensions are inserted between S and C.

nbDims

Input. Dimension of the filter.

filterDimA

Input. Array of dimension nbDims containing the size of the filter for each dimension.

Returns

CUDNN_STATUS_SUCCESS: The object was set successfully.
CUDNN_STATUS_BAD_PARAM: At least one of the elements of the array filterDimA is negative or dataType or format has an invalid enumerant value.
CUDNN_STATUS_NOT_SUPPORTED: The parameter nbDims exceeds CUDNN_DIM_MAX.

cudnnSetLRNDescriptor()#

This function initializes a previously created LRN descriptor object.

cudnnStatus_t cudnnSetLRNDescriptor(
    cudnnLRNDescriptor_t   normDesc,
    unsigned               lrnN,
    double                 lrnAlpha,
    double                 lrnBeta,
    double                 lrnK)
Note

Macros CUDNN_LRN_MIN_N, CUDNN_LRN_MAX_N, CUDNN_LRN_MIN_K, CUDNN_LRN_MIN_BETA defined in cudnn.h specify valid ranges for parameters.

Values of double parameters will be cast down to the tensor datatype during computation.

Parameters

normDesc: Output. Handle to a previously created LRN descriptor.
lrnN: Input. Normalization window width in elements. The LRN layer uses a window [center-lookBehind, center+lookAhead], where lookBehind = floor( (lrnN-1)/2 ), lookAhead = lrnN-lookBehind-1. So for n=10, the window is [k-4...k...k+5] with a total of 10 samples. For the DivisiveNormalization layer, the window has the same extent as above in all spatial dimensions (dimA[2], dimA[3], dimA[4]). By default, lrnN is set to 5 in cudnnCreateLRNDescriptor().
lrnAlpha: Input. Value of the alpha variance scaling parameter in the normalization formula. Inside the library code, this value is divided by the window width for LRN and by (window width)^#spatialDimensions for DivisiveNormalization. By default, this value is set to 1e-4 in cudnnCreateLRNDescriptor().
lrnBeta: Input. Value of the beta power parameter in the normalization formula. By default, this value is set to 0.75 in cudnnCreateLRNDescriptor().
lrnK: Input. Value of the k parameter in the normalization formula. By default, this value is set to 2.0.

Returns

CUDNN_STATUS_SUCCESS: The object was set successfully.
CUDNN_STATUS_BAD_PARAM: One of the input parameters was out of valid range as described above.

cudnnSetOpTensorDescriptor()#

This function initializes a tensor pointwise math descriptor.

cudnnStatus_t cudnnSetOpTensorDescriptor(
    cudnnOpTensorDescriptor_t   opTensorDesc,
    cudnnOpTensorOp_t           opTensorOp,
    cudnnDataType_t             opTensorCompType,
    cudnnNanPropagation_t       opTensorNanOpt)

Parameters

opTensorDesc: Output. Pointer to the structure holding the description of the tensor pointwise math descriptor.
opTensorOp: Input. Tensor pointwise math operation for this tensor pointwise math descriptor.
opTensorCompType: Input. Computation datatype for this tensor pointwise math descriptor.
opTensorNanOpt: Input. NAN propagation policy.

Returns

CUDNN_STATUS_SUCCESS: The function returned successfully.
CUDNN_STATUS_BAD_PARAM: At least one of the input parameters passed is invalid.

cudnnSetPooling2dDescriptor()#

This function initializes a previously created generic pooling descriptor object into a 2D description.

cudnnStatus_t cudnnSetPooling2dDescriptor(
    cudnnPoolingDescriptor_t    poolingDesc,
    cudnnPoolingMode_t          mode,
    cudnnNanPropagation_t       maxpoolingNanOpt,
    int                         windowHeight,
    int                         windowWidth,
    int                         verticalPadding,
    int                         horizontalPadding,
    int                         verticalStride,
    int                         horizontalStride)

Parameters

poolingDesc: Input/Output. Handle to a previously created pooling descriptor.
mode: Input. Enumerant to specify the pooling mode.
maxpoolingNanOpt: Input. Enumerant to specify the Nan propagation mode.
windowHeight: Input. Height of the pooling window.
windowWidth: Input. Width of the pooling window.
verticalPadding: Input. Size of vertical padding.
horizontalPadding: Input. Size of horizontal padding
verticalStride: Input. Pooling vertical stride.
horizontalStride: Input. Pooling horizontal stride.

Returns

CUDNN_STATUS_SUCCESS: The object was set successfully.
CUDNN_STATUS_BAD_PARAM: At least one of the parameters windowHeight, windowWidth, verticalStride, horizontalStride is negative or mode or maxpoolingNanOpt has an invalid enumerate value.

cudnnSetPoolingNdDescriptor()#

This function initializes a previously created generic pooling descriptor object.

cudnnStatus_t cudnnSetPoolingNdDescriptor(
    cudnnPoolingDescriptor_t     poolingDesc,
    const cudnnPoolingMode_t     mode,
    const cudnnNanPropagation_t  maxpoolingNanOpt,
    int                          nbDims,
    const int                    windowDimA[],
    const int                    paddingA[],
    const int                    strideA[])

Parameters

poolingDesc: Input/Output. Handle to a previously created pooling descriptor.
mode: Input. Enumerant to specify the pooling mode.
maxpoolingNanOpt: Input. Enumerant to specify the Nan propagation mode.
nbDims: Input. Dimension of the pooling operation. Must be greater than zero.
windowDimA: Input. Array of dimension nbDims containing the window size for each dimension. The value of array elements must be greater than zero.
paddingA: Input. Array of dimension nbDims containing the padding size for each dimension. Negative padding is allowed.
strideA: Input. Array of dimension nbDims containing the striding size for each dimension. The value of array elements must be greater than zero (meaning, negative striding size is not allowed).

Returns

CUDNN_STATUS_SUCCESS: The object was initialized successfully.
CUDNN_STATUS_NOT_SUPPORTED: If (nbDims > CUDNN_DIM_MAX-2).
CUDNN_STATUS_BAD_PARAM: Either nbDims, or at least one of the elements of the arrays windowDimA or strideA is negative, or mode or maxpoolingNanOpt has an invalid enumerate value.

cudnnSetReduceTensorDescriptor()#

This function initializes a previously created reduce tensor descriptor object.

cudnnStatus_t cudnnSetReduceTensorDescriptor(
    cudnnReduceTensorDescriptor_t   reduceTensorDesc,
    cudnnReduceTensorOp_t           reduceTensorOp,
    cudnnDataType_t                 reduceTensorCompType,
    cudnnNanPropagation_t           reduceTensorNanOpt,
    cudnnReduceTensorIndices_t      reduceTensorIndices,
    cudnnIndicesType_t              reduceTensorIndicesType)

Parameters

reduceTensorDesc: Input/Output. Handle to a previously created reduce tensor descriptor.
reduceTensorOp: Input. Enumerant to specify the reduced tensor operation.
reduceTensorCompType: Input. Enumerant to specify the computation datatype of the reduction.
reduceTensorNanOpt: Input. Enumerant to specify the Nan propagation mode.
reduceTensorIndices: Input. Enumerant to specify the reduced tensor indices.
reduceTensorIndicesType: Input. Enumerant to specify the reduced tensor indices type.

Returns

CUDNN_STATUS_SUCCESS: The object was set successfully.
CUDNN_STATUS_BAD_PARAM: reduceTensorDesc is NULL (reduceTensorOp, reduceTensorCompType, reduceTensorNanOpt, reduceTensorIndices or reduceTensorIndicesType has an invalid enumerant value).

cudnnSetSpatialTransformerNdDescriptor()#

This function initializes a previously created generic spatial transformer descriptor object.

cudnnStatus_t cudnnSetSpatialTransformerNdDescriptor(
    cudnnSpatialTransformerDescriptor_t     stDesc,
    cudnnSamplerType_t                      samplerType,
    cudnnDataType_t                         dataType,
    const int                               nbDims,
    const int                               dimA[])

Parameters

stDesc: Input/Output. Previously created spatial transformer descriptor object.
samplerType: Input. Enumerant to specify the sampler type.
dataType: Input. Data type.
nbDims: Input. Dimension of the transformed tensor.
dimA: Input. Array of dimension nbDims containing the size of the transformed tensor for every dimension.

Returns

CUDNN_STATUS_SUCCESS

The call was successful.

CUDNN_STATUS_BAD_PARAM

At least one of the following conditions are met:

Either stDesc or dimA is NULL.

Either dataType or samplerType has an invalid enumerant value.

cudnnSetTensor()#

This function sets all the elements of a tensor to a given value.

cudnnStatus_t cudnnSetTensor(
    cudnnHandle_t                   handle,
    const cudnnTensorDescriptor_t   yDesc,
    void                           *y,
    const void                     *valuePtr)

Parameters

handle: Input. Handle to a previously created cuDNN context.
yDesc: Input. Handle to a previously initialized tensor descriptor.
y: Input/Output. Pointer to data of the tensor described by the yDesc descriptor.
valuePtr: Input. Pointer in host memory to a single value. All elements of the y tensor will be set to value[0]. The data type of the element in value[0] has to match the data type of tensor y.

Returns

CUDNN_STATUS_SUCCESS: The function launched successfully.
CUDNN_STATUS_NOT_SUPPORTED: The function does not support the provided configuration.
CUDNN_STATUS_BAD_PARAM: One of the provided pointers is NIL.
CUDNN_STATUS_EXECUTION_FAILED: The function failed to launch on the GPU.

cudnnSetTensor4dDescriptor()#

This function initializes a previously created generic tensor descriptor object into a 4D tensor. The strides of the four dimensions are inferred from the format parameter and set in such a way that the data is contiguous in memory with no padding between dimensions.

cudnnStatus_t cudnnSetTensor4dDescriptor(
    cudnnTensorDescriptor_t tensorDesc,
    cudnnTensorFormat_t     format,
    cudnnDataType_t         dataType,
    int                     n,
    int                     c,
    int                     h,
    int                     w)

The total size of a tensor including the potential padding between dimensions is limited to 2 Giga-elements of type datatype.

Parameters

tensorDesc: Input/Output. Handle to a previously created tensor descriptor.
format: Input. Type of format.
datatype: Input. Data type.
n: Input. Number of images.
c: Input. Number of feature maps per image.
h: Input. Height of each feature map.
w: Input. Width of each feature map.

Returns

CUDNN_STATUS_SUCCESS: The object was set successfully.
CUDNN_STATUS_BAD_PARAM: At least one of the parameters n, c, h, w was negative or format has an invalid enumerant value or dataType has an invalid enumerant value.
CUDNN_STATUS_NOT_SUPPORTED: The total size of the tensor descriptor exceeds the maximum limit of 2 Giga-elements.

cudnnSetTensor4dDescriptorEx()#

This function initializes a previously created generic tensor descriptor object into a 4D tensor, similarly to cudnnSetTensor4dDescriptor() but with the strides explicitly passed as parameters. This can be used to lay out the 4D tensor in any order or simply to define gaps between dimensions.

cudnnStatus_t cudnnSetTensor4dDescriptorEx(
    cudnnTensorDescriptor_t     tensorDesc,
    cudnnDataType_t             dataType,
    int                         n,
    int                         c,
    int                         h,
    int                         w,
    int                         nStride,
    int                         cStride,
    int                         hStride,
    int                         wStride)

At present, some cuDNN routines have limited support for strides. Those routines will return CUDNN_STATUS_NOT_SUPPORTED if a 4D tensor object with an unsupported stride is used. cudnnTransformTensor() can be used to convert the data to a supported layout.

The total size of a tensor including the potential padding between dimensions is limited to 2 Giga-elements of type datatype.

Parameters

tensorDesc: Input/Output. Handle to a previously created tensor descriptor.
datatype: Input. Data type.
n: Input. Number of images.
c: Input. Number of feature maps per image.
h: Input. Height of each feature map.
w: Input. Width of each feature map.
nStride: Input. Stride between two consecutive images.
cStride: Input. Stride between two consecutive feature maps.
hStride: Input. Stride between two consecutive rows.
wStride: Input. Stride between two consecutive columns.

Returns

CUDNN_STATUS_SUCCESS: The object was set successfully.
CUDNN_STATUS_BAD_PARAM: At least one of the parameters n, c, h, w or nStride, cStride, hStride, wStride is negative or dataType has an invalid enumerant value.
CUDNN_STATUS_NOT_SUPPORTED: The total size of the tensor descriptor exceeds the maximum limit of 2 Giga-elements.

cudnnSetTensorNdDescriptor()#

This function initializes a previously created generic tensor descriptor object.

cudnnStatus_t cudnnSetTensorNdDescriptor(
    cudnnTensorDescriptor_t tensorDesc,
    cudnnDataType_t         dataType,
    int                     nbDims,
    const int               dimA[],
    const int               strideA[])

The total size of a tensor including the potential padding between dimensions is limited to 2 Giga-elements of type datatype. Tensors are restricted to having at least 4 dimensions, and at most CUDNN_DIM_MAX dimensions (defined in cudnn.h). When working with lower dimensional data, it is recommended that the user create a 4D tensor, and set the size along unused dimensions to 1.

Parameters

tensorDesc

Input/Output. Handle to a previously created tensor descriptor.

datatype

Input. Data type.

nbDims

Input. Dimension of the tensor.

Do not use 2 dimensions. Due to historical reasons, the minimum number of dimensions in the filter descriptor is three.

dimA

Input. Array of dimension nbDims that contain the size of the tensor for every dimension. The size along unused dimensions should be set to 1. By convention, the ordering of dimensions in the array follows the format - [N, C, D, H, W], with W occupying the smallest index in the array.

strideA

Input. Array of dimension nbDims that contain the stride of the tensor for every dimension. By convention, the ordering of the strides in the array follows the format - [Nstride, Cstride, Dstride, Hstride, Wstride], with Wstride occupying the smallest index in the array.

Returns

CUDNN_STATUS_SUCCESS: The object was set successfully.
CUDNN_STATUS_BAD_PARAM: At least one of the elements of the array dimA was negative or zero, or dataType has an invalid enumerant value.
CUDNN_STATUS_NOT_SUPPORTED: The parameter nbDims is outside the range [4, CUDNN_DIM_MAX], or the total size of the tensor descriptor exceeds the maximum limit of 2 Giga-elements.

cudnnSetTensorNdDescriptorEx()#

This function initializes an Nd tensor descriptor.

cudnnStatus_t cudnnSetTensorNdDescriptorEx(
    cudnnTensorDescriptor_t tensorDesc,
    cudnnTensorFormat_t     format,
    cudnnDataType_t         dataType,
    int                     nbDims,
    const int               dimA[])

Parameters

tensorDesc

Output. Pointer to the tensor descriptor struct to be initialized.

format

Input. Tensor format.

dataType

Input. Tensor data type.

nbDims

Input. Dimension of the tensor.

Do not use 2 dimensions. Due to historical reasons, the minimum number of dimensions in the filter descriptor is three.

dimA

Input. Array containing the size of each dimension.

Returns

CUDNN_STATUS_SUCCESS: The function was successful.
CUDNN_STATUS_BAD_PARAM: Tensor descriptor was not allocated properly; or input parameters are not set correctly.
CUDNN_STATUS_NOT_SUPPORTED: Dimension size requested is larger than maximum dimension size supported.

cudnnSetTensorTransformDescriptor()#

This function has been deprecated in cuDNN 9.0.

This function initializes a tensor transform descriptor that was previously created using the cudnnCreateTensorTransformDescriptor() function.

cudnnStatus_t cudnnSetTensorTransformDescriptor(
    cudnnTensorTransformDescriptor_t transformDesc,
    const uint32_t nbDims,
    const cudnnTensorFormat_t destFormat,
    const int32_t padBeforeA[],
    const int32_t padAfterA[],
    const uint32_t foldA[],
    const cudnnFoldingDirection_t direction);

Parameters

transformDesc: Output. The tensor transform descriptor to be initialized.
nbDims: Input. The dimensionality of the transform operands. Must be greater than 2. For more information, refer to Tensor Descriptor.
destFormat: Input. The desired destination format.
padBeforeA[]: Input. An array that contains the amount of padding that should be added before each dimension. Set to NULL for no padding.
padAfterA[]: Input. An array that contains the amount of padding that should be added after each dimension. Set to NULL for no padding.
foldA[]: Input. An array that contains the folding parameters for each spatial dimension (dimensions 2 and up). Set to NULL for no folding.
direction: Input. Selects folding or unfolding. This input has no effect when folding parameters are all <= 1. For more information, refer to cudnnFoldingDirection_t.

Returns

CUDNN_STATUS_SUCCESS: The function was launched successfully.
CUDNN_STATUS_BAD_PARAM: The parameter transformDesc is NULL, or if direction is invalid, or nbDims is <= 2.
CUDNN_STATUS_NOT_SUPPORTED: If the dimension size requested is larger than maximum dimension size supported (meaning, one of the nbDims is larger than CUDNN_DIM_MAX), or if destFromat is something other than NCHW or NHWC.

cudnnSoftmaxBackward()#

This routine computes the gradient of the softmax function.

cudnnStatus_t cudnnSoftmaxBackward(
    cudnnHandle_t                    handle,
    cudnnSoftmaxAlgorithm_t          algorithm,
    cudnnSoftmaxMode_t               mode,
    const void                      *alpha,
    const cudnnTensorDescriptor_t    yDesc,
    const void                      *yData,
    const cudnnTensorDescriptor_t    dyDesc,
    const void                      *dy,
    const void                      *beta,
    const cudnnTensorDescriptor_t    dxDesc,
    void                            *dx)

In-place operation is allowed for this routine; meaning, dy and dx pointers may be equal. However, this requires dyDesc and dxDesc descriptors to be identical (particularly, the strides of the input and output must match for in-place operation to be allowed).

All tensor formats are supported for all modes and algorithms with 4 and 5D tensors. Performance is expected to be highest with NCHW fully-packed tensors. For more than 5 dimensions tensors must be packed in their spatial dimensions.

Parameters

handle

Input. Handle to a previously created cuDNN context.

algorithm

Input. Enumerant to specify the softmax algorithm.

mode

Input. Enumerant to specify the softmax mode.

alpha, beta

Inputs. Pointers to scaling factors (in host memory) used to blend the computation result with prior value in the output layer as follows:

dstValue = alpha[0]*result + beta[0]*priorDstValue

For more information, refer to Scaling Parameters.

yDesc

Input. Handle to the previously initialized input tensor descriptor.

y

Input. Data pointer to GPU memory associated with the tensor descriptor yDesc.

dyDesc

Input. Handle to the previously initialized input differential tensor descriptor.

dy

Input. Data pointer to GPU memory associated with the tensor descriptor dyData.

dxDesc

Input. Handle to the previously initialized output differential tensor descriptor.

dx

Output. Data pointer to GPU memory associated with the output tensor descriptor dxDesc.

Returns

CUDNN_STATUS_SUCCESS

The function launched successfully.

CUDNN_STATUS_NOT_SUPPORTED

The function does not support the provided configuration.

CUDNN_STATUS_BAD_PARAM

At least one of the following conditions are met:

The dimensions n, c, h, w of the yDesc, dyDesc and dxDesc tensors differ.

The strides nStride, cStride, hStride, wStride of the yDesc and dyDesc tensors differ.

The datatype of the three tensors differs.

CUDNN_STATUS_EXECUTION_FAILED

The function failed to launch on the GPU.

cudnnSoftmaxForward()#

This routine computes the softmax function.

cudnnStatus_t cudnnSoftmaxForward(
    cudnnHandle_t                    handle,
    cudnnSoftmaxAlgorithm_t          algorithm,
    cudnnSoftmaxMode_t               mode,
    const void                      *alpha,
    const cudnnTensorDescriptor_t    xDesc,
    const void                      *x,
    const void                      *beta,
    const cudnnTensorDescriptor_t    yDesc,
    void                            *y)

In-place operation is allowed for this routine; meaning, x and y pointers may be equal. However, this requires xDesc and yDesc descriptors to be identical (particularly, the strides of the input and output must match for in-place operation to be allowed).

All tensor formats are supported for all modes and algorithms with 4 and 5D tensors. Performance is expected to be highest with NCHW fully-packed tensors. For more than 5 dimensions tensors must be packed in their spatial dimensions.

Parameters

handle

Input. Handle to a previously created cuDNN context.

algorithm

Input. Enumerant to specify the softmax algorithm.

mode

Input. Enumerant to specify the softmax mode.

alpha, beta

Inputs. Pointers to scaling factors (in host memory) used to blend the computation result with prior value in the output layer as follows:

dstValue = alpha[0]*result + beta[0]*priorDstValue

For more information, refer to Scaling Parameters.

xDesc

Input. Handle to the previously initialized input tensor descriptor.

x

Input. Data pointer to GPU memory associated with the tensor descriptor xDesc.

yDesc

Input. Handle to the previously initialized output tensor descriptor.

y

Output. Data pointer to GPU memory associated with the output tensor descriptor yDesc.

Returns

CUDNN_STATUS_SUCCESS

The function launched successfully.

CUDNN_STATUS_NOT_SUPPORTED

The function does not support the provided configuration.

CUDNN_STATUS_BAD_PARAM

At least one of the following conditions are met:

The dimensions n, c, h, w of the input tensor and output tensors differ.

The datatype of the input tensor and output tensors differ.

The parameters algorithm or mode have an invalid enumerant value.

CUDNN_STATUS_EXECUTION_FAILED The function failed to launch on the GPU.

cudnnSpatialTfGridGeneratorBackward()#

This function computes the gradient of a grid generation operation.

cudnnStatus_t cudnnSpatialTfGridGeneratorBackward(
    cudnnHandle_t                               handle,
    const cudnnSpatialTransformerDescriptor_t   stDesc,
    const void                                 *dgrid,
    void                                       *dtheta)

Only 2D transformation is supported.

Parameters

handle: Input. Handle to a previously created cuDNN context.
stDesc: Input. Previously created spatial transformer descriptor object.
dgrid: Input. Data pointer to GPU memory contains the input differential data.
dtheta: Output. Data pointer to GPU memory contains the output differential data.

Returns

CUDNN_STATUS_SUCCESS

The call was successful.

CUDNN_STATUS_BAD_PARAM

At least one of the following conditions are met:

handle is NULL.

One of the parameters dgrid or dtheta is NULL.

CUDNN_STATUS_NOT_SUPPORTED

The function does not support the provided configuration. Some examples incldue:

The dimension of the transformed tensor specified in stDesc > 4.

CUDNN_STATUS_EXECUTION_FAILED

The function failed to launch on the GPU.

cudnnSpatialTfGridGeneratorForward()#

This function generates a grid of coordinates in the input tensor corresponding to each pixel from the output tensor.

cudnnStatus_t cudnnSpatialTfGridGeneratorForward(
    cudnnHandle_t                               handle,
    const cudnnSpatialTransformerDescriptor_t   stDesc,
    const void                                 *theta,
    void                                       *grid)

Only 2D transformation is supported.

Parameters

handle: Input. Handle to a previously created cuDNN context.
stDesc: Input. Previously created spatial transformer descriptor object.
theta: Input. Affine transformation matrix. It should be of size n*2*3 for a 2D transformation, where n is the number of images specified in stDesc.
grid: Output. A grid of coordinates. It is of size n*h*w*2 for a 2D transformation, where n, h, w is specified in stDesc. In the 4th dimension, the first coordinate is x, and the second coordinate is y.

Returns

CUDNN_STATUS_SUCCESS

The call was successful.

CUDNN_STATUS_BAD_PARAM

At least one of the following conditions are met:

handle is NULL.

One of the parameters grid or theta is NULL.

CUDNN_STATUS_NOT_SUPPORTED

The function does not support the provided configuration. Some examples include:

The dimension of the transformed tensor specified in stDesc > 4.

CUDNN_STATUS_EXECUTION_FAILED

The function failed to launch on the GPU.

cudnnSpatialTfSamplerBackward()#

This function computes the gradient of a sampling operation.

cudnnStatus_t cudnnSpatialTfSamplerBackward(
    cudnnHandle_t                              handle,
    const cudnnSpatialTransformerDescriptor_t  stDesc,
    const void                                *alpha,
    const cudnnTensorDescriptor_t              xDesc,
    const void                                *x,
    const void                                *beta,
    const cudnnTensorDescriptor_t              dxDesc,
    void                                      *dx,
    const void                                *alphaDgrid,
    const cudnnTensorDescriptor_t              dyDesc,
    const void                                *dy,
    const void                                *grid,
    const void                                *betaDgrid,
    void                                      *dgrid)

Only 2D transformation is supported.

Parameters

handle

Input. Handle to a previously created cuDNN context.

stDesc

Input. Previously created spatial transformer descriptor object.

alpha, beta

Inputs. Pointers to scaling factors (in host memory) used to blend the source value with prior value in the destination tensor as follows:

dstValue = alpha[0]*srcValue + beta[0]*priorDstValue

For more information, refer to Scaling Parameters.

xDesc

Input. Handle to the previously initialized input tensor descriptor.

x

Input. Data pointer to GPU memory associated with the tensor descriptor xDesc.

dxDesc

Input. Handle to the previously initialized output differential tensor descriptor.

dx

Output. Data pointer to GPU memory associated with the output tensor descriptor dxDesc.

alphaDgrid, betaDgrid

Inputs. Pointers to scaling factors (in host memory) used to blend the gradient outputs dgrid with prior value in the destination pointer as follows:

dstValue = alpha[0]*srcValue + beta[0]*priorDstValue

For more information, refer to Scaling Parameters.

dyDesc

Input. Handle to the previously initialized input differential tensor descriptor.

dy

Input. Data pointer to GPU memory associated with the tensor descriptor dyDesc.

grid

Input. A grid of coordinates generated by cudnnSpatialTfGridGeneratorForward().

dgrid

Output. Data pointer to GPU memory contains the output differential data.

Returns

CUDNN_STATUS_SUCCESS

The call was successful.

CUDNN_STATUS_BAD_PARAM

At least one of the following conditions are met:

handle is NULL.

One of the parameters x, dx, y, dy, grid, and dgrid is NULL.

The dimension of dy differs from those specified in stDesc.

CUDNN_STATUS_NOT_SUPPORTED

The function does not support the provided configuration. Some examples include:

The dimension of transformed tensor > 4.

CUDNN_STATUS_EXECUTION_FAILED

The function failed to launch on the GPU.

cudnnSpatialTfSamplerForward()#

This function performs a sampler operation and generates the output tensor using the grid given by the grid generator.

cudnnStatus_t cudnnSpatialTfSamplerForward(
    cudnnHandle_t                              handle,
    const cudnnSpatialTransformerDescriptor_t  stDesc,
    const void                                *alpha,
    const cudnnTensorDescriptor_t              xDesc,
    const void                                *x,
    const void                                *grid,
    const void                                *beta,
    cudnnTensorDescriptor_t                    yDesc,
    void                                      *y)

Only 2D transformation is supported.

Parameters

handle

Input. Handle to a previously created cuDNN context.

stDesc

Input. Previously created spatial transformer descriptor object.

alpha, beta

Inputs. Pointers to scaling factors (in host memory) used to blend the source value with prior value in the destination tensor as follows:

dstValue = alpha[0]*srcValue + beta[0]*priorDstValue

For more information, refer to Scaling Parameters.

xDesc

Input. Handle to the previously initialized input tensor descriptor.

x

Input. Data pointer to GPU memory associated with the tensor descriptor xDesc.

grid

Input. A grid of coordinates generated by cudnnSpatialTfGridGeneratorForward().

yDesc

Input. Handle to the previously initialized output tensor descriptor.

y

Output. Data pointer to GPU memory associated with the output tensor descriptor yDesc.

Returns

CUDNN_STATUS_SUCCESS

The call was successful.

CUDNN_STATUS_BAD_PARAM

At least one of the following conditions are met:

handle is NULL.

One of the parameters x, y, or grid is NULL.

The dimension of dy differs from those specified in stDesc.

CUDNN_STATUS_NOT_SUPPORTED

The function does not support the provided configuration. Some examples include:

The dimension of transformed tensor > 4.

CUDNN_STATUS_EXECUTION_FAILED

The function failed to launch on the GPU.

cudnnTransformFilter()#

This function has been deprecated in cuDNN 9.0.

This function converts the filter between different formats, data types, or dimensions based on the described transformation. It can be used to convert a filter with an unsupported layout format to a filter with a supported layout format.

cudnnStatus_t cudnnTransformFilter(
    cudnnHandle_t handle,
    const cudnnTensorTransformDescriptor_t transDesc,
    const void *alpha,
    const cudnnFilterDescriptor_t srcDesc,
    const void *srcData,
    const void *beta,
    const cudnnFilterDescriptor_t destDesc,
    void *destData);

This function copies the scaled data from the input filter srcDesc to the output tensor destDesc with a different layout. If the filter descriptors of srcDesc and destDesc have different dimensions, they must be consistent with folding and padding amount and order specified in transDesc.

The srcDesc and destDesc tensors must not overlap in any way (meaning, tensors cannot be transformed in place).

When performing a folding transform or a zero-padding transform, the scaling factors (alpha, beta) should be set to (1, 0). However, unfolding transforms support any (alpha, beta) values. This function is thread safe.

Parameters

handle

Input. Handle to a previously created cuDNN context. For more information, refer to cudnnHandle_t.

transDesc

Input. A descriptor containing the details of the requested filter transformation. For more information, refer to cudnnTensorTransformDescriptor_t.

alpha, beta

Inputs. Pointers, in the host memory, to the scaling factors used to scale the data in the input tensor srcDesc. beta is used to scale the destination tensor, while alpha is used to scale the source tensor. For more information, refer to Scaling Parameters.

The beta scaling value is not honored in the folding and zero-padding cases. Unfolding supports any (alpha, beta).

srcDesc, destDesc

Inputs. Handles to the previously initiated filter descriptors. srcDesc and destDesc must not overlap. For more information, refer to cudnnTensorDescriptor_t.

srcData, destData

Inputs. Pointers, in the host memory, to the data of the tensor described by srcDesc and destDesc respectively.

Returns

CUDNN_STATUS_SUCCESS: The function launched successfully.
CUDNN_STATUS_BAD_PARAM: A parameter is uninitialized or initialized incorrectly, or the number of dimensions is different between srcDesc and destDesc.
CUDNN_STATUS_NOT_SUPPORTED: The function does not support the provided configuration. Also, in the folding and padding paths, any value other than A=1 and B=0 will result in a CUDNN_STATUS_NOT_SUPPORTED.
CUDNN_STATUS_EXECUTION_FAILED: The function failed to launch on the GPU.

cudnnTransformTensor()#

This function has been deprecated in cuDNN 9.0.

This function copies the scaled data from one tensor to another tensor with a different layout. Those descriptors need to have the same dimensions but not necessarily the same strides. The input and output tensors must not overlap in any way (meaning, tensors cannot be transformed in place). This function can be used to convert a tensor with an unsupported format to a supported one.

cudnnStatus_t cudnnTransformTensor(
    cudnnHandle_t                  handle,
    const void                    *alpha,
    const cudnnTensorDescriptor_t  xDesc,
    const void                    *x,
    const void                    *beta,
    const cudnnTensorDescriptor_t  yDesc,
    void                          *y)

Parameters

handle

Input. Handle to a previously created cuDNN context.

alpha, beta

Inputs. Pointers to scaling factors (in host memory) used to blend the source value with prior value in the destination tensor as follows:

dstValue = alpha[0]*srcValue + beta[0]*priorDstValue

For more information, refer to Scaling Parameters.

xDesc

Input. Handle to a previously initialized tensor descriptor. For more information, refer to cudnnTensorDescriptor_t.

x

Input. Pointer to data of the tensor described by the xDesc descriptor.

yDesc

Input. Handle to a previously initialized tensor descriptor. For more information, refer to cudnnTensorDescriptor_t.

y

Output. Pointer to data of the tensor described by the yDesc descriptor.

Returns

CUDNN_STATUS_SUCCESS: The function launched successfully.
CUDNN_STATUS_NOT_SUPPORTED: The function does not support the provided configuration.
CUDNN_STATUS_BAD_PARAM: The dimensions n, c, h, w or the dataType of the two tensor descriptors are different.
CUDNN_STATUS_EXECUTION_FAILED: The function failed to launch on the GPU.

cudnnTransformTensorEx()#

This function has been deprecated in cuDNN 9.0.

This function converts the tensor layouts between different formats. It can be used to convert a tensor with an unsupported layout format to a tensor with a supported layout format.

cudnnStatus_t cudnnTransformTensorEx(
    cudnnHandle_t handle,
    const cudnnTensorTransformDescriptor_t transDesc,
    const void *alpha,
    const cudnnTensorDescriptor_t srcDesc,
    const void *srcData,
    const void *beta,
    const cudnnTensorDescriptor_t destDesc,
    void *destData);

This function copies the scaled data from the input tensor srcDesc to the output tensor destDesc with a different layout. The tensor descriptors of srcDesc and destDesc should have the same dimensions but need not have the same strides.

The srcDesc and destDesc tensors must not overlap in any way (meaning, tensors cannot be transformed in place).

When performing a folding transform or a zero-padding transform, the scaling factors (alpha, beta) should be set to (1, 0). However, unfolding transforms support any (alpha, beta) values. This function is thread safe.

Parameters

handle

Input. Handle to a previously created cuDNN context. For more information, refer to cudnnHandle_t.

transDesc

Input. A descriptor containing the details of the requested tensor transformation. For more information, refer to cudnnTensorTransformDescriptor_t.

alpha, beta

Inputs. Pointers, in the host memory, to the scaling factors used to scale the data in the input tensor srcDesc. beta is used to scale the destination tensor, while alpha is used to scale the source tensor. For more information, refer to Scaling Parameters.

The beta scaling value is not honored in the folding and zero-padding cases. Unfolding supports any (alpha, beta).

srcDesc, destDesc

Inputs. Handles to the previously initiated tensor descriptors. srcDesc and destDesc must not overlap. For more information, refer to cudnnTensorDescriptor_t.

srcData, destData

Input. Pointers, in the host memory, to the data of the tensor described by srcDesc and destDesc respectively.

Returns

CUDNN_STATUS_SUCCESS: The function was launched successfully.
CUDNN_STATUS_BAD_PARAM: A parameter is uninitialized or initialized incorrectly, or the number of dimensions is different between srcDesc and destDesc.
CUDNN_STATUS_NOT_SUPPORTED: Function does not support the provided configuration. Also, in the folding and padding paths, any value other than A=1 and B=0 will result in a CUDNN_STATUS_NOT_SUPPORTED.
CUDNN_STATUS_EXECUTION_FAILED: Function failed to launch on the GPU.