cudnn_ops Library
Contents
Data Type References
These are the data type references in the cudnn_ops
library.
Pointer To Opaque Struct Types
These are the pointers to the opaque struct types in the cudnn_ops
library.
cudnnActivationDescriptor_t
This enumerated type is deprecated and is currently only used by deprecated APIs. Consider using replacements for the deprecated APIs that use this enumerated type.
cudnnActivationDescriptor_t
is a pointer to an opaque structure holding the description of an activation operation. cudnnCreateActivationDescriptor() is used to create one instance, and cudnnSetActivationDescriptor() must be used to initialize this instance.
cudnnCTCLossDescriptor_t
cudnnCTCLossDescriptor_t
is a pointer to an opaque structure holding the description of a CTC loss operation. cudnnCreateCTCLossDescriptor() is used to create one instance, cudnnSetCTCLossDescriptor() is used to initialize this instance, and cudnnDestroyCTCLossDescriptor() is used to destroy this instance.
cudnnDropoutDescriptor_t
cudnnDropoutDescriptor_t
is a pointer to an opaque structure holding the description of a dropout operation. cudnnCreateDropoutDescriptor() is used to create one instance, cudnnSetDropoutDescriptor() is used to initialize this instance, cudnnDestroyDropoutDescriptor() is used to destroy this instance, cudnnGetDropoutDescriptor() is used to query fields of a previously initialized instance, cudnnRestoreDropoutDescriptor() is used to restore an instance to a previously saved off state.
cudnnFilterDescriptor_t
This enumerated type is deprecated and is currently only used by deprecated APIs. Consider using replacements for the deprecated APIs that use this enumerated type.
cudnnFilterDescriptor_t
is a pointer to an opaque structure holding the description of a filter dataset. cudnnCreateFilterDescriptor() is used to create one instance, and cudnnSetFilter4dDescriptor() or cudnnSetFilterNdDescriptor() must be used to initialize this instance.
cudnnLRNDescriptor_t
cudnnLRNDescriptor_t
is a pointer to an opaque structure holding the parameters of a local response normalization. cudnnCreateLRNDescriptor() is used to create one instance, and the routine cudnnSetLRNDescriptor() must be used to initialize this instance.
cudnnOpTensorDescriptor_t
This enumerated type is deprecated and is currently only used by deprecated APIs. Consider using replacements for the deprecated APIs that use this enumerated type.
cudnnOpTensorDescriptor_t
is a pointer to an opaque structure holding the description of a Tensor Core operation, used as a parameter to cudnnOpTensor(). cudnnCreateOpTensorDescriptor() is used to create one instance, and cudnnSetOpTensorDescriptor() must be used to initialize this instance.
cudnnPoolingDescriptor_t
This enumerated type is deprecated and is currently only used by deprecated APIs. Consider using replacements for the deprecated APIs that use this enumerated type.
cudnnPoolingDescriptor_t
is a pointer to an opaque structure holding the description of a pooling operation. cudnnCreatePoolingDescriptor() is used to create one instance, and cudnnSetPoolingNdDescriptor() or cudnnSetPooling2dDescriptor() must be used to initialize this instance.
cudnnReduceTensorDescriptor_t
This enumerated type is deprecated and is currently only used by deprecated APIs. Consider using replacements for the deprecated APIs that use this enumerated type.
cudnnReduceTensorDescriptor_t
is a pointer to an opaque structure holding the description of a tensor reduction operation, used as a parameter to cudnnReduceTensor(). cudnnCreateReduceTensorDescriptor() is used to create one instance, and cudnnSetReduceTensorDescriptor() must be used to initialize this instance.
cudnnSpatialTransformerDescriptor_t
cudnnSpatialTransformerDescriptor_t
is a pointer to an opaque structure holding the description of a spatial transformation operation. cudnnCreateSpatialTransformerDescriptor() is used to create one instance, cudnnSetSpatialTransformerNdDescriptor() is used to initialize this instance, and cudnnDestroySpatialTransformerDescriptor() is used to destroy this instance.
cudnnTensorDescriptor_t
cudnnTensorDescriptor_t
is a pointer to an opaque structure holding the description of a generic nD dataset. cudnnCreateTensorDescriptor() is used to create one instance, and one of the routines cudnnSetTensorNdDescriptor(), cudnnSetTensor4dDescriptor(), or cudnnSetTensor4dDescriptorEx() must be used to initialize this instance.
cudnnTensorTransformDescriptor_t
This enumerated type is deprecated and is currently only used by deprecated APIs. Consider using replacements for the deprecated APIs that use this enumerated type.
cudnnTensorTransformDescriptor_t
is an opaque structure containing the description of the tensor transform. Use the cudnnCreateTensorTransformDescriptor() function to create an instance of this descriptor, and cudnnDestroyTensorTransformDescriptor() function to destroy a previously created instance.
Enumeration Types
These are the enumeration types in the cudnn_ops
library.
cudnnBatchNormMode_t
This enumerated type is deprecated and is currently only used by deprecated APIs. Consider using replacements for the deprecated APIs that use this enumerated type.
cudnnBatchNormMode_t
is an enumerated type used to specify the mode of operation in cudnnBatchNormalizationForwardInference(), cudnnBatchNormalizationForwardTraining(), cudnnBatchNormalizationBackward(), and cudnnDeriveBNTensorDescriptor() routines.
Values
CUDNN_BATCHNORM_PER_ACTIVATION
Normalization is performed peractivation. This mode is intended to be used after the nonconvolutional network layers. In this mode, the tensor dimensions of
bnBias
andbnScale
and the parameters used in thecudnnBatchNormalization*
functions are 1xCxHxW.CUDNN_BATCHNORM_SPATIAL
Normalization is performed over N+spatial dimensions. This mode is intended for use after convolutional layers (where spatial invariance is desired). In this mode, the
bnBias
andbnScale
tensor dimensions are 1xCx1x1.CUDNN_BATCHNORM_SPATIAL_PERSISTENT
This mode is similar to
CUDNN_BATCHNORM_SPATIAL
but it can be faster for some tasks.An optimized path may be selected for
CUDNN_DATA_FLOAT
andCUDNN_DATA_HALF
types, compute capability 6.0 or higher for the following two batch normalization API calls: cudnnBatchNormalizationForwardTraining(), and cudnnBatchNormalizationBackward(). In the case of cudnnBatchNormalizationBackward(), thesavedMean
andsavedInvVariance
arguments should not beNULL
.
NCHW Mode Only This mode may use a scaled atomic integer reduction that is deterministic but imposes more restrictions on the input data range. When a numerical overflow occurs, the algorithm may produce NaNs or Infs (infinity) in output buffers.
When Infs/NaNs are present in the input data, the output in this mode is the same as from a pure floatingpoint implementation.
For finite but very large input values, the algorithm may encounter overflows more frequently due to a lower dynamic range and emit Infs/NaNs while CUDNN_BATCHNORM_SPATIAL
will produce finite results. The user can invoke cudnnQueryRuntimeError() to check if a numerical overflow occurred in this mode.
cudnnBatchNormOps_t
This enumerated type is deprecated and is currently only used by deprecated APIs. Consider using replacements for the deprecated APIs that use this enumerated type.
cudnnBatchNormOps_t
is an enumerated type used to specify the mode of operation in cudnnGetBatchNormalizationForwardTrainingExWorkspaceSize(), cudnnBatchNormalizationForwardTrainingEx(), cudnnGetBatchNormalizationBackwardExWorkspaceSize(), cudnnBatchNormalizationBackwardEx(), and cudnnGetBatchNormalizationTrainingExReserveSpaceSize() functions.
Values
CUDNN_BATCHNORM_OPS_BN
Only batch normalization is performed, peractivation.
CUDNN_BATCHNORM_OPS_BN_ACTIVATION
First, the batch normalization is performed, and then the activation is performed.
CUDNN_BATCHNORM_OPS_BN_ADD_ACTIVATION
Performs the batch normalization, then elementwise addition, followed by the activation operation.
cudnnConvolutionBwdDataAlgo_t
cudnnConvolutionBwdDataAlgo_t
is an enumerated type that exposes the different algorithms available to execute the backward data convolution operation.
Values
CUDNN_CONVOLUTION_BWD_DATA_ALGO_0
This algorithm expresses the convolution as a sum of matrix products without actually explicitly forming the matrix that holds the input tensor data. The sum is done using the atomic add operation, thus the results are nondeterministic.
CUDNN_CONVOLUTION_BWD_DATA_ALGO_1
This algorithm expresses the convolution as a matrix product without actually explicitly forming the matrix that holds the input tensor data. The results are deterministic.
CUDNN_CONVOLUTION_BWD_DATA_ALGO_FFT
This algorithm uses a FastFourier Transform approach to compute the convolution. A significant memory workspace is needed to store intermediate results. The results are deterministic.
CUDNN_CONVOLUTION_BWD_DATA_ALGO_FFT_TILING
This algorithm uses the FastFourier Transform approach but splits the inputs into tiles. A significant memory workspace is needed to store intermediate results but less than
CUDNN_CONVOLUTION_BWD_DATA_ALGO_FFT
for large size images. The results are deterministic.CUDNN_CONVOLUTION_BWD_DATA_ALGO_WINOGRAD
This algorithm uses the Winograd Transform approach to compute the convolution. A reasonably sized workspace is needed to store intermediate results. The results are deterministic.
CUDNN_CONVOLUTION_BWD_DATA_ALGO_WINOGRAD_NONFUSED
This algorithm uses the Winograd Transform approach to compute the convolution. A significant workspace may be needed to store intermediate results. The results are deterministic.
cudnnConvolutionBwdFilterAlgo_t
cudnnConvolutionBwdFilterAlgo_t
is an enumerated type that exposes the different algorithms available to execute the backward filter convolution operation.
Values
CUDNN_CONVOLUTION_BWD_FILTER_ALGO_0
This algorithm expresses the convolution as a sum of matrix products without actually explicitly forming the matrix that holds the input tensor data. The sum is done using the atomic add operation, thus the results are nondeterministic.
CUDNN_CONVOLUTION_BWD_FILTER_ALGO_1
This algorithm expresses the convolution as a matrix product without actually explicitly forming the matrix that holds the input tensor data. The results are deterministic.
CUDNN_CONVOLUTION_BWD_FILTER_ALGO_FFT
This algorithm uses the FastFourier Transform approach to compute the convolution. A significant workspace is needed to store intermediate results. The results are deterministic.
CUDNN_CONVOLUTION_BWD_FILTER_ALGO_3
This algorithm is similar to CUDNN_CONVOLUTION_BWD_FILTER_ALGO_0 but uses some small workspace to precompute some indices. The results are also nondeterministic.
CUDNN_CONVOLUTION_BWD_FILTER_WINOGRAD_NONFUSED
This algorithm uses the Winograd Transform approach to compute the convolution. A significant workspace may be needed to store intermediate results. The results are deterministic.
CUDNN_CONVOLUTION_BWD_FILTER_ALGO_FFT_TILING
This algorithm uses the FastFourier Transform approach to compute the convolution but splits the input tensor into tiles. A significant workspace may be needed to store intermediate results. The results are deterministic.
cudnnConvolutionFwdAlgo_t
cudnnConvolutionFwdAlgo_t
is an enumerated type that exposes the different algorithms available to execute the forward convolution operation.
Values
CUDNN_CONVOLUTION_FWD_ALGO_IMPLICIT_GEMM
This algorithm expresses the convolution as a matrix product without actually explicitly forming the matrix that holds the input tensor data.
CUDNN_CONVOLUTION_FWD_ALGO_IMPLICIT_PPRECOMP_GEMM
This algorithm expresses the convolution as a matrix product without actually explicitly forming the matrix that holds the input tensor data, but still needs some memory workspace to precompute some indices in order to facilitate the implicit construction of the matrix that holds the input tensor data.
CUDNN_CONVOLUTION_FWD_ALGO_GEMM
This algorithm expresses convolution as an explicit matrix product. A significant memory workspace is needed to store the matrix that holds the input tensor data.
CUDNN_CONVOLUTION_FWD_ALGO_DIRECT
This algorithm expresses the convolution as a direct convolution (for example, without implicitly or explicitly doing a matrix multiplication).
CUDNN_CONVOLUTION_FWD_ALGO_FFT
This algorithm uses the FastFourier Transform approach to compute the convolution. A significant memory workspace is needed to store intermediate results.
CUDNN_CONVOLUTION_FWD_ALGO_FFT_TILING
This algorithm uses the FastFourier Transform approach but splits the inputs into tiles. A significant memory workspace is needed to store intermediate results but less than
CUDNN_CONVOLUTION_FWD_ALGO_FFT
for large size images.CUDNN_CONVOLUTION_FWD_ALGO_WINOGRAD
This algorithm uses the Winograd Transform approach to compute the convolution. A reasonably sized workspace is needed to store intermediate results.
CUDNN_CONVOLUTION_FWD_ALGO_WINOGRAD_NONFUSED
This algorithm uses the Winograd Transform approach to compute the convolution. A significant workspace may be needed to store intermediate results.
cudnnCTCLossAlgo_t
cudnnCTCLossAlgo_t
is an enumerated type that exposes the different algorithms available to execute the CTC loss operation.
Values
CUDNN_CTC_LOSS_ALGO_DETERMINISTIC
Results are guaranteed to be reproducible.
CUDNN_CTC_LOSS_ALGO_NON_DETERMINISTIC
Results are not guaranteed to be reproducible.
cudnnDeterminism_t
cudnnDeterminism_t
is an enumerated type used to indicate if the computed results are deterministic (reproducible). For more information, refer to Reproducibility (Determinism).
Values
CUDNN_NON_DETERMINISTIC
Results are not guaranteed to be reproducible.
CUDNN_DETERMINISTIC
Results are guaranteed to be reproducible.
cudnnDivNormMode_t
cudnnDivNormMode_t
is an enumerated type used to specify the mode of operation in cudnnDivisiveNormalizationForward() and cudnnDivisiveNormalizationBackward().
Values
CUDNN_DIVNORM_PRECOMPUTED_MEANS
The means tensor data pointer is expected to contain means or other kernel convolution values precomputed by the user. The means pointer can also be
NULL
, in that case, it’s considered to be filled with zeroes. This is equivalent to spatial LRN.In the backward pass, the means are treated as independent inputs and the gradient over means is computed independently. In this mode, to yield a net gradient over the entire LCN computational graph, the
destDiffMeans
result should be backpropagated through the user’s means layer (which can be implemented using average pooling) and added to thedestDiffData
tensor produced by cudnnDivisiveNormalizationBackward().
cudnnFoldingDirection_t
cudnnFoldingDirection_t
is an enumerated type used to select the folding direction. For more information, refer to cudnnTensorTransformDescriptor_t.
Data Member
CUDNN_TRANSFORM_FOLD = 0U
Selects folding.
CUDNN_TRANSFORM_UNFOLD = 1U
Selects unfolding.
cudnnIndicesType_t
This enumerated type is deprecated and is currently only used by deprecated APIs. Consider using replacements for the deprecated APIs that use this enumerated type.
cudnnIndicesType_t
is an enumerated type used to indicate the data type for the indices to be computed by the cudnnReduceTensor() routine. This enumerated type is used as a field for the cudnnReduceTensorDescriptor_t descriptor.
Values
CUDNN_32BIT_INDICES
Compute unsigned int indices.
CUDNN_64BIT_INDICES
Compute unsigned long indices.
CUDNN_16BIT_INDICES
Compute unsigned short indices.
CUDNN_8BIT_INDICES
Compute unsigned char indices.
cudnnLRNMode_t
cudnnLRNMode_t
is an enumerated type used to specify the mode of operation in cudnnLRNCrossChannelForward() and cudnnLRNCrossChannelBackward().
Values
CUDNN_LRN_CROSS_CHANNEL_DIM1
LRN computation is performed across the tensor’s dimension
dimA[1]
.
cudnnNormAlgo_t
This enumerated type is deprecated and is currently only used by deprecated APIs. Consider using replacements for the deprecated APIs that use this enumerated type.
cudnnNormAlgo_t
is an enumerated type used to specify the algorithm to execute the normalization operation.
Values
CUDNN_NORM_ALGO_STANDARD
Standard normalization is performed.
CUDNN_NORM_ALGO_PERSIST
This mode is similar to
CUDNN_NORM_ALGO_STANDARD
, however it only supportsCUDNN_NORM_PER_CHANNEL
and can be faster for some tasks.An optimized path may be selected for
CUDNN_DATA_FLOAT
andCUDNN_DATA_HALF
types, compute capability 6.0 or higher for the following two normalization API calls: cudnnNormalizationForwardTraining() and cudnnNormalizationBackward(). In the case of cudnnNormalizationBackward(), thesavedMean
andsavedInvVariance
arguments should not beNULL
.
NCHW Mode Only This mode may use a scaled atomic integer reduction that is deterministic but imposes more restrictions on the input data range. When a numerical overflow occurs, the algorithm may produce NaNs or Infs (infinity) in output buffers.
When Infs/NaNs are present in the input data, the output in this mode is the same as from a pure floatingpoint implementation.
For finite but very large input values, the algorithm may encounter overflows more frequently due to a lower dynamic range and emit Infs/NaNs while CUDNN_NORM_ALGO_STANDARD
will produce finite results. The user can invoke cudnnQueryRuntimeError() to check if a numerical overflow occurred in this mode.
cudnnNormMode_t
This enumerated type is deprecated and is currently only used by deprecated APIs. Consider using replacements for the deprecated APIs that use this enumerated type.
cudnnNormMode_t
is an enumerated type used to specify the mode of operation in cudnnNormalizationForwardInference(), cudnnNormalizationForwardTraining(), cudnnBatchNormalizationBackward(), cudnnGetNormalizationForwardTrainingWorkspaceSize(), cudnnGetNormalizationBackwardWorkspaceSize(), and cudnnGetNormalizationTrainingReserveSpaceSize() routines.
Values
CUDNN_NORM_PER_ACTIVATION
Normalization is performed peractivation. This mode is intended to be used after the nonconvolutional network layers. In this mode, the tensor dimensions of
normBias
andnormScale
and the parameters used in thecudnnNormalization*
functions are 1xCxHxW.CUDNN_NORM_PER_CHANNEL
Normalization is performed perchannel over N+spatial dimensions. This mode is intended for use after convolutional layers (where spatial invariance is desired). In this mode, the
normBias
andnormScale
tensor dimensions are 1xCx1x1.
cudnnNormOps_t
This enumerated type is deprecated and is currently only used by deprecated APIs. Consider using replacements for the deprecated APIs that use this enumerated type.
cudnnNormOps_t
is an enumerated type used to specify the mode of operation in cudnnGetNormalizationForwardTrainingWorkspaceSize(), cudnnNormalizationForwardTraining(), cudnnGetNormalizationBackwardWorkspaceSize(), cudnnNormalizationBackward(), and cudnnGetNormalizationTrainingReserveSpaceSize() functions.
Values
CUDNN_NORM_OPS_NORM
Only normalization is performed.
CUDNN_NORM_OPS_NORM_ACTIVATION
First, the normalization is performed, then the activation is performed.
CUDNN_NORM_OPS_NORM_ADD_ACTIVATION
Performs the normalization, then elementwise addition, followed by the activation operation.
cudnnOpTensorOp_t
cudnnOpTensorOp_t
is an enumerated type used to indicate the Tensor Core operation to be used by the cudnnOpTensor() routine. This enumerated type is used as a field for the cudnnOpTensorDescriptor_t descriptor.
Values
CUDNN_OP_TENSOR_ADD
The operation to be performed is addition.
CUDNN_OP_TENSOR_MUL
The operation to be performed is multiplication.
CUDNN_OP_TENSOR_MIN
The operation to be performed is a minimum comparison.
CUDNN_OP_TENSOR_MAX
The operation to be performed is a maximum comparison.
CUDNN_OP_TENSOR_SQRT
The operation to be performed is square root, performed on only the
A
tensor.CUDNN_OP_TENSOR_NOT
The operation to be performed is negation, performed on only the
A
tensor.
cudnnPoolingMode_t
This enumerated type is deprecated and is currently only used by deprecated APIs. Consider using replacements for the deprecated APIs that use this enumerated type.
cudnnPoolingMode_t
is an enumerated type passed to cudnnSetPooling2dDescriptor() to select the pooling method to be used by cudnnPoolingForward() and cudnnPoolingBackward().
Values
CUDNN_POOLING_MAX
The maximum value inside the pooling window is used.
CUDNN_POOLING_AVERAGE_COUNT_INCLUDE_PADDING
Values inside the pooling window are averaged. The number of elements used to calculate the average includes spatial locations falling in the padding region.
CUDNN_POOLING_AVERAGE_COUNT_EXCLUDE_PADDING
Values inside the pooling window are averaged. The number of elements used to calculate the average excludes spatial locations falling in the padding region.
CUDNN_POOLING_MAX_DETERMINISTIC
The maximum value inside the pooling window is used. The algorithm used is deterministic.
cudnnReduceTensorIndices_t
This enumerated type is deprecated and is currently only used by deprecated APIs. Consider using replacements for the deprecated APIs that use this enumerated type.
cudnnReduceTensorIndices_t
is an enumerated type used to indicate whether indices are to be computed by the cudnnReduceTensor() routine. This enumerated type is used as a field for the cudnnReduceTensorDescriptor_t descriptor.
Values
CUDNN_REDUCE_TENSOR_NO_INDICES
Do not compute indices.
CUDNN_REDUCE_TENSOR_FLATTENED_INDICES
Compute indices. The resulting indices are relative, and flattened.
cudnnSamplerType_t
cudnnSamplerType_t
is an enumerated type passed to cudnnSetSpatialTransformerNdDescriptor() to select the sampler type to be used by cudnnSpatialTfSamplerForward() and cudnnSpatialTfSamplerBackward().
Values
CUDNN_SAMPLER_BILINEAR
Selects the bilinear sampler.
cudnnSoftmaxAlgorithm_t
cudnnSoftmaxAlgorithm_t
is used to select an implementation of the softmax function used in cudnnSoftmaxForward() and cudnnSoftmaxBackward().
Values
CUDNN_SOFTMAX_FAST
This implementation applies the straightforward softmax operation.
CUDNN_SOFTMAX_ACCURATE
This implementation scales each point of the softmax input domain by its maximum value to avoid potential floating point overflows in the softmax evaluation.
CUDNN_SOFTMAX_LOG
This entry performs the log softmax operation, avoiding overflows by scaling each point in the input domain as in
CUDNN_SOFTMAX_ACCURATE
.
cudnnSoftmaxMode_t
cudnnSoftmaxMode_t
is used to select over which data the cudnnSoftmaxForward() and cudnnSoftmaxBackward() are computing their results.
Values
CUDNN_SOFTMAX_MODE_INSTANCE
The softmax operation is computed per image (N
) across the dimensions C,H,W
.
CUDNN_SOFTMAX_MODE_CHANNEL
The softmax operation is computed per spatial location (H,W
) per image (N
) across dimension C
.
API Functions
These are the API functions in the cudnn_ops
library.
cudnnActivationBackward()
This function has been deprecated in cuDNN 9.0.
This routine computes the gradient of a neuron activation function.
cudnnStatus_t cudnnActivationBackward( cudnnHandle_t handle, cudnnActivationDescriptor_t activationDesc, const void *alpha, const cudnnTensorDescriptor_t yDesc, const void *y, const cudnnTensorDescriptor_t dyDesc, const void *dy, const cudnnTensorDescriptor_t xDesc, const void *x, const void *beta, const cudnnTensorDescriptor_t dxDesc, void *dx)
Inplace operation is allowed for this routine; meaning dy
and dx
pointers may be equal. However, this requires the corresponding tensor descriptors to be identical (particularly, the strides of the input and output must match for an inplace operation to be allowed).
All tensor formats are supported for 4 and 5 dimensions, however, the best performance is obtained when the strides of yDesc
and xDesc
are equal and HWpacked. For more than 5 dimensions the tensors must have their spatial dimensions packed.
Parameters
handle
Input. Handle to a previously created cuDNN context. For more information, refer to cudnnHandle_t.
activationDesc
Input. Activation descriptor. For more information, refer to cudnnActivationDescriptor_t.
alpha
,beta
Input. Pointers to scaling factors (in host memory) used to blend the computation result with prior value in the output layer as follows:
dstValue = alpha[0]*result + beta[0]*priorDstValue
For more information, refer to Scaling Parameters.
yDesc
Input. Handle to the previously initialized input tensor descriptor. For more information, refer to cudnnTensorDescriptor_t.
y
Input. Data pointer to GPU memory associated with the tensor descriptor
yDesc
.dyDesc
Input. Handle to the previously initialized input differential tensor descriptor.
dy
Input. Data pointer to GPU memory associated with the tensor descriptor
dyDesc
.xDesc
Input. Handle to the previously initialized output tensor descriptor.
x
Input. Data pointer to GPU memory associated with the output tensor descriptor
xDesc
.dxDesc
Input. Handle to the previously initialized output differential tensor descriptor.
dx
Output. Data pointer to GPU memory associated with the output tensor descriptor
dxDesc
.
Returns
CUDNN_STATUS_SUCCESS
The function launched successfully.
CUDNN_STATUS_BAD_PARAM
At least one of the following conditions are met:
The strides
nStride
,cStride
,hStride
,wStride
of the input differential tensor and output differential tensor differ and inplace operation is used.
CUDNN_STATUS_NOT_SUPPORTED
The function does not support the provided configuration. Refer to the following for some examples of nonsupported configurations:
The dimensions
n
,c
,h
, andw
of the input tensor and output tensor differ.The
datatype
of the input tensor and output tensor differs.The strides
nStride
,cStride
,hStride
, andwStride
of the input tensor and the input differential tensor differ.The strides
nStride
,cStride
,hStride
, andwStride
of the output tensor and the output differential tensor differ.
CUDNN_STATUS_EXECUTION_FAILED
The function failed to launch on the GPU.
cudnnActivationForward()
This function has been deprecated in cuDNN 9.0.
This routine applies a specified neuron activation function elementwise over each input value.
cudnnStatus_t cudnnActivationForward( cudnnHandle_t handle, cudnnActivationDescriptor_t activationDesc, const void *alpha, const cudnnTensorDescriptor_t xDesc, const void *x, const void *beta, const cudnnTensorDescriptor_t yDesc, void *y)
Inplace operation is allowed for this routine; meaning, xData
and yData
pointers may be equal. However, this requires xDesc
and yDesc
descriptors to be identical (particularly, the strides of the input and output must match for an inplace operation to be allowed).
All tensor formats are supported for 4 and 5 dimensions, however, the best performance is obtained when the strides of xDesc
and yDesc
are equal and HWpacked. For more than 5 dimensions the tensors must have their spatial dimensions packed.
Parameters
handle
Input. Handle to a previously created cuDNN context. For more information, refer to cudnnHandle_t.
activationDesc
Input. Activation descriptor. For more information, refer to cudnnActivationDescriptor_t.
alpha
,beta
Input. Pointers to scaling factors (in host memory) used to blend the computation result with prior value in the output layer as follows:
dstValue = alpha[0]*result + beta[0]*priorDstValue
For more information, refer to Scaling Parameters.
xDesc
Input. Handle to the previously initialized input tensor descriptor. For more information, refer to cudnnTensorDescriptor_t.
x
Input. Data pointer to GPU memory associated with the tensor descriptor
xDesc
.yDesc
Input. Handle to the previously initialized input tensor descriptor.
y
Input. Data pointer to GPU memory associated with the tensor descriptor
yDesc
.
Returns
CUDNN_STATUS_SUCCESS
The function launched successfully.
CUDNN_STATUS_NOT_SUPPORTED
The function does not support the provided configuration.
CUDNN_STATUS_BAD_PARAM
At least one of the following conditions are met:
The parameter
mode
has an invalid enumerant value.The dimensions
n
,c
,h
,w
of the input tensor and output tensor differ.The
datatype
of the input tensor and output tensor differs.The strides
nStride
,cStride
,hStride
,wStride
of the input tensor and output tensor differ and inplace operation is used (meaning,x
andy
pointers are equal).
CUDNN_STATUS_EXECUTION_FAILED
The function failed to launch on the GPU.
cudnnAddTensor()
This function has been deprecated in cuDNN 9.0.
This function adds the scaled values of a bias tensor to another tensor. Each dimension of the bias tensor A
must match the corresponding dimension of the destination tensor C
or must be equal to 1
. In the latter case, the same value from the bias tensor for those dimensions will be used to blend into the C
tensor.
cudnnStatus_t cudnnAddTensor( cudnnHandle_t handle, const void *alpha, const cudnnTensorDescriptor_t aDesc, const void *A, const void *beta, const cudnnTensorDescriptor_t cDesc, void *C)
Only 4D and 5D tensors are supported. Beyond these dimensions, this routine is not supported.
Parameters
handle
Input. Handle to a previously created cuDNN context. For more information, refer to cudnnHandle_t.
alpha
,beta
Input. Pointers to scaling factors (in host memory) used to blend the source value with the prior value in the destination tensor as follows:
dstValue = alpha[0]*srcValue + beta[0]*priorDstValue
For more information, refer to Scaling Parameters.
aDesc
Input. Handle to a previously initialized tensor descriptor. For more information, refer to cudnnTensorDescriptor_t.
A
Input. Pointer to data of the tensor described by the
aDesc
descriptor.cDesc
Input. Handle to a previously initialized tensor descriptor.
C
Input/Output. Pointer to data of the tensor described by the
cDesc
descriptor.
Returns
CUDNN_STATUS_SUCCESS
The function executed successfully.
CUDNN_STATUS_NOT_SUPPORTED
The function does not support the provided configuration.
CUDNN_STATUS_BAD_PARAM
The dimensions of the bias tensor refer to an amount of data that is incompatible with the output tensor dimensions or the
dataType
of the two tensor descriptors are different.CUDNN_STATUS_EXECUTION_FAILED
The function failed to launch on the GPU.
cudnnBatchNormalizationBackward()
This function has been deprecated in cuDNN 9.0.
This function performs the backward batch normalization layer computation. This layer is based on the Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift paper.
cudnnStatus_t cudnnBatchNormalizationBackward( cudnnHandle_t handle, cudnnBatchNormMode_t mode, const void *alphaDataDiff, const void *betaDataDiff, const void *alphaParamDiff, const void *betaParamDiff, const cudnnTensorDescriptor_t xDesc, const void *x, const cudnnTensorDescriptor_t dyDesc, const void *dy, const cudnnTensorDescriptor_t dxDesc, void *dx, const cudnnTensorDescriptor_t bnScaleBiasDiffDesc, const void *bnScale, void *resultBnScaleDiff, void *resultBnBiasDiff, double epsilon, const void *savedMean, const void *savedInvVariance)
Only 4D and 5D tensors are supported.
The epsilon
value has to be the same during training, backpropagation, and inference.
Higher performance can be obtained when HWpacked tensors are used for all of x
, dy
, and dx
.
For more information, refer to cudnnDeriveBNTensorDescriptor() for the secondary tensor descriptor generation for the parameters used in this function.
Parameters
handle
Input. Handle to a previously created cuDNN library descriptor. For more information, refer to cudnnHandle_t.
mode
Input. Mode of operation (spatial or peractivation). For more information, refer to cudnnBatchNormMode_t.
*alphaDataDiff
,*betaDataDiff
Inputs. Pointers to scaling factors (in host memory) used to blend the gradient output
dx
with a prior value in the destination tensor as follows:dstValue = alphaDataDiff[0]*resultValue + betaDataDiff[0]*priorDstValue
For more information, refer to Scaling Parameters.
*alphaParamDiff
,*betaParamDiff
Inputs. Pointers to scaling factors (in host memory) used to blend the gradient outputs
resultBnScaleDiff
andresultBnBiasDiff
with prior values in the destination tensor as follows:dstValue = alphaParamDiff[0]*resultValue + betaParamDiff[0]*priorDstValue
For more information, refer to Scaling Parameters.
xDesc
,dxDesc
,dyDesc
Inputs. Handles to the previously initialized tensor descriptors.
*x
Inputs. Data pointer to GPU memory associated with the tensor descriptor
xDesc
, for the layer’sx
data.*dy
Inputs. Data pointer to GPU memory associated with the tensor descriptor
dyDesc
, for the backpropagated differentialdy
input.*dx
Inputs/Outputs. Data pointer to GPU memory associated with the tensor descriptor
dxDesc
, for the resulting differential output with respect tox
.bnScaleBiasDiffDesc
Input. Shared tensor descriptor for the following five tensors:
bnScale
,resultBnScaleDiff
,resultBnBiasDiff
,savedMean
, andsavedInvVariance
. The dimensions for this tensor descriptor are dependent on normalization mode. For more information, refer to cudnnDeriveBNTensorDescriptor().Note
The data type of this tensor descriptor must be
float
for FP16 and FP32 input tensors, anddouble
for FP64 input tensors.*bnScale
Input. Pointer in the device memory for the batch normalization
scale
parameter (in the original paper the quantityscale
is referred to as gamma).Note
The
bnBias
parameter is not needed for this layer’s computation.resultBnScaleDiff
,resultBnBiasDiff
Outputs. Pointers in device memory for the resulting scale and bias differentials computed by this routine. Note that these scale and bias gradients are weight gradients specific to this batch normalization operation, and by definition are not backpropagated.
epsilon
Input. Epsilon value used in batch normalization formula. Its value should be equal to or greater than the value defined for
CUDNN_BN_MIN_EPSILON
incudnn.h
. The sameepsilon
value should be used in forward and backward functions.*savedMean
,*savedInvVariance
Inputs. Optional cache parameters containing saved intermediate results that were computed during the forward pass. For this to work correctly, the layer’s
x
andbnScale
data have to remain unchanged until this backward function is called.Note
Both these parameters can be
NULL
but only at the same time. It is recommended to use this cache since the memory overhead is relatively small.
Supported Configurations
This function supports the following combinations of data types for various descriptors.
Data Type Configurations Supported 

























Returns
CUDNN_STATUS_SUCCESS
The computation was performed successfully.
CUDNN_STATUS_NOT_SUPPORTED
The function does not support the provided configuration.
CUDNN_STATUS_BAD_PARAM
At least one of the following conditions are met:
Any of the pointers
alpha
,beta
,x
,dy
,dx
,bnScale
,resultBnScaleDiff
, andresultBnBiasDiff
isNULL
.The number of
xDesc
,yDesc
, ordxDesc
tensor descriptor dimensions is not within the range of [4,5] (only 4D and 5D tensors are supported).bnScaleBiasDiffDesc
dimensions are not 1xCx1x1 for 4D and 1xCx1x1x1 for 5D for spatial, and are not 1xCxHxW for 4D and 1xCxDxHxW for 5D for peractivation mode.Exactly one of
savedMean
,savedInvVariance
pointers isNULL
.epsilon
value is less thanCUDNN_BN_MIN_EPSILON
.Dimensions or data types mismatch for any pair of
xDesc
,dyDesc
, ordxDesc
.
cudnnBatchNormalizationBackwardEx()
This function has been deprecated in cuDNN 9.0.
This function is an extension of the cudnnBatchNormalizationBackward() for performing the backward batch normalization layer computation with a fast NHWC semipersistent kernel.
cudnnStatus_t cudnnBatchNormalizationBackwardEx ( cudnnHandle_t handle, cudnnBatchNormMode_t mode, cudnnBatchNormOps_t bnOps, const void *alphaDataDiff, const void *betaDataDiff, const void *alphaParamDiff, const void *betaParamDiff, const cudnnTensorDescriptor_t xDesc, const void *xData, const cudnnTensorDescriptor_t yDesc, const void *yData, const cudnnTensorDescriptor_t dyDesc, const void *dyData, const cudnnTensorDescriptor_t dzDesc, void *dzData, const cudnnTensorDescriptor_t dxDesc, void *dxData, const cudnnTensorDescriptor_t dBnScaleBiasDesc, const void *bnScaleData, const void *bnBiasData, void *dBnScaleData, void *dBnBiasData, double epsilon, const void *savedMean, const void *savedInvVariance, const cudnnActivationDescriptor_t activationDesc, void *workspace, size_t workSpaceSizeInBytes void *reserveSpace size_t reserveSpaceSizeInBytes);
This API will trigger the new semipersistent NHWC kernel when the following conditions are true:
All tensors, namely,
x
,y
,dz
,dy
, anddx
must be NHWCfully packed, and must be of the typeCUDNN_DATA_HALF
.The input parameter mode must be set to
CUDNN_BATCHNORM_SPATIAL_PERSISTENT
.Before cuDNN version 8.2.0, the tensor
C
dimension should always be a multiple of 4. After 8.2.0, the tensorC
dimension should be a multiple of 4 only whenbnOps
isCUDNN_BATCHNORM_OPS_BN_ADD_ACTIVATION
.
workspace
is notNULL
.
workSpaceSizeInBytes
is equal to or larger than the amount required by cudnnGetBatchNormalizationBackwardExWorkspaceSize().
reserveSpaceSizeInBytes
is equal to or larger than the amount required by cudnnGetBatchNormalizationTrainingExReserveSpaceSize().The content in
reserveSpace
stored by cudnnBatchNormalizationForwardTrainingEx() must be preserved.
If workspace
is NULL
and workSpaceSizeInBytes
of zero is passed in, this API will function exactly like the nonextended function cudnnBatchNormalizationBackward()
.
This workspace
is not required to be clean. Moreover, the workspace
does not have to remain unchanged between the forward and backward pass, as it is not used for passing any information.
This extended function can accept a *workspace
pointer to the GPU workspace, and workSpaceSizeInBytes
, the size of the workspace, from the user.
The bnOps
input can be used to set this function to perform either only the batch normalization, or batch normalization followed by activation, or batch normalization followed by elementwise addition and then activation.
Only 4D and 5D tensors are supported. The epsilon
value has to be the same during the training, the backpropagation, and the inference.
When the tensor layout is NCHW, higher performance can be obtained when HWpacked tensors are used for x
, dy
, and dx
.
Parameters
handle
Input. Handle to a previously created cuDNN library descriptor. For more information, refer to cudnnHandle_t.
mode
Input. Mode of operation (spatial or peractivation). For more information, refer to cudnnBatchNormMode_t.
bnOps
Input. Mode of operation. Currently,
CUDNN_BATCHNORM_OPS_BN_ACTIVATION
andCUDNN_BATCHNORM_OPS_BN_ADD_ACTIVATION
are only supported in the NHWC layout. For more information, refer to cudnnBatchNormOps_t. This input can be used to set this function to perform either only the batch normalization, or batch normalization followed by activation, or batch normalization followed by elementwise addition and then activation.*alphaDataDiff
,*betaDataDiff
Inputs. Pointers to scaling factors (in host memory) used to blend the gradient output
dx
with a prior value in the destination tensor as follows:dstValue = alpha[0]*resultValue + beta[0]*priorDstValue
For more information, refer to Scaling Parameters.
*alphaParamDiff
,*betaParamDiff
Inputs. Pointers to scaling factors (in host memory) used to blend the gradient outputs
dBnScaleData
anddBnBiasData
with prior values in the destination tensor as follows:dstValue = alpha[0]*resultValue + beta[0]*priorDstValue
For more information, refer to Scaling Parameters.
xDesc
,*x
,yDesc
,*yData
,dyDesc
,*dyData
Inputs. Tensor descriptors and pointers in the device memory for the layer’s
x
data, backpropagated gradient inputdy
, the original forward outputy
data.yDesc
andyData
are not needed ifbnOps
is set toCUDNN_BATCHNORM_OPS_BN
, users may passNULL
. For more information, refer to cudnnTensorDescriptor_t.dzDesc
,dxDesc
Inputs. Tensor descriptors and pointers in the device memory for the computed gradient output
dz
, anddx
.dzDesc
is not needed whenbnOps
isCUDNN_BATCHNORM_OPS_BN
orCUDNN_BATCHNORM_OPS_BN_ACTIVATION
, users may passNULL
. For more information, refer to cudnnTensorDescriptor_t.*dzData
,*dxData
Outputs. Tensor descriptors and pointers in the device memory for the computed gradient output
dz
, anddx
.*dzData
is not needed whenbnOps
isCUDNN_BATCHNORM_OPS_BN
orCUDNN_BATCHNORM_OPS_BN_ACTIVATION
, users may passNULL
. For more information, refer to cudnnTensorDescriptor_t.dBnScaleBiasDesc
Input. Shared tensor descriptor for the following six tensors:
bnScaleData
,bnBiasData
,dBnScaleData
,dBnBiasData
,savedMean
, andsavedInvVariance
. For more information, refer to cudnnDeriveBNTensorDescriptor().The dimensions for this tensor descriptor are dependent on normalization mode.
Note
The data type of this tensor descriptor must be
float
for FP16 and FP32 input tensors anddouble
for FP64 input tensors. For more information, refer to cudnnTensorDescriptor_t.*bnScaleData
Input. Pointer in the device memory for the batch normalization scale parameter (in the Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift paper, the quantity scale is referred to as gamma).
*bnBiasData
Input. Pointers in the device memory for the batch normalization bias parameter (in the Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift paper, bias is referred to as beta). This parameter is used only when activation should be performed.
*dBnScaleData
,*dBnBiasData
Outputs. Pointers in the device memory for the gradients of
bnScaleData
andbnBiasData
, respectively.epsilon
Input. Epsilon value used in batch normalization formula. Its value should be equal to or greater than the value defined for
CUDNN_BN_MIN_EPSILON
incudnn.h
. The same epsilon value should be used in forward and backward functions.*savedMean
,*savedInvVariance
Inputs. Optional cache parameters containing saved intermediate results computed during the forward pass. For this to work correctly, the layer’s
x
andbnScaleData
,bnBiasData
data has to remain unchanged until this backward function is called. Note that both these parameters can be NULL but only at the same time. It is recommended to use this cache since the memory overhead is relatively small.activationDesc
Input. Descriptor for the activation operation. When the
bnOps
input is set to eitherCUDNN_BATCHNORM_OPS_BN_ACTIVATION
orCUDNN_BATCHNORM_OPS_BN_ADD_ACTIVATION
then this activation is used, otherwise the user may passNULL
.workspace
Input. Pointer to the GPU workspace. If workspace is
NULL
andworkSpaceSizeInBytes
of zero is passed in, then this API will function exactly like the nonextended function cudnnBatchNormalizationBackward().workSpaceSizeInBytes
Input. The size of the workspace. It must be large enough to trigger the fast NHWC semipersistent kernel by this function.
*reserveSpace
Input. Pointer to the GPU workspace for the
reserveSpace
.reserveSpaceSizeInBytes
Input. The size of the
reserveSpace
. It must be equal or larger than the amount required by cudnnGetBatchNormalizationTrainingExReserveSpaceSize().
Supported Configurations
This function supports the following combinations of data types for various descriptors.
Data Type Configurations Supported 

























Returns
CUDNN_STATUS_SUCCESS
The computation was performed successfully.
CUDNN_STATUS_NOT_SUPPORTED
The function does not support the provided configuration.
CUDNN_STATUS_BAD_PARAM
At least one of the following conditions are met:
Any of the pointers
alphaDataDiff
,betaDataDiff
,alphaParamDiff
,betaParamDiff
,x
,dy
,dx
,bnScale
,resultBnScaleDiff
, andresultBnBiasDiff
isNULL
.The number of
xDesc
,yDesc
, ordxDesc
tensor descriptor dimensions is not within the range of [4,5] (only 4D and 5D tensors are supported).dBnScaleBiasDesc
dimensions not 1xCx1x1 for 4D and 1xCx1x1x1 for 5D for spatial, and are not 1xCxHxW for 4D and 1xCxDxHxW for 5D for peractivation mode.Exactly one of
savedMean
,savedInvVariance
pointers isNULL
.epsilon
value is less thanCUDNN_BN_MIN_EPSILON
.Dimensions or data types mismatch for any pair of
xDesc
,dyDesc
, anddxDesc
.
cudnnBatchNormalizationForwardInference()
This function has been deprecated in cuDNN 9.0.
This function performs the forward batch normalization layer computation for the inference phase. This layer is based on the Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift paper.
cudnnStatus_t cudnnBatchNormalizationForwardInference( cudnnHandle_t handle, cudnnBatchNormMode_t mode, const void *alpha, const void *beta, const cudnnTensorDescriptor_t xDesc, const void *x, const cudnnTensorDescriptor_t yDesc, void *y, const cudnnTensorDescriptor_t bnScaleBiasMeanVarDesc, const void *bnScale, const void *bnBias, const void *estimatedMean, const void *estimatedVariance, double epsilon)
Only 4D and 5D tensors are supported.
The input transformation performed by this function is defined as:
y = beta*y + alpha *[bnBias + (bnScale * (xestimatedMean)/sqrt(epsilon + estimatedVariance)]
The epsilon
value has to be the same during training, backpropagation and inference.
For the training phase, refer to cudnnBatchNormalizationForwardTraining().
Higher performance can be obtained when HWpacked tensors are used for all of x
and dx
.
For more information, refer to cudnnDeriveBNTensorDescriptor() for the secondary tensor descriptor generation for the parameters used in this function.
Parameters
handle
Input. Handle to a previously created cuDNN library descriptor. For more information, refer to cudnnHandle_t.
mode
Input. Mode of operation (spatial or peractivation). For more information, refer to cudnnBatchNormMode_t.
alpha
,beta
Inputs. Pointers to scaling factors (in host memory) used to blend the layer output value with prior value in the destination tensor as follows:
dstValue = alpha[0]*resultValue + beta[0]*priorDstValue
For more information, refer to Scaling Parameters.
xDesc
,yDesc
Input. Handles to the previously initialized tensor descriptors.
*x
Input. Data pointer to GPU memory associated with the tensor descriptor
xDesc
, for the layer’sx
input data.*y
Input/Output. Data pointer to GPU memory associated with the tensor descriptor
yDesc
, for they
output of the batch normalization layer.bnScaleBiasMeanVarDesc
,bnScale
,bnBias
Inputs. Tensor descriptors and pointers in device memory for the batch normalization scale and bias parameters (in the Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift paper, bias is referred to as beta and scale as gamma).
estimatedMean
,estimatedVariance
Inputs. Mean and variance tensors (these have the same descriptor as the bias and scale). The
resultRunningMean
andresultRunningVariance
, accumulated during the training phase from the cudnnBatchNormalizationForwardTraining() call, should be passed as inputs here.epsilon
Input. Epsilon value used in batch normalization formula. Its value should be equal to or greater than the value defined for
CUDNN_BN_MIN_EPSILON
incudnn.h
. The same epsilon value should be used in forward and backward functions.
Supported Configurations
This function supports the following combinations of data types for various descriptors.
Data Type Configurations Supported 






























Returns
CUDNN_STATUS_SUCCESS
The computation was performed successfully.
CUDNN_STATUS_NOT_SUPPORTED
The function does not support the provided configuration.
CUDNN_STATUS_BAD_PARAM
At least one of the following conditions are met:
One of the pointers
alpha
,beta
,x
,y
,bnScale
,bnBias
,estimatedMean
, andestimatedInvVariance
isNULL
.The number of
xDesc
oryDesc
tensor descriptor dimensions is not within the range of [4,5] (only 4D and 5D tensors are supported.)bnScaleBiasMeanVarDesc
dimensions are not 1xCx1x1 for 4D and 1xCx1x1x1 for 5D for spatial, and are not 1xCxHxW for 4D and 1xCxDxHxW for 5D for peractivation mode.epsilon
value is less thanCUDNN_BN_MIN_EPSILON
.Dimensions or data types mismatch for
xDesc
,yDesc
.
cudnnBatchNormalizationForwardTraining()
This function has been deprecated in cuDNN 9.0.
This function performs the forward batch normalization layer computation for the training phase. This layer is based on the Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift paper.
cudnnStatus_t cudnnBatchNormalizationForwardTraining( cudnnHandle_t handle, cudnnBatchNormMode_t mode, const void *alpha, const void *beta, const cudnnTensorDescriptor_t xDesc, const void *x, const cudnnTensorDescriptor_t yDesc, void *y, const cudnnTensorDescriptor_t bnScaleBiasMeanVarDesc, const void *bnScale, const void *bnBias, double exponentialAverageFactor, void *resultRunningMean, void *resultRunningVariance, double epsilon, void *resultSaveMean, void *resultSaveInvVariance)
Only 4D and 5D tensors are supported.
The epsilon
value has to be the same during training, backpropagation, and inference.
For the inference phase, use cudnnBatchNormalizationForwardInference()
.
Higher performance can be obtained when HWpacked tensors are used for both x
and y
.
Refer to cudnnDeriveBNTensorDescriptor() for the secondary tensor descriptor generation for the parameters used in this function.
Parameters
handle
Input. Handle to a previously created cuDNN library descriptor. For more information, refer to cudnnHandle_t.
mode
Input. Mode of operation (spatial or peractivation). For more information, refer to cudnnBatchNormMode_t.
alpha
,beta
Inputs. Pointers to scaling factors (in host memory) used to blend the layer output value with prior value in the destination tensor as follows:
dstValue = alpha[0]*resultValue + beta[0]*priorDstValue
For more information, refer to Scaling Parameters.
xDesc
,yDesc
Input. Tensor descriptors and pointers in device memory for the layer’s
x
andy
data. For more information, refer to cudnnTensorDescriptor_t.*x
Input. Data pointer to GPU memory associated with the tensor descriptor
xDesc
, for the layer’sx
input data.*y
Input. Data pointer to GPU memory associated with the tensor descriptor
yDesc
, for they
output of the batch normalization layer.bnScaleBiasMeanVarDesc
Input. Shared tensor descriptor desc for the secondary tensor that was derived by cudnnDeriveBNTensorDescriptor(). The dimensions for this tensor descriptor are dependent on the normalization mode.
bnScale
,bnBias
Inputs. Pointers in device memory for the batch normalization scale and bias parameters (in the Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift paper, bias is referred to as beta and scale as gamma). Note that
bnBias
parameter can replace the previous layer’s bias parameter for improved efficiency.exponentialAverageFactor
Input. Factor used in the moving average computation as follows:
runningMean = runningMean*(1factor) + newMean*factor
Use a
factor=1/(1+n)
atNth
call to the function to get the Cumulative Moving Average (CMA) behavior, for example:CMA[n] = (x[1]+...+x[n])/n
For example:
CMA[n+1] = (n*CMA[n]+x[n+1])/(n+1) = ((n+1)*CMA[n]CMA[n])/(n+1) + x[n+1]/(n+1) = CMA[n]*(11/(n+1))+x[n+1]*1/(n+1) = CMA[n]*(1factor) + x(n+1)*factor
resultRunningMean
,resultRunningVariance
Inputs/Outputs. Running mean and variance tensors (these have the same descriptor as the bias and scale). Both of these pointers can be
NULL
but only at the same time. The value stored inresultRunningVariance
(or passed as an input in inference mode) is the sample variance and is the moving average ofvariance[x]
where the variance is computed either over batch or spatial+batch dimensions depending on the mode. If these pointers are notNULL
, the tensors should be initialized to some reasonable values or to0
.epsilon
Input. Epsilon value used in the batch normalization formula. Its value should be equal to or greater than the value defined for
CUDNN_BN_MIN_EPSILON
incudnn.h
. The same epsilon value should be used in forward and backward functions.resultSaveMean
,resultSaveInvVariance
Outputs. Optional cache to save intermediate results computed during the forward pass. These buffers can be used to speed up the backward pass when supplied to the cudnnBatchNormalizationBackward() function. The intermediate results stored in
resultSaveMean
andresultSaveInvVariance
buffers should not be used directly by the user. Depending on the batch normalization mode, the results stored inresultSaveInvVariance
may vary. For the cache to work correctly, the input layer data must remain unchanged until the backward function is called. Note that both parameters can beNULL
but only at the same time. In such a case, intermediate statistics will not be saved, and cudnnBatchNormalizationBackward() will have to recompute them. It is recommended to use this cache as the memory overhead is relatively small because these tensors have a much lower product of dimensions than the data tensors.
Supported Configurations
This function supports the following combinations of data types for various descriptors.
Data Type Configurations Supported 

























Returns
CUDNN_STATUS_SUCCESS
The computation was performed successfully.
CUDNN_STATUS_NOT_SUPPORTED
The function does not support the provided configuration.
CUDNN_STATUS_BAD_PARAM
At least one of the following conditions are met:
One of the pointers
alpha
,beta
,x
,y
,bnScale
, andbnBias
isNULL
.The number of
xDesc
oryDesc
tensor descriptor dimensions is not within the range of [4,5] (only 4D and 5D tensors are supported.)bnScaleBiasMeanVarDesc
dimensions are not 1xCx1x1 for 4D and 1xCx1x1x1 for 5D for spatial, and are not 1xCxHxW for 4D and 1xCxDxHxW for 5D for peractivation mode.Exactly one of
resultSaveMean
,resultSaveInvVariance
pointers areNULL
.Exactly one of
resultRunningMean
,resultRunningInvVariance
pointers areNULL
.epsilon
value is less thanCUDNN_BN_MIN_EPSILON
.Dimensions or data types mismatch for
xDesc
,yDesc
.
cudnnBatchNormalizationForwardTrainingEx()
This function has been deprecated in cuDNN 9.0.
This function is an extension of the cudnnBatchNormalizationForwardTraining() for performing the forward batch normalization layer computation.
cudnnStatus_t cudnnBatchNormalizationForwardTrainingEx( cudnnHandle_t handle, cudnnBatchNormMode_t mode, cudnnBatchNormOps_t bnOps, const void *alpha, const void *beta, const cudnnTensorDescriptor_t xDesc, const void *xData, const cudnnTensorDescriptor_t zDesc, const void *zData, const cudnnTensorDescriptor_t yDesc, void *yData, const cudnnTensorDescriptor_t bnScaleBiasMeanVarDesc, const void *bnScaleData, const void *bnBiasData, double exponentialAverageFactor, void *resultRunningMeanData, void *resultRunningVarianceData, double epsilon, void *saveMean, void *saveInvVariance, const cudnnActivationDescriptor_t activationDesc, void *workspace, size_t workSpaceSizeInBytes void *reserveSpace size_t reserveSpaceSizeInBytes);
This API will trigger the new semipersistent NHWC kernel when the following conditions are true:
All tensors, namely,
x
,y
,dz
,dy
, anddx
must be NHWCfully packed and must be of the typeCUDNN_DATA_HALF
.The input parameter mode must be set to
CUDNN_BATCHNORM_SPATIAL_PERSISTENT
.
workspace
is notNULL
.Before cuDNN version 8.2.0, the tensor
C
dimension should always be a multiple of 4. After 8.2.0, the tensorC
dimension should be a multiple of 4 only whenbnOps
isCUDNN_BATCHNORM_OPS_BN_ADD_ACTIVATION
.
workSpaceSizeInBytes
is equal to or larger than the amount required by cudnnGetBatchNormalizationForwardTrainingExWorkspaceSize().
reserveSpaceSizeInBytes
is equal to or larger than the amount required by cudnnGetBatchNormalizationTrainingExReserveSpaceSize().The content in
reserveSpace
stored by cudnnBatchNormalizationForwardTrainingEx() must be preserved.
If workspace
is NULL
and workSpaceSizeInBytes
of zero is passed in, this API will function exactly like the nonextended function cudnnBatchNormalizationForwardTraining().
This workspace is not required to be clean. Moreover, the workspace does not have to remain unchanged between the forward and backward pass, as it is not used for passing any information.
This extended function can accept a *workspace
pointer to the GPU workspace, and workSpaceSizeInBytes
, the size of the workspace, from the user.
The bnOps
input can be used to set this function to perform either only the batch normalization, or batch normalization followed by activation, or batch normalization followed by elementwise addition and then activation.
Only 4D and 5D tensors are supported. The epsilon
value has to be the same during the training, the backpropagation, and the inference.
When the tensor layout is NCHW, higher performance can be obtained when HWpacked tensors are used for x
, dy
, and dx
.
Parameters
handle
Input. Handle to a previously created cuDNN library descriptor. For more information, refer to cudnnHandle_t.
mode
Input. Mode of operation (spatial or peractivation). For more information, refer to cudnnBatchNormMode_t.
bnOps
Input. Mode of operation for the fast NHWC kernel. For more information, refer to cudnnBatchNormOps_t. This input can be used to set this function to perform either only the batch normalization, or batch normalization followed by activation, or batch normalization followed by elementwise addition and then activation.
*alpha
,*beta
Inputs. Pointers to scaling factors (in host memory) used to blend the layer output value with prior value in the destination tensor as follows:
dstValue = alpha[0]*resultValue + beta[0]*priorDstValue
For more information, refer to Scaling Parameters.
xDesc
,*xData
,zDesc
,*zData
,yDesc
,*yData
Inputs. Tensor descriptors and pointers in device memory for the layer’s input
x
and outputy
, and for the optionalz
tensor input for residual addition to the result of the batch normalization operation, prior to the activation. The optionalzDesc
and*zData
descriptors are only used whenbnOps
isCUDNN_BATCHNORM_OPS_BN_ADD_ACTIVATION
, otherwise users may passNULL
. When in use,z
should have exactly the same dimension asx
and the final outputy
. For more information, refer to cudnnTensorDescriptor_t.bnScaleBiasMeanVarDesc
Input. Shared tensor descriptor
desc
for the secondary tensor that was derived by cudnnDeriveBNTensorDescriptor(). The dimensions for this tensor descriptor are dependent on the normalization mode.*bnScaleData
,*bnBiasData
Inputs. Pointers in the device memory for the batch normalization scale and bias data. In the Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift paper, bias is referred to as beta and scale as gamma. Note that
bnBiasData
parameter can replace the previous operations bias parameter for improved efficiency.exponentialAverageFactor
Input. Factor used in the moving average computation as follows:
runningMean = runningMean*(1factor) + newMean*factor
Use a
factor=1/(1+n)
atNth
call to the function to get the Cumulative Moving Average (CMA) behavior, for example:CMA[n] = (x[1]+...+x[n])/n
For example:
CMA[n+1] = (n*CMA[n]+x[n+1])/(n+1) = ((n+1)*CMA[n]CMA[n])/(n+1) + x[n+1]/(n+1) = CMA[n]*(11/(n+1))+x[n+1]*1/(n+1) = CMA[n]*(1factor) + x(n+1)*factor
*resultRunningMeanData
,*resultRunningVarianceData
Inputs/Outputs. Pointers to the running mean and running variance data. Both these pointers can be
NULL
but only at the same time. The value stored inresultRunningVarianceData
(or passed as an input in inference mode) is the sample variance and is the moving average ofvariance[x]
where the variance is computed either over batch or spatial+batch dimensions depending on the mode. If these pointers are notNULL
, the tensors should be initialized to some reasonable values or to0
.epsilon
Input. Epsilon value used in the batch normalization formula. Its value should be equal to or greater than the value defined for
CUDNN_BN_MIN_EPSILON
incudnn.h
. The same epsilon value should be used in forward and backward functions.*saveMean
,*saveInvVariance
Outputs. Optional cache parameters containing saved intermediate results computed during the forward pass. For this to work correctly, the layer’s
x
andbnScaleData
,bnBiasData
data has to remain unchanged until this backward function is called. Note that both these parameters can beNULL
but only at the same time. It is recommended to use this cache since the memory overhead is relatively small.activationDesc
Input. The tensor descriptor for the activation operation. When the
bnOps
input is set to eitherCUDNN_BATCHNORM_OPS_BN_ACTIVATION
orCUDNN_BATCHNORM_OPS_BN_ADD_ACTIVATION
then this activation is used, otherwise user may passNULL
.*workspace
,workSpaceSizeInBytes
Inputs.
*workspace
is a pointer to the GPU workspace, andworkSpaceSizeInBytes
is the size of the workspace. When*workspace
is notNULL
and*workSpaceSizeInBytes
is large enough, and the tensor layout is NHWC and the data type configuration is supported, then this function will trigger a new semipersistent NHWC kernel for batch normalization. The workspace is not required to be clean. Also, the workspace does not need to remain unchanged between the forward and backward passes.*reserveSpace
Input. Pointer to the GPU workspace for the
reserveSpace
.reserveSpaceSizeInBytes
Input. The size of the
reserveSpace
. Must be equal or larger than the amount required by cudnnGetBatchNormalizationTrainingExReserveSpaceSize().
Supported Configurations
This function supports the following combinations of data types for various descriptors.
Data Type Configurations Supported 

























Returns
CUDNN_STATUS_SUCCESS
The computation was performed successfully.
CUDNN_STATUS_NOT_SUPPORTED
The function does not support the provided configuration.
CUDNN_STATUS_BAD_PARAM
At least one of the following conditions are met:
One of the pointers
alpha
,beta
,x
,y
,bnScaleData
, andbnBiasData
isNULL
.The number of
xDesc
oryDesc
tensor descriptor dimensions is not within the [4,5] range (only 4D and 5D tensors are supported.)bnScaleBiasMeanVarDesc
dimensions are not 1xCx1x1 for 4D and 1xCx1x1x1 for 5D for spatial, and are not 1xCxHxW for 4D and 1xCxDxHxW for 5D for peractivation mode.Exactly one of
saveMean
,saveInvVariance
pointers areNULL
.Exactly one of
resultRunningMeanData
,resultRunningInvVarianceData
pointers areNULL
.epsilon
value is less thanCUDNN_BN_MIN_EPSILON
.Dimensions or data types mismatch for
xDesc
andyDesc
.
cudnnCreateActivationDescriptor()
This function creates an activation descriptor object by allocating the memory needed to hold its opaque structure. For more information, refer to cudnnActivationDescriptor_t.
cudnnStatus_t cudnnCreateActivationDescriptor( cudnnActivationDescriptor_t *activationDesc)
Returns
CUDNN_STATUS_SUCCESS
The object was created successfully.
CUDNN_STATUS_ALLOC_FAILED
The resources could not be allocated.
cudnnCreateDropoutDescriptor()
This function creates a generic dropout descriptor object by allocating the memory needed to hold its opaque structure. For more information, refer to cudnnDropoutDescriptor_t.
cudnnStatus_t cudnnCreateDropoutDescriptor( cudnnDropoutDescriptor_t *dropoutDesc)
Returns
CUDNN_STATUS_SUCCESS
The object was created successfully.
CUDNN_STATUS_ALLOC_FAILED
The resources could not be allocated.
cudnnCreateFilterDescriptor()
This function has been deprecated in cuDNN 9.0.
This function creates a filter descriptor object by allocating the memory needed to hold its opaque structure. For more information, refer to cudnnFilterDescriptor_t.
cudnnStatus_t cudnnCreateFilterDescriptor( cudnnFilterDescriptor_t *filterDesc)
Returns
CUDNN_STATUS_SUCCESS
The object was created successfully.
CUDNN_STATUS_ALLOC_FAILED
The resources could not be allocated.
cudnnCreateLRNDescriptor()
This function allocates the memory needed to hold the data needed for LRN and DivisiveNormalization
layers operation and returns a descriptor used with subsequent layer forward and backward calls.
cudnnStatus_t cudnnCreateLRNDescriptor( cudnnLRNDescriptor_t *poolingDesc)
Returns
CUDNN_STATUS_SUCCESS
The object was created successfully.
CUDNN_STATUS_ALLOC_FAILED
The resources could not be allocated.
cudnnCreateOpTensorDescriptor()
This function creates a tensor pointwise math descriptor. For more information, refer to cudnnOpTensorDescriptor_t.
cudnnStatus_t cudnnCreateOpTensorDescriptor( cudnnOpTensorDescriptor_t* opTensorDesc)
Parameters
opTensorDesc
Output. Pointer to the structure holding the description of the tensor pointwise math such as add, multiply, and more.
Returns
CUDNN_STATUS_SUCCESS
The function returned successfully.
CUDNN_STATUS_BAD_PARAM
Tensor pointwise math descriptor passed to the function is invalid.
CUDNN_STATUS_ALLOC_FAILED
Memory allocation for this tensor pointwise math descriptor failed.
cudnnCreatePoolingDescriptor()
This function creates a pooling descriptor object by allocating the memory needed to hold its opaque structure.
cudnnStatus_t cudnnCreatePoolingDescriptor( cudnnPoolingDescriptor_t *poolingDesc)
Returns
CUDNN_STATUS_SUCCESS
The object was created successfully.
CUDNN_STATUS_ALLOC_FAILED
The resources could not be allocated.
cudnnCreateReduceTensorDescriptor()
This function creates a reduced tensor descriptor object by allocating the memory needed to hold its opaque structure.
cudnnStatus_t cudnnCreateReduceTensorDescriptor( cudnnReduceTensorDescriptor_t* reduceTensorDesc)
Returns
CUDNN_STATUS_SUCCESS
The object was created successfully.
CUDNN_STATUS_BAD_PARAM
reduceTensorDesc
is aNULL
pointer.CUDNN_STATUS_ALLOC_FAILED
The resources could not be allocated.
cudnnCreateSpatialTransformerDescriptor()
This function creates a generic spatial transformer descriptor object by allocating the memory needed to hold its opaque structure.
cudnnStatus_t cudnnCreateSpatialTransformerDescriptor( cudnnSpatialTransformerDescriptor_t *stDesc)
Returns
CUDNN_STATUS_SUCCESS
The object was created successfully.
CUDNN_STATUS_ALLOC_FAILED
The resources could not be allocated.
cudnnCreateTensorDescriptor()
This function creates a generic tensor descriptor object by allocating the memory needed to hold its opaque structure. The data is initialized to all zeros.
cudnnStatus_t cudnnCreateTensorDescriptor( cudnnTensorDescriptor_t *tensorDesc)
Parameters
tensorDesc
Input. Pointer to pointer where the address to the allocated tensor descriptor object should be stored.
Returns
CUDNN_STATUS_BAD_PARAM
Invalid input argument.
CUDNN_STATUS_ALLOC_FAILED
The resources could not be allocated.
CUDNN_STATUS_SUCCESS
The object was created successfully.
cudnnCreateTensorTransformDescriptor()
This function has been deprecated in cuDNN 9.0.
This function creates a tensor transform descriptor object by allocating the memory needed to hold its opaque structure. The tensor data is initialized to all zeros. Use the cudnnSetTensorTransformDescriptor() function to initialize the descriptor created by this function.
cudnnStatus_t cudnnCreateTensorTransformDescriptor( cudnnTensorTransformDescriptor_t *transformDesc);
Parameters
transformDesc
Output. A pointer to an uninitialized tensor transform descriptor.
Returns
CUDNN_STATUS_SUCCESS
The descriptor object was created successfully.
CUDNN_STATUS_BAD_PARAM
The
transformDesc
isNULL
.CUDNN_STATUS_ALLOC_FAILED
The memory allocation failed.
cudnnDeriveBNTensorDescriptor()
This function derives a secondary tensor descriptor for the batch normalization scale
, invVariance
, bnBias
, and bnScale
subtensors from the layer’s x
data descriptor.
cudnnStatus_t cudnnDeriveBNTensorDescriptor( cudnnTensorDescriptor_t derivedBnDesc, const cudnnTensorDescriptor_t xDesc, cudnnBatchNormMode_t mode)
Use the tensor descriptor produced by this function as the bnScaleBiasMeanVarDesc
parameter for the cudnnBatchNormalizationForwardInference() and cudnnBatchNormalizationForwardTraining() functions, and as the bnScaleBiasDiffDesc
parameter in the cudnnBatchNormalizationBackward() function.
The resulting dimensions will be:
1xCx1x1 for 4D and 1xCx1x1x1 for 5D for
BATCHNORM_MODE_SPATIAL
1xCxHxW for 4D and 1xCxDxHxW for 5D for
BATCHNORM_MODE_PER_ACTIVATION
mode
For HALF
input data type the resulting tensor descriptor will have a FLOAT
type. For other data types, it will have the same type as the input data.
Note
Only 4D and 5D tensors are supported.
The
derivedBnDesc
should be first created using cudnnCreateTensorDescriptor().
xDesc
is the descriptor for the layer’sx
data and has to be set up with proper dimensions prior to calling this function.
Parameters
derivedBnDesc
Output. Handle to a previously created tensor descriptor.
xDesc
Input. Handle to a previously created and initialized layer’s
x
data descriptor.mode
Input. Batch normalization layer mode of operation.
Returns
CUDNN_STATUS_SUCCESS
The computation was performed successfully.
CUDNN_STATUS_BAD_PARAM
Invalid batch normalization mode.
cudnnDeriveNormTensorDescriptor()
This function derives tensor descriptors for the normalization mean
, invariance
, normBias
, and normScale
subtensors from the layer’s x
data descriptor and norm mode. Normalization mean
and invariance
share the same descriptor while bias
and scale
share the same descriptor.
cudnnStatus_t CUDNNWINAPI cudnnDeriveNormTensorDescriptor(cudnnTensorDescriptor_t derivedNormScaleBiasDesc, cudnnTensorDescriptor_t derivedNormMeanVarDesc, const cudnnTensorDescriptor_t xDesc, cudnnNormMode_t mode, int groupCnt)
Use the tensor descriptor produced by this function as the normScaleBiasDesc
or normMeanVarDesc
parameter for the cudnnNormalizationForwardInference() and cudnnNormalizationForwardTraining() functions, and as the dNormScaleBiasDesc
and normMeanVarDesc
parameters in the cudnnNormalizationBackward() function.
The resulting dimensions will be:
1xCx1x1 for 4D and 1xCx1x1x1 for 5D for
CUDNN_NORM_PER_ACTIVATION
1xCxHxW for 4D and 1xCxDxHxW for 5D for
CUDNN_NORM_PER_CHANNEL
mode
For HALF
input data type the resulting tensor descriptor will have a FLOAT
type. For other data types, it will have the same type as the input data.
Note
Only 4D and 5D tensors are supported.
The
derivedNormScaleBiasDesc
andderivedNormMeanVarDesc
should be first created using cudnnCreateTensorDescriptor().
xDesc
is the descriptor for the layer’sx
data and has to be set up with proper dimensions prior to calling this function.
Parameters
derivedNormScaleBiasDesc
Output. Handle to a previously created tensor descriptor.
derivedNormMeanVarDesc
Output. Handle to a previously created tensor descriptor.
xDesc
Input. Handle to a previously created and initialized layer’s
x
data descriptor.mode
Input. Batch normalization layer mode of operation.
Returns
CUDNN_STATUS_SUCCESS
The computation was performed successfully.
CUDNN_STATUS_BAD_PARAM
Invalid batch normalization mode.
cudnnDestroyActivationDescriptor()
This function destroys a previously created activation descriptor object.
cudnnStatus_t cudnnDestroyActivationDescriptor( cudnnActivationDescriptor_t activationDesc)
Returns
CUDNN_STATUS_SUCCESS
The object was destroyed successfully.
cudnnDestroyDropoutDescriptor()
This function destroys a previously created dropout descriptor object.
cudnnStatus_t cudnnDestroyDropoutDescriptor( cudnnDropoutDescriptor_t dropoutDesc)
Returns
CUDNN_STATUS_SUCCESS
The object was destroyed successfully.
cudnnDestroyFilterDescriptor()
This function has been deprecated in cuDNN 9.0.
This function destroys a filter object.
cudnnStatus_t cudnnDestroyFilterDescriptor( cudnnFilterDescriptor_t filterDesc)
Returns
CUDNN_STATUS_SUCCESS
The object was destroyed successfully.
cudnnDestroyLRNDescriptor()
This function destroys a previously created LRN descriptor object.
cudnnStatus_t cudnnDestroyLRNDescriptor( cudnnLRNDescriptor_t lrnDesc)
Returns
CUDNN_STATUS_SUCCESS
The object was destroyed successfully.
cudnnDestroyOpTensorDescriptor()
This function deletes a tensor pointwise math descriptor object.
cudnnStatus_t cudnnDestroyOpTensorDescriptor( cudnnOpTensorDescriptor_t opTensorDesc)
Parameters
opTensorDesc
Input. Pointer to the structure holding the description of the tensor pointwise math to be deleted.
Returns
CUDNN_STATUS_SUCCESS
The function returned successfully.
cudnnDestroyPoolingDescriptor()
This function destroys a previously created pooling descriptor object.
cudnnStatus_t cudnnDestroyPoolingDescriptor( cudnnPoolingDescriptor_t poolingDesc)
Returns
CUDNN_STATUS_SUCCESS
The object was destroyed successfully.
cudnnDestroyReduceTensorDescriptor()
This function destroys a previously created reduce tensor descriptor object. When the input pointer is NULL
, this function performs no destroy operation.
cudnnStatus_t cudnnDestroyReduceTensorDescriptor( cudnnReduceTensorDescriptor_t tensorDesc)
Parameters
tensorDesc
Input. Pointer to the reduce tensor descriptor object to be destroyed.
Returns
CUDNN_STATUS_SUCCESS
The object was destroyed successfully.
cudnnDestroySpatialTransformerDescriptor()
This function destroys a previously created spatial transformer descriptor object.
cudnnStatus_t cudnnDestroySpatialTransformerDescriptor( cudnnSpatialTransformerDescriptor_t stDesc)
Returns
CUDNN_STATUS_SUCCESS
The object was destroyed successfully.
cudnnDestroyTensorDescriptor()
This function destroys a previously created tensor descriptor object. When the input pointer is NULL
, this function performs no destroy operation.
cudnnStatus_t cudnnDestroyTensorDescriptor(cudnnTensorDescriptor_t tensorDesc)
Parameters
tensorDesc
Input. Pointer to the tensor descriptor object to be destroyed.
Returns
CUDNN_STATUS_SUCCESS
The object was destroyed successfully.
cudnnDestroyTensorTransformDescriptor()
This function has been deprecated in cuDNN 9.0.
Destroys a previously created tensor transform descriptor.
cudnnStatus_t cudnnDestroyTensorTransformDescriptor( cudnnTensorTransformDescriptor_t transformDesc);
Parameters
transformDesc
Input. The tensor transform descriptor to be destroyed.
Returns
CUDNN_STATUS_SUCCESS
The object was destroyed successfully.
cudnnDivisiveNormalizationBackward()
This function performs the backward DivisiveNormalization
layer computation.
cudnnStatus_t cudnnDivisiveNormalizationBackward( cudnnHandle_t handle, cudnnLRNDescriptor_t normDesc, cudnnDivNormMode_t mode, const void *alpha, const cudnnTensorDescriptor_t xDesc, const void *x, const void *means, const void *dy, void *temp, void *temp2, const void *beta, const cudnnTensorDescriptor_t dxDesc, void *dx, void *dMeans)
Supported tensor formats are NCHW for 4D and NCDHW for 5D with any nonoverlapping nonnegative strides. Only 4D and 5D tensors are supported.
Parameters
handle
Input. Handle to a previously created cuDNN library descriptor.
normDesc
Input. Handle to a previously initialized LRN parameter descriptor (this descriptor is used for both LRN and
DivisiveNormalization
layers).mode
Input.
DivisiveNormalization
layer mode of operation. Currently onlyCUDNN_DIVNORM_PRECOMPUTED_MEANS
is implemented. Normalization is performed using the means input tensor that is expected to be precomputed by the user.alpha
,beta
Inputs. Pointers to scaling factors (in host memory) used to blend the layer output value with prior value in the destination tensor as follows:
dstValue = alpha[0]*resultValue + beta[0]*priorDstValue.
For more information, refer to Scaling Parameters.
xDesc
,x
,means
Inputs. Tensor descriptor and pointers in device memory for the layer’s
x
and means data. Note that themeans
tensor is expected to be precomputed by the user. It can also contain any valid values (not required to be actualmeans
, and can be for instance a result of a convolution with a Gaussian kernel).dy
Input. Tensor pointer in device memory for the layer’s
dy
cumulative loss differential data (error backpropagation).temp
,temp2
Workspace. Temporary tensors in device memory. These are used for computing intermediate values during the backward pass. These tensors do not have to be preserved from forward to backward pass. Both use
xDesc
as a descriptor.dxDesc
Input. Tensor descriptor for
dx
anddMeans
.dx
,dMeans
Outputs. Tensor pointers (in device memory) for the layers resulting in cumulative gradients
dx
anddMeans
(dLoss/dx
anddLoss/dMeans
). Both share the same descriptor.
Returns
CUDNN_STATUS_SUCCESS
The computation was performed successfully.
CUDNN_STATUS_BAD_PARAM
At least one of the following conditions are met:
One of the tensor pointers
x
,dx
,temp
,tmep2
, anddy
isNULL
.Number of any of the input or output tensor dimensions is not within the [4,5] range.
Either
alpha
orbeta
pointer isNULL
.A mismatch in dimensions between
xDesc
anddxDesc
.LRN descriptor parameters are outside of their valid ranges.
Any of the tensor strides is negative.
CUDNN_STATUS_UNSUPPORTED
The function does not support the provided configuration, for example, any of the input and output tensor strides mismatch (for the same dimension) is a nonsupported configuration.
cudnnDivisiveNormalizationForward()
This function performs the forward spatial DivisiveNormalization
layer computation. It divides every value in a layer by the standard deviation of its spatial neighbors. Note that DivisiveNormalization
only implements the x/max(c, sigma_x)
portion of the computation, where sigma_x
is the variance over the spatial neighborhood of x
.
cudnnStatus_t cudnnDivisiveNormalizationForward( cudnnHandle_t handle, cudnnLRNDescriptor_t normDesc, cudnnDivNormMode_t mode, const void *alpha, const cudnnTensorDescriptor_t xDesc, const void *x, const void *means, void *temp, void *temp2, const void *beta, const cudnnTensorDescriptor_t yDesc, void *y)
The full LCN (Local Contrastive Normalization) computation can be implemented as a twostep process:
x_m = xmean(x); y = x_m/max(c, sigma(x_m));
The xmean(x)
which is often referred to as “subtractive normalization” portion of the computation can be implemented using cuDNN average pooling layer followed by a call to addTensor
.
Supported tensor formats are NCHW for 4D and NCDHW for 5D with any nonoverlapping nonnegative strides. Only 4D and 5D tensors are supported.
Parameters
handle
Input. Handle to a previously created cuDNN library descriptor.
normDesc
Input. Handle to a previously initialized LRN parameter descriptor. This descriptor is used for both LRN and
DivisiveNormalization
layers.divNormMode
Input.
DivisiveNormalization
layer mode of operation. Currently onlyCUDNN_DIVNORM_PRECOMPUTED_MEANS
is implemented. Normalization is performed using the means input tensor that is expected to be precomputed by the user.alpha
,beta
Input. Pointers to scaling factors (in host memory) used to blend the layer output value with prior value in the destination tensor as follows:
dstValue = alpha[0]*resultValue + beta[0]*priorDstValue
For more information, refer to Scaling Parameters.
xDesc
,yDesc
Input. Tensor descriptor objects for the input and output tensors. Note that
xDesc
is shared betweenx
,means
,temp
, andtemp2
tensors.x
Input. Input tensor data pointer in device memory.
means
Input. Input means tensor data pointer in device memory. Note that this tensor can be
NULL
(in that case its values are assumed to be zero during the computation). This tensor also doesn’t have to containmeans
, these can be any values, a frequently used variation is a result of convolution with a normalized positive kernel (such as Gaussian).temp
,temp2
Workspace. Temporary tensors in device memory. These are used for computing intermediate values during the forward pass. These tensors do not have to be preserved as inputs from forward to the backward pass. Both use
xDesc
as their descriptor.y
Output. Pointer in device memory to a tensor for the result of the forward
DivisiveNormalization
computation.
Returns
CUDNN_STATUS_SUCCESS
The computation was performed successfully.
CUDNN_STATUS_BAD_PARAM
At least one of the following conditions are met:
One of the tensor pointers
x
,y
,temp
, andtemp2
isNULL
.Number of input tensor or output tensor dimensions is outside of [4,5] range.
A mismatch in dimensions between any two of the input or output tensors.
For inplace computation when pointers
x == y
, a mismatch in strides between the input data and output data tensors.alpha
orbeta
pointer isNULL
.LRN descriptor parameters are outside of their valid ranges.
Any of the tensor strides are negative.
CUDNN_STATUS_UNSUPPORTED
The function does not support the provided configuration, for example, any of the input and output tensor strides mismatch (for the same dimension) is a nonsupported configuration.
cudnnDropoutBackward()
This function performs backward dropout operation over dy
returning results in dx
. If during forward dropout operation value from x
was propagated to y
then during backward operation value from dy
will be propagated to dx
, otherwise, dx
value will be set to 0
.
cudnnStatus_t cudnnDropoutBackward( cudnnHandle_t handle, const cudnnDropoutDescriptor_t dropoutDesc, const cudnnTensorDescriptor_t dydesc, const void *dy, const cudnnTensorDescriptor_t dxdesc, void *dx, void *reserveSpace, size_t reserveSpaceSizeInBytes)
Better performance is obtained for fully packed tensors.
Parameters
handle
Input. Handle to a previously created cuDNN context.
dropoutDesc
Input. Previously created dropout descriptor object.
dyDesc
Input. Handle to a previously initialized tensor descriptor.
dy
Input. Pointer to data of the tensor described by the
dyDesc
descriptor.dxDesc
Input. Handle to a previously initialized tensor descriptor.
dx
Output. Pointer to data of the tensor described by the
dxDesc
descriptor.reserveSpace
Input. Pointer to userallocated GPU memory used by this function. It is expected that
reserveSpace
was populated during a call tocudnnDropoutForward
and has not been changed.reserveSpaceSizeInBytes
Input. Specifies the size in bytes of the provided memory for the reserve space
Returns
CUDNN_STATUS_SUCCESS
The call was successful.
CUDNN_STATUS_NOT_SUPPORTED
The function does not support the provided configuration.
CUDNN_STATUS_BAD_PARAM
At least one of the following conditions are met:
The number of elements of input tensor and output tensors differ.
The
datatype
of the input tensor and output tensors differs.The strides of the input tensor and output tensors differ and inplace operation is used (meaning,
x
andy
pointers are equal).The provided
reserveSpaceSizeInBytes
is less than the value returned bycudnnDropoutGetReserveSpaceSize()
.cudnnSetDropoutDescriptor()
has not been called ondropoutDesc
with nonNULL
states argument.
CUDNN_STATUS_EXECUTION_FAILED
The function failed to launch on the GPU.
cudnnDropoutForward()
This function performs forward dropout operation over x
returning results in y
. If dropout was used as a parameter to cudnnSetDropoutDescriptor(), the approximate dropout
fraction of x
values will be replaced by a 0
, and the rest will be scaled by 1/(1dropout)
. This function should not be running concurrently with another cudnnDropoutForward()
function using the same states.
cudnnStatus_t cudnnDropoutForward( cudnnHandle_t handle, const cudnnDropoutDescriptor_t dropoutDesc, const cudnnTensorDescriptor_t xdesc, const void *x, const cudnnTensorDescriptor_t ydesc, void *y, void *reserveSpace, size_t reserveSpaceSizeInBytes)Note
Better performance is obtained for fully packed tensors.
This function should not be called during inference.
Parameters
handle
Input. Handle to a previously created cuDNN context.
dropoutDesc
Input. Previously created dropout descriptor object.
xDesc
Input. Handle to a previously initialized tensor descriptor.
x
Input. Pointer to data of the tensor described by the
xDesc
descriptor.yDesc
Input. Handle to a previously initialized tensor descriptor.
y
Output. Pointer to data of the tensor described by the
yDesc
descriptor.reserveSpace
Output. Pointer to userallocated GPU memory used by this function. It is expected that the contents of
reserveSpace
does not change betweencudnnDropoutForward()
and cudnnDropoutBackward() calls.reserveSpaceSizeInBytes
Input. Specifies the size in bytes of the provided memory for the reserve space.
Returns
CUDNN_STATUS_SUCCESS
The call was successful.
CUDNN_STATUS_NOT_SUPPORTED
The function does not support the provided configuration.
CUDNN_STATUS_BAD_PARAM
At least one of the following conditions are met:
The number of elements of input tensor and output tensors differ.
The
datatype
of the input tensor and output tensors differs.The strides of the input tensor and output tensors differ and inplace operation is used (meaning,
x
andy
pointers are equal).The provided
reserveSpaceSizeInBytes
is less than the value returned by cudnnDropoutGetReserveSpaceSize().cudnnSetDropoutDescriptor() has not been called on
dropoutDesc
with nonNULL
states argument.
CUDNN_STATUS_EXECUTION_FAILED
The function failed to launch on the GPU.
cudnnDropoutGetReserveSpaceSize()
This function is used to query the amount of reserve needed to run dropout with the input dimensions given by xDesc
. The same reserve space is expected to be passed to cudnnDropoutForward() and cudnnDropoutBackward(), and its contents is expected to remain unchanged between cudnnDropoutForward() and cudnnDropoutBackward() calls.
cudnnStatus_t cudnnDropoutGetReserveSpaceSize( cudnnTensorDescriptor_t xDesc, size_t *sizeInBytes)
Parameters
xDesc
Input. Handle to a previously initialized tensor descriptor, describing input to a dropout operation.
sizeInBytes
Output. Amount of GPU memory needed as reserve space to be able to run dropout with an input tensor descriptor specified by
xDesc
.
Returns
CUDNN_STATUS_SUCCESS
The query was successful.
cudnnDropoutGetStatesSize()
This function is used to query the amount of space required to store the states of the random number generators used by the cudnnDropoutForward() function.
cudnnStatus_t cudnnDropoutGetStatesSize( cudnnHandle_t handle, size_t *sizeInBytes)
Parameters
handle
Input. Handle to a previously created cuDNN context.
sizeInBytes
Output. Amount of GPU memory needed to store random generator states.
Returns
CUDNN_STATUS_SUCCESS
The query was successful.
cudnnGetActivationDescriptor()
This function queries a previously initialized generic activation descriptor object.
cudnnStatus_t cudnnGetActivationDescriptor( const cudnnActivationDescriptor_t activationDesc, cudnnActivationMode_t *mode, cudnnNanPropagation_t *reluNanOpt, double *coef)
Parameters
activationDesc
Input. Handle to a previously created activation descriptor.
mode
Output. Enumerant to specify the activation mode.
reluNanOpt
Output. Enumerant to specify the Nan propagation mode.
coef
Output. Floating point number to specify the clipping threshold when the activation mode is set to
CUDNN_ACTIVATION_CLIPPED_RELU
or to specify the alpha coefficient when the activation mode is set toCUDNN_ACTIVATION_ELU
.
Returns
CUDNN_STATUS_SUCCESS
The object was queried successfully.
cudnnGetActivationDescriptorSwishBeta()
This function queries the current beta parameter set for SWISH
activation.
cudnnStatus_t cudnnGetActivationDescriptorSwishBeta(cudnnActivationDescriptor_t activationDesc, double* swish_beta)
Parameters
activationDesc
Input. Handle to a previously created activation descriptor.
swish_beta
Output. Pointer to a double value that will receive the currently configured SWISH beta parameter.
Returns
CUDNN_STATUS_SUCCESS
The beta parameter was queried successfully.
CUDNN_STATUS_BAD_PARAM
At least one of
activationDesc
orswish_beta
wereNULL
.
cudnnGetBatchNormalizationBackwardExWorkspaceSize()
This function returns the amount of GPU memory workspace the user should allocate to be able to call cudnnGetBatchNormalizationBackwardExWorkspaceSize()
function for the specified bnOps
input setting. The workspace allocated will then be passed to the function cudnnGetBatchNormalizationBackwardExWorkspaceSize()
.
cudnnStatus_t cudnnGetBatchNormalizationBackwardExWorkspaceSize( cudnnHandle_t handle, cudnnBatchNormMode_t mode, cudnnBatchNormOps_t bnOps, const cudnnTensorDescriptor_t xDesc, const cudnnTensorDescriptor_t yDesc, const cudnnTensorDescriptor_t dyDesc, const cudnnTensorDescriptor_t dzDesc, const cudnnTensorDescriptor_t dxDesc, const cudnnTensorDescriptor_t dBnScaleBiasDesc, const cudnnActivationDescriptor_t activationDesc, size_t *sizeInBytes);
Parameters
handle
Input. Handle to a previously created cuDNN library descriptor. For more information, refer to cudnnHandle_t.
mode
Input. Mode of operation (spatial or peractivation). For more information, refer to cudnnBatchNormMode_t.
bnOps
Input. Mode of operation for the fast NHWC kernel. For more information, refer to cudnnBatchNormOps_t. This input can be used to set this function to perform either only the batch normalization, or batch normalization followed by activation, or batch normalization followed by elementwise addition and then activation.
xDesc
,yDesc
,dyDesc
,dzDesc
,dxDesc
Inputs. Tensor descriptors and pointers in the device memory for the layer’s
x
data, back propagated differentialdy
(inputs), the optionaly
input data, the optionaldz
output, and thedx
output, which is the resulting differential with respect tox
. For more information, refer to cudnnTensorDescriptor_t.dBnScaleBiasDesc
Input. Shared tensor descriptor for the following six tensors:
bnScaleData
,bnBiasData
,dBnScaleData
,dBnBiasData
,savedMean
, andsavedInvVariance
. This is the shared tensor descriptor desc for the secondary tensor that was derived by cudnnDeriveBNTensorDescriptor(). The dimensions for this tensor descriptor are dependent on normalization mode. Note that the data type of this tensor descriptor must befloat
for FP16 and FP32 input tensors, anddouble
for FP64 input tensors.activationDesc
Input. Descriptor for the activation operation. When the
bnOps
input is set to eitherCUDNN_BATCHNORM_OPS_BN_ACTIVATION
orCUDNN_BATCHNORM_OPS_BN_ADD_ACTIVATION
, then this activation is used, otherwise user may passNULL
.*sizeInBytes
Output. Amount of GPU memory required for the workspace, as determined by this function, to be able to execute the
cudnnGetBatchNormalizationBackwardExWorkspaceSize()
function with the specifiedbnOps
input setting.
Returns
CUDNN_STATUS_SUCCESS
The computation was performed successfully.
CUDNN_STATUS_NOT_SUPPORTED
The function does not support the provided configuration.
CUDNN_STATUS_BAD_PARAM
At least one of the following conditions are met:
Number of
xDesc
,yDesc
, ordxDesc
tensor descriptor dimensions is not within the range of [4,5] (only 4D and 5D tensors are supported).dBnScaleBiasDesc
dimensions not 1xCx1x1 for 4D and 1xCx1x1x1 for 5D for spatial, and are not 1xCxHxW for 4D and 1xCxDxHxW for 5D for peractivation mode.Dimensions or data types mismatch for any pair of
xDesc
,dyDesc
, ordxDesc
.
cudnnGetBatchNormalizationForwardTrainingExWorkspaceSize()
This function has been deprecated in cuDNN 9.0.
This function returns the amount of GPU memory workspace the user should allocate to be able to call cudnnGetBatchNormalizationForwardTrainingExWorkspaceSize()
function for the specified bnOps
input setting. The workspace allocated should then be passed by the user to the function cudnnGetBatchNormalizationForwardTrainingExWorkspaceSize()
.
cudnnStatus_t cudnnGetBatchNormalizationForwardTrainingExWorkspaceSize( cudnnHandle_t handle, cudnnBatchNormMode_t mode, cudnnBatchNormOps_t bnOps, const cudnnTensorDescriptor_t xDesc, const cudnnTensorDescriptor_t zDesc, const cudnnTensorDescriptor_t yDesc, const cudnnTensorDescriptor_t bnScaleBiasMeanVarDesc, const cudnnActivationDescriptor_t activationDesc, size_t *sizeInBytes);
Parameters
handle
Input. Handle to a previously created cuDNN library descriptor. For more information, refer to cudnnHandle_t.
mode
Input. Mode of operation (spatial or peractivation). For more information, refer to cudnnBatchNormMode_t.
bnOps
Input. Mode of operation for the fast NHWC kernel. For more information, refer to cudnnBatchNormOps_t. This input can be used to set this function to perform either only the batch normalization, or batch normalization followed by activation, or batch normalization followed by elementwise addition and then activation.
xDesc
,zDesc
,yDesc
Inputs. Tensor descriptors and pointers in the device memory for the layer’s
x
data, the optionalz
input data, and they
output.zDesc
is only needed whenbnOps
isCUDNN_BATCHNORM_OPS_BN_ADD_ACTIVATION
, otherwise the user may passNULL
. For more information, refer to cudnnTensorDescriptor_t.bnScaleBiasMeanVarDesc
Input. Shared tensor descriptor for the following six tensors:
bnScaleData
,bnBiasData
,dBnScaleData
,dBnBiasData
,savedMean
, andsavedInvVariance
. This is the shared tensor descriptor desc for the secondary tensor that was derived by cudnnDeriveBNTensorDescriptor(). The dimensions for this tensor descriptor are dependent on normalization mode. Note that the data type of this tensor descriptor must befloat
for FP16 and FP32 input tensors, anddouble
for FP64 input tensors.activationDesc
Input. Descriptor for the activation operation. When the
bnOps
input is set to eitherCUDNN_BATCHNORM_OPS_BN_ACTIVATION
orCUDNN_BATCHNORM_OPS_BN_ADD_ACTIVATION
, then this activation is used, otherwise user may passNULL
.*sizeInBytes
Output. Amount of GPU memory required for the workspace, as determined by this function, to be able to execute the
cudnnGetBatchNormalizationBackwardExWorkspaceSize()
function with the specifiedbnOps
input setting.
Returns
CUDNN_STATUS_SUCCESS
The computation was performed successfully.
CUDNN_STATUS_NOT_SUPPORTED
The function does not support the provided configuration.
CUDNN_STATUS_BAD_PARAM
At least one of the following conditions are met:
Number of
xDesc
,yDesc
, ordxDesc
tensor descriptor dimensions is not within the range of [4,5] (only 4D and 5D tensors are supported).dBnScaleBiasDesc
dimensions not 1xCx1x1 for 4D and 1xCx1x1x1 for 5D for spatial, and are not 1xCxHxW for 4D and 1xCxDxHxW for 5D for peractivation mode.Dimensions or data types mismatch for any pair of
xDesc
ordyDesc
.
cudnnGetBatchNormalizationTrainingExReserveSpaceSize()
This function has been deprecated in cuDNN 9.0.
This function returns the amount of reserve GPU memory workspace the user should allocate for the batch normalization operation, for the specified bnOps
input setting. In contrast to the workspace
, the reserved space should be preserved between the forward and backward calls, and the data should not be altered.
cudnnStatus_t cudnnGetBatchNormalizationTrainingExReserveSpaceSize( cudnnHandle_t handle, cudnnBatchNormMode_t mode, cudnnBatchNormOps_t bnOps, const cudnnActivationDescriptor_t activationDesc, const cudnnTensorDescriptor_t xDesc, size_t *sizeInBytes);
Parameters
handle
Input. Handle to a previously created cuDNN library descriptor. For more information, refer to cudnnHandle_t.
mode
Input. Mode of operation (spatial or peractivation). For more information, refer to cudnnBatchNormMode_t.
bnOps
Input. Mode of operation for the fast NHWC kernel. For more information, refer to cudnnBatchNormOps_t. This input can be used to set this function to perform either only the batch normalization, or batch normalization followed by activation, or batch normalization followed by elementwise addition and then activation.
xDesc
Input. Tensor descriptors and pointers in the device memory for the layer’s
x
data. For more information, refer to cudnnTensorDescriptor_t.activationDesc
Input. Descriptor for the activation operation. When the
bnOps
input is set to eitherCUDNN_BATCHNORM_OPS_BN_ACTIVATION
orCUDNN_BATCHNORM_OPS_BN_ADD_ACTIVATION
, then this activation is used, otherwise user may passNULL
.*sizeInBytes
Output. Amount of GPU memory reserved.
Returns
CUDNN_STATUS_SUCCESS
The computation was performed successfully.
CUDNN_STATUS_NOT_SUPPORTED
The function does not support the provided configuration.
CUDNN_STATUS_BAD_PARAM
At least one of the following conditions are met:
The
xDesc
tensor descriptor dimension is not within the [4,5] range (only 4D and 5D tensors are supported).
cudnnGetDropoutDescriptor()
This function queries the fields of a previously initialized dropout descriptor.
cudnnStatus_t cudnnGetDropoutDescriptor( cudnnDropoutDescriptor_t dropoutDesc, cudnnHandle_t handle, float *dropout, void **states, unsigned long long *seed)
Parameters
dropoutDesc
Input. Previously initialized dropout descriptor.
handle
Input. Handle to a previously created cuDNN context.
dropout
Output. The probability with which the value from input is set to 0 during the dropout layer.
states
Output. Pointer to userallocated GPU memory that holds random number generator states.
seed
Output. Seed used to initialize random number generator states.
Returns
CUDNN_STATUS_SUCCESS
The call was successful.
CUDNN_STATUS_BAD_PARAM
One or more of the arguments was an invalid pointer.
cudnnGetFilter4dDescriptor()
This function has been deprecated in cuDNN 9.0.
This function queries the parameters of the previously initialized Filter4d
descriptor object.
cudnnStatus_t cudnnGetFilter4dDescriptor( const cudnnFilterDescriptor_t filterDesc, cudnnDataType_t *dataType, cudnnTensorFormat_t *format, int *k, int *c, int *h, int *w)
Parameters
filterDesc
Input. Handle to a previously created filter descriptor.
datatype
Output. Data type.
format
Output. Type of format.
k
Output. Number of output feature maps.
c
Output. Number of input feature maps.
h
Output. Height of each filter.
w
Output. Width of each filter.
Returns
CUDNN_STATUS_SUCCESS
The object was set successfully.
cudnnGetFilterNdDescriptor()
This function has been deprecated in cuDNN 9.0.
This function queries a previously initialized FilterNd
descriptor object.
cudnnStatus_t cudnnGetFilterNdDescriptor( const cudnnFilterDescriptor_t wDesc, int nbDimsRequested, cudnnDataType_t *dataType, cudnnTensorFormat_t *format, int *nbDims, int filterDimA[])
Parameters
wDesc
Input. Handle to a previously initialized filter descriptor.
nbDimsRequested
Input. Dimension of the expected filter descriptor. It is also the minimum size of the arrays filterDimA in order to be able to hold the results
datatype
Output. Data type.
format
Output. Type of format.
nbDims
Output. Actual dimension of the filter.
filterDimA
Output. Array of dimensions of at least
nbDimsRequested
that will be filled with the filter parameters from the provided filter descriptor.
Returns
CUDNN_STATUS_SUCCESS
The object was set successfully.
CUDNN_STATUS_BAD_PARAM
The parameter
nbDimsRequested
is negative.
cudnnGetFilterSizeInBytes()
This function has been deprecated in cuDNN 9.0.
This function returns the size of the filter tensor in memory with respect to the given descriptor. It can be used to know the amount of GPU memory to be allocated to hold that filter tensor.
cudnnStatus_t cudnnGetFilterSizeInBytes(const cudnnFilterDescriptor_t filterDesc, size_t *size);
Parameters
filterDesc
Input. handle to a previously initialized filter descriptor.
size
Output. size in bytes needed to hold the tensor in GPU memory.
Returns
CUDNN_STATUS_SUCCESS
filterDesc
is valid.CUDNN_STATUS_BAD_PARAM
filerDesc
is invalid.
cudnnGetLRNDescriptor()
This function retrieves values stored in the previously initialized LRN
descriptor object.
cudnnStatus_t cudnnGetLRNDescriptor( cudnnLRNDescriptor_t normDesc, unsigned *lrnN, double *lrnAlpha, double *lrnBeta, double *lrnK)
Parameters
normDesc
Output. Handle to a previously created LRN descriptor.
lrnN
, lrnAlpha
, lrnBeta
, lrnK
Outputs. Pointers to receive values of parameters stored in the descriptor object. For more information, refer to cudnnSetLRNDescriptor(). Any of these pointers can be NULL
(no value is returned for the corresponding parameter).
Returns
CUDNN_STATUS_SUCCESS
Function completed successfully.
cudnnGetNormalizationBackwardWorkspaceSize()
This function returns the amount of GPU memory workspace the user should allocate to be able to call cudnnNormalizationBackward() function for the specified normOps
and algo
input setting. The workspace allocated will then be passed to the function cudnnNormalizationBackward().
cudnnStatus_t cudnnGetNormalizationBackwardWorkspaceSize(cudnnHandle_t handle, cudnnNormMode_t mode, cudnnNormOps_t normOps, cudnnNormAlgo_t algo, const cudnnTensorDescriptor_t xDesc, const cudnnTensorDescriptor_t yDesc, const cudnnTensorDescriptor_t dyDesc, const cudnnTensorDescriptor_t dzDesc, const cudnnTensorDescriptor_t dxDesc, const cudnnTensorDescriptor_t dNormScaleBiasDesc, const cudnnActivationDescriptor_t activationDesc, const cudnnTensorDescriptor_t normMeanVarDesc, size_t *sizeInBytes, int groupCnt);
Parameters
handle
Input. Handle to a previously created cuDNN library descriptor. For more information, refer to cudnnHandle_t.
mode
Input. Mode of operation (perchannel or peractivation). For more information, refer to cudnnNormMode_t.
normOps
Input. Mode of postoperative. Currently
CUDNN_NORM_OPS_NORM_ACTIVATION
andCUDNN_NORM_OPS_NORM_ADD_ACTIVATION
are only supported in the NHWC layout. For more information, refer to cudnnNormOps_t. This input can be used to set this function to perform either only the normalization, or normalization followed by activation, or normalization followed by elementwise addition and then activation.algo
Input. Algorithm to be performed. For more information, refer to cudnnNormAlgo_t.
xDesc
,yDesc
,dyDesc
,dzDesc
,dxDesc
Inputs. Tensor descriptors and pointers in the device memory for the layer’s
x
data, back propagated differentialdy
(inputs), the optionaly
input data, the optionaldz
output, and thedx
output, which is the resulting differential with respect tox
. For more information, refer to cudnnTensorDescriptor_t.dNormScaleBiasDesc
Input. Shared tensor descriptor for the following four tensors:
normScaleData
,normBiasData
,dNormScaleData
,dNormBiasData
. The dimensions for this tensor descriptor are dependent on normalization mode. Note that the data type of this tensor descriptor must befloat
for FP16 and FP32 input tensors, anddouble
for FP64 input tensors.activationDesc
Input. Descriptor for the activation operation. When the
normOps
input is set to eitherCUDNN_NORM_OPS_NORM_ACTIVATION
orCUDNN_NORM_OPS_NORM_ADD_ACTIVATION
, then this activation is used, otherwise the user may passNULL
.normMeanVarDesc
Input. Shared tensor descriptor for the following tensors:
savedMean
andsavedInvVariance
. The dimensions for this tensor descriptor are dependent on normalization mode. Note that the data type of this tensor descriptor must befloat
for FP16 and FP32 input tensors, anddouble
for FP64 input tensors.*sizeInBytes
Output. Amount of GPU memory required for the workspace, as determined by this function, to be able to execute the cudnnGetNormalizationForwardTrainingWorkspaceSize() function with the specified
normOps
input setting.groutCnt
Input. Only support 1 for now.
Returns
CUDNN_STATUS_SUCCESS
The computation was performed successfully.
CUDNN_STATUS_NOT_SUPPORTED
The function does not support the provided configuration.
CUDNN_STATUS_BAD_PARAM
At least one of the following conditions are met:
Number of
xDesc
,yDesc
, ordxDesc
tensor descriptor dimensions is not within the range of [4,5] (only 4D and 5D tensors are supported).dNormScaleBiasDesc
dimensions not 1xCx1x1 for 4D and 1xCx1x1x1 for 5D for perchannel, and are not 1xCxHxW for 4D and 1xCxDxHxW for 5D for peractivation mode.Dimensions or data types mismatch for any pair of
xDesc
,dyDesc
, ordxDesc
.
cudnnGetNormalizationForwardTrainingWorkspaceSize()
This function returns the amount of GPU memory workspace the user should allocate to be able to call cudnnNormalizationForwardTraining() function for the specified normOps
and algo
input setting. The workspace allocated should then be passed by the user to the function cudnnNormalizationForwardTraining().
cudnnStatus_t cudnnGetNormalizationForwardTrainingWorkspaceSize(cudnnHandle_t handle, cudnnNormMode_t mode, cudnnNormOps_t normOps, cudnnNormAlgo_t algo, const cudnnTensorDescriptor_t xDesc, const cudnnTensorDescriptor_t zDesc, const cudnnTensorDescriptor_t yDesc, const cudnnTensorDescriptor_t normScaleBiasDesc, const cudnnActivationDescriptor_t activationDesc, const cudnnTensorDescriptor_t normMeanVarDesc, size_t *sizeInBytes, int groupCnt);
Parameters
handle
Input. Handle to a previously created cuDNN library descriptor. For more information, refer to cudnnHandle_t.
mode
Input. Mode of operation (perchannel or peractivation). For more information, refer to cudnnNormMode_t.
normOps
Input. Mode of postoperative. Currently
CUDNN_NORM_OPS_NORM_ACTIVATION
andCUDNN_NORM_OPS_NORM_ADD_ACTIVATION
are only supported in the NHWC layout. For more information, refer to cudnnNormOps_t. This input can be used to set this function to perform either only the normalization, or normalization followed by activation, or normalization followed by elementwise addition and then activation.algo
Input. Algorithm to be performed. For more information, refer to cudnnNormAlgo_t.
xDesc
,zDesc
,yDesc
Inputs. Tensor descriptors and pointers in the device memory for the layer’s
x
data, back propagated differentialdy
(inputs), the optionaly
input data, the optionaldz
output, and thedx
output, which is the resulting differential with respect tox
. For more information, refer to cudnnTensorDescriptor_t.normScaleBiasDesc
Input. Shared tensor descriptor for the following four tensors:
normScaleData
andnormBiasData
. The dimensions for this tensor descriptor are dependent on normalization mode. Note that the data type of this tensor descriptor must befloat
for FP16 and FP32 input tensors, anddouble
for FP64 input tensors.activationDesc
Input. Descriptor for the activation operation. When the
normOps
input is set to eitherCUDNN_NORM_OPS_NORM_ACTIVATION
orCUDNN_NORM_OPS_NORM_ADD_ACTIVATION
, then this activation is used, otherwise the user may passNULL
.normMeanVarDesc
Input. Shared tensor descriptor for the following tensors:
savedMean
andsavedInvVariance
. The dimensions for this tensor descriptor are dependent on normalization mode. Note that the data type of this tensor descriptor must befloat
for FP16 and FP32 input tensors, anddouble
for FP64 input tensors.*sizeInBytes
Output. Amount of GPU memory required for the workspace, as determined by this function, to be able to execute the cudnnGetNormalizationForwardTrainingWorkspaceSize() function with the specified
normOps
input setting.groutCnt
Input. Only support 1 for now.
Returns
CUDNN_STATUS_SUCCESS
The computation was performed successfully.
CUDNN_STATUS_NOT_SUPPORTED
The function does not support the provided configuration.
CUDNN_STATUS_BAD_PARAM
At least one of the following conditions are met:
Number of
xDesc
,yDesc
, orzDesc
tensor descriptor dimensions is not within the range of [4,5] (only 4D and 5D tensors are supported).normScaleBiasDesc
dimensions not 1xCx1x1 for 4D and 1xCx1x1x1 for 5D for perchannel, and are not 1xCxHxW for 4D and 1xCxDxHxW for 5D for peractivation mode.Dimensions or data types mismatch for any pair of
xDesc
oryDesc
.
cudnnGetNormalizationTrainingReserveSpaceSize()
This function returns the amount of reserve GPU memory workspace the user should allocate for the normalization operation, for the specified normOps
input setting. In contrast to the workspace, the reserved space should be preserved between the forward and backward calls, and the data should not be altered.
cudnnStatus_t cudnnGetNormalizationTrainingReserveSpaceSize(cudnnHandle_t handle, cudnnNormMode_t mode, cudnnNormOps_t normOps, cudnnNormAlgo_t algo, const cudnnActivationDescriptor_t activationDesc, const cudnnTensorDescriptor_t xDesc, size_t *sizeInBytes, int groupCnt);
Parameters
handle
Input. Handle to a previously created cuDNN library descriptor. For more information, refer to cudnnHandle_t.
mode
Input. Mode of operation (perchannel or peractivation). For more information, refer to cudnnNormMode_t.
normOps
Input. Mode of postoperative. Currently
CUDNN_NORM_OPS_NORM_ACTIVATION
andCUDNN_NORM_OPS_NORM_ADD_ACTIVATION
are only supported in the NHWC layout. For more information, refer to cudnnNormOps_t. This input can be used to set this function to perform either only the normalization, or normalization followed by activation, or normalization followed by elementwise addition and then activation.algo
Input. Algorithm to be performed. For more information, refer to cudnnNormAlgo_t.
xDesc
Input. Tensor descriptors for the layer’s
x
data. For more information, refer to cudnnTensorDescriptor_t.activationDesc
Input. Descriptor for the activation operation. When the
normOps
input is set to eitherCUDNN_NORM_OPS_NORM_ACTIVATION
orCUDNN_NORM_OPS_NORM_ADD_ACTIVATION
, then this activation is used, otherwise the user may passNULL
.*sizeInBytes
Output. Amount of GPU memory reserved.
groutCnt
Input. Only support 1 for now.
Returns
CUDNN_STATUS_SUCCESS
The computation was performed successfully.
CUDNN_STATUS_NOT_SUPPORTED
The function does not support the provided configuration.
CUDNN_STATUS_BAD_PARAM
At least one of the following conditions are met:
The
xDesc
tensor descriptor dimension is not within the [4,5] range (only 4D and 5D tensors are supported).
cudnnGetOpTensorDescriptor()
This function returns the configuration of the passed tensor pointwise math descriptor.
cudnnStatus_t cudnnGetOpTensorDescriptor( const cudnnOpTensorDescriptor_t opTensorDesc, cudnnOpTensorOp_t *opTensorOp, cudnnDataType_t *opTensorCompType, cudnnNanPropagation_t *opTensorNanOpt)
Parameters
opTensorDesc
Input. Tensor pointwise math descriptor passed to get the configuration from.
opTensorOp
Output. Pointer to the tensor pointwise math operation type, associated with this tensor pointwise math descriptor.
opTensorCompType
Output. Pointer to the cuDNN datatype associated with this tensor pointwise math descriptor.
opTensorNanOpt
Output. Pointer to the NAN propagation option associated with this tensor pointwise math descriptor.
Returns
CUDNN_STATUS_SUCCESS
The function returned successfully.
CUDNN_STATUS_BAD_PARAM
Input tensor pointwise math descriptor passed is invalid.
cudnnGetPooling2dDescriptor()
This function queries a previously created Pooling2d
descriptor object.
cudnnStatus_t cudnnGetPooling2dDescriptor( const cudnnPoolingDescriptor_t poolingDesc, cudnnPoolingMode_t *mode, cudnnNanPropagation_t *maxpoolingNanOpt, int *windowHeight, int *windowWidth, int *verticalPadding, int *horizontalPadding, int *verticalStride, int *horizontalStride)
Parameters
poolingDesc
Input. Handle to a previously created pooling descriptor.
mode
Output. Enumerant to specify the pooling mode.
maxpoolingNanOpt
Output. Enumerant to specify the Nan propagation mode.
windowHeight
Output. Height of the pooling window.
windowWidth
Output. Width of the pooling window.
verticalPadding
Output. Size of vertical padding.
horizontalPadding
Output. Size of horizontal padding.
verticalStride
Output. Pooling vertical stride.
horizontalStride
Output. Pooling horizontal stride.
Returns
CUDNN_STATUS_SUCCESS
The object was set successfully.
cudnnGetPooling2dForwardOutputDim()
This function provides the output dimensions of a tensor after Pooling2d
has been applied.
cudnnStatus_t cudnnGetPooling2dForwardOutputDim( const cudnnPoolingDescriptor_t poolingDesc, const cudnnTensorDescriptor_t inputDesc, int *outN, int *outC, int *outH, int *outW)
Each dimension h
and w
of the output images is computed as follows:
outputDim = 1 + (inputDim + 2*padding  windowDim)/poolingStride;
Parameters
poolingDesc
Input. Handle to a previously initialized pooling descriptor.
inputDesc
Input. Handle to the previously initialized input tensor descriptor.
N
Output. Number of images in the output.
C
Output. Number of channels in the output.
H
Output. Height of images in the output.
W
Output. Width of images in the output.
Returns
CUDNN_STATUS_SUCCESS
The function launched successfully.
CUDNN_STATUS_BAD_PARAM
At least one of the following conditions are met:
poolingDesc
has not been initialized.poolingDesc
orinputDesc
has an invalid number of dimensions (2 and 4 respectively are required).
cudnnGetPoolingNdDescriptor()
This function queries a previously initialized generic PoolingNd
descriptor object.
cudnnStatus_t cudnnGetPoolingNdDescriptor( const cudnnPoolingDescriptor_t poolingDesc, int nbDimsRequested, cudnnPoolingMode_t *mode, cudnnNanPropagation_t *maxpoolingNanOpt, int *nbDims, int windowDimA[], int paddingA[], int strideA[])
Parameters
poolingDesc
Input. Handle to a previously created pooling descriptor.
nbDimsRequested
Input. Dimension of the expected pooling descriptor. It is also the minimum size of the arrays
windowDimA
,paddingA
, andstrideA
in order to be able to hold the results.mode
Output. Enumerant to specify the pooling mode.
maxpoolingNanOpt
Input. Enumerant to specify the Nan propagation mode.
nbDims
Output. Actual dimension of the pooling descriptor.
windowDimA
Output. Array of dimension of at least
nbDimsRequested
that will be filled with the window parameters from the provided pooling descriptor.paddingA
Output. Array of dimension of at least
nbDimsRequested
that will be filled with the padding parameters from the provided pooling descriptor.strideA
Output. Array of dimension at least
nbDimsRequested
that will be filled with the stride parameters from the provided pooling descriptor.
Returns
CUDNN_STATUS_SUCCESS
The object was queried successfully.
CUDNN_STATUS_NOT_SUPPORTED
The parameter
nbDimsRequested
is greater thanCUDNN_DIM_MAX
.
cudnnGetPoolingNdForwardOutputDim()
This function provides the output dimensions of a tensor after PoolingNd
has been applied.
cudnnStatus_t cudnnGetPoolingNdForwardOutputDim( const cudnnPoolingDescriptor_t poolingDesc, const cudnnTensorDescriptor_t inputDesc, int nbDims, int outDimA[])
Each dimension of the (nbDims2)D
images of the output tensor is computed as follows:
outputDim = 1 + (inputDim + 2*padding  windowDim)/poolingStride;
Parameters
poolingDesc
Input. Handle to a previously initialized pooling descriptor.
inputDesc
Input. Handle to the previously initialized input tensor descriptor.
nbDims
Input. Number of dimensions in which pooling is to be applied.
outDimA
Output. Array of
nbDims
output dimensions.
Returns
CUDNN_STATUS_SUCCESS
The function launched successfully.
CUDNN_STATUS_BAD_PARAM
At least one of the following conditions are met:
poolingDesc
has not been initialized.The value of
nbDims
is inconsistent with the dimensionality ofpoolingDesc
andinputDesc
.
cudnnGetReduceTensorDescriptor()
This function queries a previously initialized reduce tensor descriptor object.
cudnnStatus_t cudnnGetReduceTensorDescriptor( const cudnnReduceTensorDescriptor_t reduceTensorDesc, cudnnReduceTensorOp_t *reduceTensorOp, cudnnDataType_t *reduceTensorCompType, cudnnNanPropagation_t *reduceTensorNanOpt, cudnnReduceTensorIndices_t *reduceTensorIndices, cudnnIndicesType_t *reduceTensorIndicesType)
Parameters
reduceTensorDesc
Input. Pointer to a previously initialized reduce tensor descriptor object.
reduceTensorOp
Output. Enumerant to specify the reduced tensor operation.
reduceTensorCompType
Output. Enumerant to specify the computation datatype of the reduction.
reduceTensorNanOpt
Input. Enumerant to specify the Nan propagation mode.
reduceTensorIndices
Output. Enumerant to specify the reduced tensor indices.
reduceTensorIndicesType
Output. Enumerant to specify the reduced tensor indices type.
Returns
CUDNN_STATUS_SUCCESS
The object was queried successfully.
CUDNN_STATUS_BAD_PARAM
reduceTensorDesc
isNULL
.
cudnnGetReductionIndicesSize()
This is a helper function to return the minimum size of the index space to be passed to the reduction given the input and output tensors.
cudnnStatus_t cudnnGetReductionIndicesSize( cudnnHandle_t handle, const cudnnReduceTensorDescriptor_t reduceDesc, const cudnnTensorDescriptor_t aDesc, const cudnnTensorDescriptor_t cDesc, size_t *sizeInBytes)
Parameters
handle
Input. Handle to a previously created cuDNN library descriptor.
reduceDesc
Input. Pointer to a previously initialized reduce tensor descriptor object.
aDesc
Input. Pointer to the input tensor descriptor.
cDesc
Input. Pointer to the output tensor descriptor.
sizeInBytes
Output. Minimum size of the index space to be passed to the reduction.
Returns
CUDNN_STATUS_SUCCESS
The index space size is returned successfully.
cudnnGetReductionWorkspaceSize()
This is a helper function to return the minimum size of the workspace to be passed to the reduction given the input and output tensors.
cudnnStatus_t cudnnGetReductionWorkspaceSize( cudnnHandle_t handle, const cudnnReduceTensorDescriptor_t reduceDesc, const cudnnTensorDescriptor_t aDesc, const cudnnTensorDescriptor_t cDesc, size_t *sizeInBytes)
Parameters
handle
Input. Handle to a previously created cuDNN library descriptor.
reduceDesc
Input. Pointer to a previously initialized reduce tensor descriptor object.
aDesc
Input. Pointer to the input tensor descriptor.
cDesc
Input. Pointer to the output tensor descriptor.
sizeInBytes
Output. Minimum size of the index space to be passed to the reduction.
Returns
CUDNN_STATUS_SUCCESS
The workspace size is returned successfully.
cudnnGetTensor4dDescriptor()
This function queries the parameters of the previously initialized Tensor4d
descriptor object.
cudnnStatus_t cudnnGetTensor4dDescriptor( const cudnnTensorDescriptor_t tensorDesc, cudnnDataType_t *dataType, int *n, int *c, int *h, int *w, int *nStride, int *cStride, int *hStride, int *wStride)
Parameters
tensorDesc
Input. Handle to a previously initialized tensor descriptor.
datatype
Output. Data type.
n
Output. Number of images.
c
Output. Number of feature maps per image.
h
Output. Height of each feature map.
w
Output. Width of each feature map.
nStride
Output. Stride between two consecutive images.
cStride
Output. Stride between two consecutive feature maps.
hStride
Output. Stride between two consecutive rows.
wStride
Output. Stride between two consecutive columns.
Returns
CUDNN_STATUS_SUCCESS
The operation succeeded.
cudnnGetTensorNdDescriptor()
This function retrieves values stored in a previously initialized TensorNd
descriptor object.
cudnnStatus_t cudnnGetTensorNdDescriptor( const cudnnTensorDescriptor_t tensorDesc, int nbDimsRequested, cudnnDataType_t *dataType, int *nbDims, int dimA[], int strideA[])
Parameters
tensorDesc
Input. Handle to a previously initialized tensor descriptor.
nbDimsRequested
Input. Number of dimensions to extract from a given tensor descriptor. It is also the minimum size of the arrays
dimA
andstrideA
. If this number is greater than the resultingnbDims[0]
, onlynbDims[0]
dimensions will be returned.datatype
Output. Data type.
nbDims
Output. Actual number of dimensions of the tensor will be returned in
nbDims[0]
.dimA
Output. Array of dimensions of at least
nbDimsRequested
that will be filled with the dimensions from the provided tensor descriptor.strideA
Output. Array of dimensions of at least
nbDimsRequested
that will be filled with the strides from the provided tensor descriptor.
Returns
CUDNN_STATUS_SUCCESS
The results were returned successfully.
CUDNN_STATUS_BAD_PARAM
Either
tensorDesc
ornbDims
pointer isNULL
.
cudnnGetTensorSizeInBytes()
This function returns the size of the tensor in memory in respect to the given descriptor. This function can be used to know the amount of GPU memory to be allocated to hold that tensor.
cudnnStatus_t cudnnGetTensorSizeInBytes( const cudnnTensorDescriptor_t tensorDesc, size_t *size)
Parameters
tensorDesc
Input. Handle to a previously initialized tensor descriptor.
size
Output. Size in bytes needed to hold the tensor in GPU memory.
Returns
CUDNN_STATUS_SUCCESS
The results were returned successfully.
cudnnGetTensorTransformDescriptor()
This function has been deprecated in cuDNN 9.0.
This function returns the values stored in a previously initialized tensor transform descriptor.
cudnnStatus_t cudnnGetTensorTransformDescriptor( cudnnTensorTransformDescriptor_t transformDesc, uint32_t nbDimsRequested, cudnnTensorFormat_t *destFormat, int32_t padBeforeA[], int32_t padAfterA[], uint32_t foldA[], cudnnFoldingDirection_t *direction);
Parameters
transformDesc
Input. A previously initialized tensor transform descriptor.
nbDimsRequested
Input. The number of dimensions to consider. For more information, refer to Tensor Descriptor.
destFormat
Output. The transform format that will be returned.
padBeforeA[]
Output. An array filled with the amount of padding to add before each dimension. The dimension of this
padBeforeA[]
parameter is equal tonbDimsRequested
.padAfterA[]
Output. An array filled with the amount of padding to add after each dimension. The dimension of this
padBeforeA[]
parameter is equal tonbDimsRequested
.foldA[]
Output. An array that was filled with the folding parameters for each spatial dimension. The dimension of this
foldA[]
array isnbDimsRequested2
.direction
Output. The setting that selects folding or unfolding. For more information, refer to cudnnFoldingDirection_t.
Returns
CUDNN_STATUS_SUCCESS
The results were obtained successfully.
CUDNN_STATUS_BAD_PARAM
If
transformDesc
isNULL
or ifnbDimsRequested
is less than3
or greater thanCUDNN_DIM_MAX
.
cudnnInitTransformDest()
This function has been deprecated in cuDNN 9.0.
This function initializes and returns a destination tensor descriptor destDesc
for tensor transform operations. The initialization is done with the desired parameters described in the transform descriptor cudnnTensorDescriptor_t.
cudnnStatus_t cudnnInitTransformDest( const cudnnTensorTransformDescriptor_t transformDesc, const cudnnTensorDescriptor_t srcDesc, cudnnTensorDescriptor_t destDesc, size_t *destSizeInBytes);
The returned Tensor descriptor will be packed.
Parameters
transformDesc
Input. Handle to a previously initialized tensor transform descriptor.
srcDesc
Input. Handle to a previously initialized tensor descriptor.
destDesc
Output. Handle of the tensor descriptor that will be initialized and returned.
destSizeInBytes
Output. A pointer to hold the size, in bytes, of the new tensor.
Returns
CUDNN_STATUS_SUCCESS
The tensor descriptor was initialized successfully.
CUDNN_STATUS_BAD_PARAM
If either
srcDesc
ordestDesc
isNULL
, or if the tensor descriptorsnbDims
is incorrect. For more information, refer to Tensor Descriptor..CUDNN_STATUS_NOT_SUPPORTED
If the provided configuration is not 4D.
CUDNN_STATUS_EXECUTION_FAILED
Function failed to launch on the GPU.
cudnnLRNCrossChannelBackward()
This function performs the backward LRN layer computation.
cudnnStatus_t cudnnLRNCrossChannelBackward( cudnnHandle_t handle, cudnnLRNDescriptor_t normDesc, cudnnLRNMode_t lrnMode, const void *alpha, const cudnnTensorDescriptor_t yDesc, const void *y, const cudnnTensorDescriptor_t dyDesc, const void *dy, const cudnnTensorDescriptor_t xDesc, const void *x, const void *beta, const cudnnTensorDescriptor_t dxDesc, void *dx)
Supported formats are: positivestrided
, NCHW and NHWC for 4D x
and y
, and only NCDHW DHWpacked for 5D (for both x
and y
). Only nonoverlapping 4D and 5D tensors are supported. NCHW layout is preferred for performance.
Parameters
handle
Input. Handle to a previously created cuDNN library descriptor.
normDesc
Input. Handle to a previously initialized LRN parameter descriptor.
lrnMode
Input. LRN layer mode of operation. Currently only
CUDNN_LRN_CROSS_CHANNEL_DIM1
is implemented. Normalization is performed along the tensor’sdimA[1]
.alpha
,beta
Inputs. Pointers to scaling factors (in host memory) used to blend the layer output value with prior value in the destination tensor as follows:
dstValue = alpha[0]*resultValue + beta[0]*priorDstValue
For more information, refer to Scaling Parameters.
yDesc
,y
Inputs. Tensor descriptor and pointer in device memory for the layer’s
y
data.dyDesc
,dy
Inputs. Tensor descriptor and pointer in device memory for the layer’s input cumulative loss differential data
dy
(including error backpropagation).xDesc
,x
Inputs. Tensor descriptor and pointer in device memory for the layer’s
x
data. Note that these values are not modified during backpropagation.dxDesc
,dx
Outputs. Tensor descriptor and pointer in device memory for the layer’s resulting cumulative loss differential data
dx
(including error backpropagation).
Returns
CUDNN_STATUS_SUCCESS
The computation was performed successfully.
CUDNN_STATUS_BAD_PARAM
At least one of the following conditions are met:
One of the tensor pointers
x
,y
isNULL
.Number of input tensor dimensions is 2 or less.
LRN descriptor parameters are outside of their valid ranges.
One of the tensor parameters is 5D but is not in NCDHW DHWpacked format.
CUDNN_STATUS_NOT_SUPPORTED
The function does not support the provided configuration. Some examples include:
Any of the input tensor datatypes is not the same as any of the output tensor datatype.
Any pairwise tensor dimensions mismatch for
x
,y
,dx
, ordy
.Any tensor parameters strides are negative.
cudnnLRNCrossChannelForward()
This function performs the forward LRN layer computation.
cudnnStatus_t cudnnLRNCrossChannelForward( cudnnHandle_t handle, cudnnLRNDescriptor_t normDesc, cudnnLRNMode_t lrnMode, const void *alpha, const cudnnTensorDescriptor_t xDesc, const void *x, const void *beta, const cudnnTensorDescriptor_t yDesc, void *y)
Supported formats are: positivestrided
, NCHW and NHWC for 4D x
and y
, and only NCDHW DHWpacked for 5D (for both x
and y
). Only nonoverlapping 4D and 5D tensors are supported. NCHW layout is preferred for performance.
Parameters
handle
Input. Handle to a previously created cuDNN library descriptor.
normDesc
Input. Handle to a previously initialized LRN parameter descriptor.
lrnMode
Input. LRN layer mode of operation. Currently only
CUDNN_LRN_CROSS_CHANNEL_DIM1
is implemented. Normalization is performed along the tensor’sdimA[1]
.alpha
,beta
Inputs. Pointers to scaling factors (in host memory) used to blend the layer output value with prior value in the destination tensor as follows:
dstValue = alpha[0]*resultValue + beta[0]*priorDstValue
For more information, refer to Scaling Parameters.
xDesc
,yDesc
Inputs. Tensor descriptor objects for the input and output tensors.
x
Input. Input tensor data pointer in device memory.
y
Output. Output tensor data pointer in device memory.
Returns
CUDNN_STATUS_SUCCESS
The computation was performed successfully.
CUDNN_STATUS_BAD_PARAM
At least one of the following conditions are met:
One of the tensor pointers
x
,y
isNULL
.Number of input tensor dimensions is 2 or less.
LRN descriptor parameters are outside of their valid ranges.
One of the tensor parameters is 5D but is not in NCDHW DHWpacked format.
CUDNN_STATUS_NOT_SUPPORTED
The function does not support the provided configuration. Some examples include:
Any of the input tensor datatypes is not the same as any of the output tensor datatype.
x
andy
tensor dimensions mismatch.Any tensor parameters strides are negative.
cudnnNormalizationBackward()
This function has been deprecated in cuDNN 9.0.
This function performs backward normalization layer computation that is specified by mode. Perchannel normalization layer is based on the Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift paper.
cudnnStatus_t cudnnNormalizationBackward(cudnnHandle_t handle, cudnnNormMode_t mode, cudnnNormOps_t normOps, cudnnNormAlgo_t algo, const void *alphaDataDiff, const void *betaDataDiff, const void *alphaParamDiff, const void *betaParamDiff, const cudnnTensorDescriptor_t xDesc, const void *xData, const cudnnTensorDescriptor_t yDesc, const void *yData, const cudnnTensorDescriptor_t dyDesc, const void *dyData, const cudnnTensorDescriptor_t dzDesc, void *dzData, const cudnnTensorDescriptor_t dxDesc, void *dxData, const cudnnTensorDescriptor_t dNormScaleBiasDesc, const void *normScaleData, const void *normBiasData, void *dNormScaleData, void *dNormBiasData, double epsilon, const cudnnTensorDescriptor_t normMeanVarDesc, const void *savedMean, const void *savedInvVariance, cudnnActivationDescriptor_t activationDesc, void *workSpace, size_t workSpaceSizeInBytes, void *reserveSpace, size_t reserveSpaceSizeInBytes, int groupCnt)
Only 4D and 5D tensors are supported.
The epsilon
value has to be the same during training, backpropagation, and inference. This workspace is not required to be clean. Moreover, the workspace does not have to remain unchanged between the forward and backward pass, as it is not used for passing any information.
This function can accept a *workspace
pointer to the GPU workspace, and workSpaceSizeInBytes
, the size of the workspace, from the user.
The normOps
input can be used to set this function to perform either only the normalization, or normalization followed by activation, or normalization followed by elementwise addition and then activation.
When the tensor layout is NCHW, higher performance can be obtained when HWpacked tensors are used for x
, dy
, dx
.
Higher performance for CUDNN_NORM_PER_CHANNEL
mode can be obtained when the following conditions are true:
All tensors, namely,
x
,y
,dz
,dy
, anddx
must be NHWCfully packed, and must be of the typeCUDNN_DATA_HALF
.The tensor C dimension should be a multiple of 4.
The input parameter mode must be set to
CUDNN_NORM_PER_CHANNEL
.The input parameter algo must be set to
CUDNN_NORM_ALGO_PERSIST
.
Workspace
is notNULL
.
workSpaceSizeInBytes
is equal to or larger than the amount required by cudnnGetNormalizationBackwardWorkspaceSize().
reserveSpaceSizeInBytes
is equal to or larger than the amount required by cudnnGetNormalizationTrainingReserveSpaceSize().The content in
reserveSpace
stored by cudnnNormalizationForwardTraining() must be preserved.
Parameters
handle
Input. Handle to a previously created cuDNN library descriptor. For more information, refer to cudnnHandle_t.
mode
Input. Mode of operation (perchannel or peractivation). For more information, refer to cudnnNormMode_t.
normOps
Input. Mode of postoperative. Currently
CUDNN_NORM_OPS_NORM_ACTIVATION
andCUDNN_NORM_OPS_NORM_ADD_ACTIVATION
are only supported in the NHWC layout. For more information, refer to cudnnNormOps_t. This input can be used to set this function to perform either only the normalization, or normalization followed by activation, or normalization followed by elementwise addition and then activation.algo
Input. Algorithm to be performed. For more information, refer to cudnnNormAlgo_t.
*alphaDataDiff
,*betaDataDiff
Inputs. Pointers to scaling factors (in host memory) used to blend the gradient output
dx
with a prior value in the destination tensor as follows:dstValue = alpha[0]*resultValue + beta[0]*priorDstValue
For more information, refer to Scaling Parameters.
*alphaParamDiff
,*betaParamDiff
Inputs. Pointers to scaling factors (in host memory) used to blend the gradient outputs
dNormScaleData
anddNormBiasData
with prior values in the destination tensor as follows:dstValue = alpha[0]*resultValue + beta[0]*priorDstValue
For more information, refer to Scaling Parameters.
xDesc
,*xData
,yDesc
,*yData
,dyDesc
,*dyData
Inputs. Tensor descriptors and pointers in the device memory for the layer’s
x
data, backpropagated gradient inputdy
, the original forward outputy
data.yDesc
andyData
are not needed ifnormOps
is set toCUDNN_NORM_OPS_NORM
, users may passNULL
. For more information, refer to cudnnTensorDescriptor_t.dzDesc
,dxDesc
Inputs. Tensor descriptors and pointers in the device memory for the computed gradient output
dz
anddx
.dzDesc
is not needed whennormOps
isCUDNN_NORM_OPS_NORM
orCUDNN_NORM_OPS_NORM_ACTIVATION
, users may passNULL
. For more information, refer to cudnnTensorDescriptor_t.*dzData
,*dxData
Outputs. Tensor descriptors and pointers in the device memory for the computed gradient output
dz
anddx
.*dzData
is not needed whennormOps
isCUDNN_NORM_OPS_NORM
orCUDNN_NORM_OPS_NORM_ACTIVATION
, users may passNULL
. For more information, refer to cudnnTensorDescriptor_t.dNormScaleBiasDesc
Input. Shared tensor descriptor for the following six tensors:
normScaleData
,normBiasData
,dNormScaleData
, anddNormBiasData
. The dimensions for this tensor descriptor are dependent on normalization mode.The data type of this tensor descriptor must be
float
for FP16 and FP32 input tensors anddouble
for FP64 input tensors. For more information, refer to cudnnTensorDescriptor_t.*normScaleData
Input. Pointer in the device memory for the normalization scale parameter (in the Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift paper, the quantity scale is referred to as gamma).
*normBiasData
Input. Pointers in the device memory for the normalization bias parameter (in the Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift paper, bias is referred to as beta). This parameter is used only when activation should be performed.
*dNormScaleData
,*dNormBiasData
Outputs. Pointers in the device memory for the gradients of
normScaleData
andnormBiasData
, respectively.epsilon
Input. Epsilon value used in normalization formula. Its value should be equal to or greater than zero. The same epsilon value should be used in forward and backward functions.
normMeanVarDesc
Input. Shared tensor descriptor for the following tensors:
savedMean
andsavedInvVariance
. The dimensions for this tensor descriptor are dependent on normalization mode.The data type of this tensor descriptor must be
float
for FP16 and FP32 input tensors anddouble
for FP64 input tensors. For more information, refer to cudnnTensorDescriptor_t.*savedMean
,*savedInvVariance
Inputs. Optional cache parameters containing saved intermediate results computed during the forward pass. For this to work correctly, the layer’s
x
andnormScaleData
,normBiasData
data has to remain unchanged until this backward function is called. Note that both these parameters can beNULL
but only at the same time. It is recommended to use this cache since the memory overhead is relatively small.activationDesc
Input. Descriptor for the activation operation. When the
normOps
input is set to eitherCUDNN_NORM_OPS_NORM_ACTIVATION
orCUDNN_NORM_OPS_NORM_ADD_ACTIVATION
then this activation is used, otherwise the user may passNULL
.workspace
Input. Pointer to the GPU workspace.
workSpaceSizeInBytes
Input. The size of the workspace. It must be large enough to trigger the fast NHWC semipersistent kernel by this function.
*reserveSpace
Input. Pointer to the GPU workspace for the
reserveSpace
.reserveSpaceSizeInBytes
Input. The size of the
reserveSpace
. It must be equal or larger than the amount required by cudnnGetNormalizationTrainingReserveSpaceSize().groutCnt
Input. Only support 1 for now.
Returns
CUDNN_STATUS_SUCCESS
The computation was performed successfully.
CUDNN_STATUS_NOT_SUPPORTED
The function does not support the provided configuration.
CUDNN_STATUS_BAD_PARAM
At least one of the following conditions are met:
Any of the pointers
alphaDataDiff
,betaDataDiff
,alphaParamDiff
,betaParamDiff
,xData
,dyData
,dxData
,normScaleData
,dNormScaleData
, anddNormBiasData
isNULL
.The number of
xDesc
,yDesc
, ordxDesc
tensor descriptor dimensions is not within the range of [4,5] (only 4D and 5D tensors are supported).dNormScaleBiasDesc
dimensions not 1xCx1x1 for 4D and 1xCx1x1x1 for 5D for perchannel, and are not 1xCxHxW for 4D and 1xCxDxHxW for 5D for peractivation mode.Exactly one of
savedMean
,savedInvVariance
pointers isNULL
.epsilon
value is less than zero.Dimensions or data types mismatch for any pair of
xDesc
,dyDesc
,dxDesc
,dNormScaleBiasDesc
, ornormMeanVarDesc
.
cudnnNormalizationForwardInference()
This function has been deprecated in cuDNN 9.0.
This function performs the forward normalization layer computation for the inference phase. Perchannel normalization layer is based on the Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift paper.
cudnnStatus_t cudnnNormalizationForwardInference(cudnnHandle_t handle, cudnnNormMode_t mode, cudnnNormOps_t normOps, cudnnNormAlgo_t algo, const void *alpha, const void *beta, const cudnnTensorDescriptor_t xDesc, const void *x, const cudnnTensorDescriptor_t normScaleBiasDesc, const void *normScale, const void *normBias, const cudnnTensorDescriptor_t normMeanVarDesc, const void *estimatedMean, const void *estimatedVariance, const cudnnTensorDescriptor_t zDesc, const void *z, cudnnActivationDescriptor_t activationDesc, const cudnnTensorDescriptor_t yDesc, void *y, double epsilon, int groupCnt);
Only 4D and 5D tensors are supported.
The input transformation performed by this function is defined as:
y = beta*y + alpha *[normBias + (normScale * (xestimatedMean)/sqrt(epsilon + estimatedVariance)]
The epsilon
value has to be the same during training, backpropagation, and inference.
For the training phase, refer to cudnnNormalizationForwardTraining().
Higher performance can be obtained when HWpacked tensors are used for all of x
and y
.
Parameters
handle
Input. Handle to a previously created cuDNN library descriptor. For more information, refer to cudnnHandle_t.
mode
Input. Mode of operation (perchannel or peractivation). For more information, refer to cudnnNormMode_t.
normOps
Input. Mode of postoperative. Currently
CUDNN_NORM_OPS_NORM_ACTIVATION
andCUDNN_NORM_OPS_NORM_ADD_ACTIVATION
are only supported in the NHWC layout. For more information, refer to cudnnNormOps_t. This input can be used to set this function to perform either only the normalization, or normalization followed by activation, or normalization followed by elementwise addition and then activation.algo
Input. Algorithm to be performed. For more information, refer to cudnnNormAlgo_t.
*alpha
,*beta
Inputs. Pointers to scaling factors (in host memory) used to blend the layer output value with prior value in the destination tensor as follows:
dstValue = alpha[0]*resultValue + beta[0]*priorDstValue
For more information, refer to Scaling Parameters.
xDesc
,yDesc
Inputs. Handles to the previously initialized tensor descriptors.
*x
Input. Data pointer to GPU memory associated with the tensor descriptor
xDesc
, for the layer’sx
input data.*y
Output. Data pointer to GPU memory associated with the tensor descriptor
yDesc
, for they
output of the normalization layer.zDesc
,*z
Inputs. Tensor descriptors and pointers in device memory for residual addition to the result of the normalization operation, prior to the activation.
zDesc
and*z
are optional and are only used whennormOps
isCUDNN_NORM_OPS_NORM_ADD_ACTIVATION
, otherwise users may passNULL
. When in use,z
should have exactly the same dimension asx
and the final outputy
. For more information, refer to cudnnTensorDescriptor_t.Since
normOps
is only supported forCUDNN_NORM_OPS_NORM
, we can set these toNULL
for now.normScaleBiasDesc
,normScale
,normBias
Inputs. Tensor descriptors and pointers in device memory for the normalization scale and bias parameters (in the Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift paper, bias is referred to as beta and scale as gamma).
normMeanVarDesc
,estimatedMean
,estimatedVariance
Inputs. Mean and variance tensors and their tensor descriptors. The
estimatedMean
andestimatedVariance
inputs, accumulated during the training phase from the cudnnNormalizationForwardTraining() call, should be passed as inputs here.activationDesc
Input. Descriptor for the activation operation. When the
normOps
input is set to eitherCUDNN_NORM_OPS_NORM_ACTIVATION
orCUDNN_NORM_OPS_NORM_ADD_ACTIVATION
then this activation is used, otherwise the user may passNULL
. SincenormOps
is only supported forCUDNN_NORM_OPS_NORM
, we can set these toNULL
for now.epsilon
Input. Epsilon value used in the normalization formula. Its value should be equal to or greater than zero.
groutCnt
Input. Only support 1 for now.
Returns
CUDNN_STATUS_SUCCESS
The computation was performed successfully.
CUDNN_STATUS_NOT_SUPPORTED
A compute or data type other than what is supported was chosen, or an unknown algorithm type was chosen.
CUDNN_STATUS_BAD_PARAM
At least one of the following conditions are met:
One of the pointers
alpha
,beta
,x
,y
,normScale
,normBias
,estimatedMean
, andestimatedInvVariance
isNULL
.The number of
xDesc
oryDesc
tensor descriptor dimensions is not within the range of [4,5] (only 4D and 5D tensors are supported).normScaleBiasDesc
andnormMeanVarDesc
dimensions are not 1xCx1x1 for 4D and 1xCx1x1x1 for 5D for perchannel, and are not 1xCxHxW for 4D and 1xCxDxHxW for 5D for peractivation mode.epsilon
value is less than zero.Dimensions or data types mismatch for
xDesc
andyDesc
.
cudnnNormalizationForwardTraining()
This function has been deprecated in cuDNN 9.0.
This function performs the forward normalization layer computation for the training phase. Depending on mode, different normalization operations will be performed. Perchannel layer is based on the Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift paper.
cudnnStatus_t cudnnNormalizationForwardTraining(cudnnHandle_t handle, cudnnNormMode_t mode, cudnnNormOps_t normOps, cudnnNormAlgo_t algo, const void *alpha, const void *beta, const cudnnTensorDescriptor_t xDesc, const void *xData, const cudnnTensorDescriptor_t normScaleBiasDesc, const void *normScale, const void *normBias, double exponentialAverageFactor, const cudnnTensorDescriptor_t normMeanVarDesc, void *resultRunningMean, void *resultRunningVariance, double epsilon, void *resultSaveMean, void *resultSaveInvVariance, cudnnActivationDescriptor_t activationDesc, const cudnnTensorDescriptor_t zDesc, const void *zData, const cudnnTensorDescriptor_t yDesc, void *yData, void *workspace, size_t workSpaceSizeInBytes, void *reserveSpace, size_t reserveSpaceSizeInBytes, int groupCnt);
Only 4D and 5D tensors are supported.
The epsilon
value has to be the same during training, back propagation, and inference.
For the inference phase, refer to cudnnNormalizationForwardInference().
Higher performance can be obtained when HWpacked tensors are used for both x
and y
.
This API will trigger the new semipersistent NHWC kernel when the following conditions are true:
All tensors, namely,
xData
,yData
must be NHWCfully packed and must be of the typeCUDNN_DATA_HALF
.The tensor C dimension should be a multiple of 4.
The input parameter mode must be set to
CUDNN_NORM_PER_CHANNEL
.The input parameter algo must be set to
CUDNN_NORM_ALGO_PERSIST
.
workspace
is notNULL
.
workSpaceSizeInBytes
is equal to or larger than the amount required by cudnnGetNormalizationForwardTrainingWorkspaceSize().
reserveSpaceSizeInBytes
is equal to or larger than the amount required by cudnnGetNormalizationTrainingReserveSpaceSize().The content in
reserveSpace
stored by cudnnNormalizationForwardTraining() must be preserved.
This workspace
is not required to be clean. Moreover, the workspace
does not have to remain unchanged between the forward and backward pass, as it is not used for passing any information. This extended function can accept a *workspace
pointer to the GPU workspace, and workSpaceSizeInBytes
, the size of the workspace, from the user.
The normOps
input can be used to set this function to perform either only the normalization, or normalization followed by activation, or normalization followed by elementwise addition and then activation.
Only 4D and 5D tensors are supported. The epsilon
value has to be the same during the training, the backpropagation, and the inference.
When the tensor layout is NCHW, higher performance can be obtained when HWpacked tensors are used for xData
, yData
.
Parameters
handle
Input. Handle to a previously created cuDNN library descriptor. For more information, refer to cudnnHandle_t.
mode
Input. Mode of operation (perchannel or peractivation). For more information, refer to cudnnNormMode_t.
normOps
Input. Mode of postoperative. Currently
CUDNN_NORM_OPS_NORM_ACTIVATION
andCUDNN_NORM_OPS_NORM_ADD_ACTIVATION
are only supported in the NHWC layout. For more information, refer to cudnnNormOps_t. This input can be used to set this function to perform either only the normalization, or normalization followed by activation, or normalization followed by elementwise addition and then activation.algo
Input. Algorithm to be performed. For more information, refer to cudnnNormAlgo_t.
*alpha
,*beta
Inputs. Pointers to scaling factors (in host memory) used to blend the layer output value with prior value in the destination tensor as follows:
dstValue = alpha[0]*resultValue + beta[0]*priorDstValue
For more information, refer to Scaling Parameters.
xDesc
,yDesc
Inputs. Handles to the previously initialized tensor descriptors.
*xData
Input. Data pointer to GPU memory associated with the tensor descriptor
xDesc
, for the layer’sx
input data.*yData
Output. Data pointer to GPU memory associated with the tensor descriptor
yDesc
, for they
output of the normalization layer.zDesc
,*zData
Inputs. Tensor descriptors and pointers in device memory for residual addition to the result of the normalization operation, prior to the activation.
zDesc
and*zData
are optional and are only used whennormOps
isCUDNN_NORM_OPS_NORM_ADD_ACTIVATION
, otherwise the user may passNULL
. When in use,z
should have exactly the same dimension asxData
and the final outputyData
. For more information, refer to cudnnTensorDescriptor_t.normScaleBiasDesc
,normScale
,normBias
Inputs. Tensor descriptors and pointers in device memory for the normalization scale and bias parameters (in the Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift paper, bias is referred to as beta and scale as gamma). The dimensions for the tensor descriptor are dependent on the normalization mode.
exponentialAverageFactor
Input. Factor used in the moving average computation as follows:
runningMean = runningMean*(1factor) + newMean*factor
Use a
factor=1/(1+n)
atNth
call to the function to get Cumulative Moving Average (CMA) behavior such that:CMA[n] = (x[1]+...+x[n])/n
This is proved below:
Writing CMA[n+1] = (n*CMA[n]+x[n+1])/(n+1) = ((n+1)*CMA[n]CMA[n])/(n+1) + x[n+1]/(n+1) = CMA[n]*(11/(n+1))+x[n+1]*1/(n+1) = CMA[n]*(1factor) + x(n+1)*factor
normMeanVarDesc
Inputs. Tensor descriptor used for following tensors:
resultRunningMean
,resultRunningVariance
,resultSaveMean
,resultSaveInvVariance
.*resultRunningMean
,*resultRunningVariance
Inputs/Outputs. Pointers to the running mean and running variance data. Both these pointers can be
NULL
but only at the same time. The value stored inresultRunningVariance
(or passed as an input in inference mode) is the sample variance and is the moving average ofvariance[x]
where the variance is computed either over batch or spatial+batch dimensions depending on the mode. If these pointers are notNULL
, the tensors should be initialized to some reasonable values or to0
.epsilon
Input. Epsilon value used in the normalization formula. Its value should be equal to or greater than zero.
*resultSaveMean
,*resultSaveInvVariance
Outputs. Optional cache parameters containing saved intermediate results computed during the forward pass. For this to work correctly, the layer’s
x
andnormScale
,normBias
data has to remain unchanged until this backward function is called. Note that both these parameters can beNULL
but only at the same time. It is recommended to use this cache since the memory overhead is relatively small.activationDesc
Input. The tensor descriptor for the activation operation. When the
normOps
input is set to eitherCUDNN_NORM_OPS_NORM_ACTIVATION
orCUDNN_NORM_OPS_NORM_ADD_ACTIVATION
then this activation is used, otherwise the user may passNULL
.*workspace
,workSpaceSizeInBytes
Inputs.
*workspace
is a pointer to the GPU workspace, andworkSpaceSizeInBytes
is the size of the workspace. When*workspace
is notNULL
and*workSpaceSizeInBytes
is large enough, and the tensor layout is NHWC and the data type configuration is supported, then this function will trigger a semipersistent NHWC kernel for normalization. The workspace is not required to be clean. Also, the workspace does not need to remain unchanged between the forward and backward passes.*reserveSpace
Input. Pointer to the GPU workspace for the
reserveSpace
.reserveSpaceSizeInBytes
Input. The size of the
reserveSpace
. Must be equal or larger than the amount required by cudnnGetNormalizationTrainingReserveSpaceSize().groutCnt
Input. Only support 1 for now.
Returns
CUDNN_STATUS_SUCCESS
The computation was performed successfully.
CUDNN_STATUS_NOT_SUPPORTED
The function does not support the provided configuration.
CUDNN_STATUS_BAD_PARAM
At least one of the following conditions are met:
One of the pointers
alpha
,beta
,xData
,yData
,normScale
, andnormBias
isNULL
.The number of
xDesc
oryDesc
tensor descriptor dimensions is not within the [4,5] range (only 4D and 5D tensors are supported).normScaleBiasDesc
dimensions are not 1xCx1x1 for 4D and 1xCx1x1x1 for 5D for perchannel mode, and are not 1xCxHxW for 4D and 1xCxDxHxW for 5D for peractivation mode.Exactly one of
resultSaveMean
,resultSaveInvVariance
pointers areNULL
.Exactly one of
resultRunningMean
,resultRunningInvVariance
pointers areNULL
.epsilon
value is less than zero.Dimensions or data types mismatch for
xDesc
oryDesc
.
cudnnOpTensor()
This function has been deprecated in cuDNN 9.0.
This function implements the equation C = op(alpha1[0] * A
, alpha2[0] * B) + beta[0] * C
, given the tensors A, B, and C and the scaling factors alpha1
, alpha2
, and beta
. The op to use is indicated by the descriptor cudnnOpTensorDescriptor_t, meaning, the type of opTensorDesc
. Currentlysupported ops are listed by the cudnnOpTensorOp_t enum.
cudnnStatus_t cudnnOpTensor( cudnnHandle_t handle, const cudnnOpTensorDescriptor_t opTensorDesc, const void *alpha1, const cudnnTensorDescriptor_t aDesc, const void *A, const void *alpha2, const cudnnTensorDescriptor_t bDesc, const void *B, const void *beta, const cudnnTensorDescriptor_t cDesc, void *C)
The following restrictions on the input and destination tensors apply:
Each dimension of the input tensor A must match the corresponding dimension of the destination tensor C, and each dimension of the input tensor B must match the corresponding dimension of the destination tensor C or must be equal to 1. In the latter case, the same value from the input tensor B for those dimensions will be used to blend into the C tensor.

Tensor A 
Tensor B 
Destination Tensor C 













































CUDNN_TENSOR_NCHW_VECT_C
is not supported as input tensor format. All tensors up to dimension five (5) are supported. This routine does not support tensor formats beyond these dimensions.
Parameters
handle
Input. Handle to a previously created cuDNN context.
opTensorDesc
Input. Handle to a previously initialized op tensor descriptor.
alpha1
,alpha2
,beta
Inputs. Pointers to scaling factors (in host memory) used to blend the source value with prior value in the destination tensor as follows:
dstValue = alpha[0]*resultValue + beta[0]*priorDstValue
For more information, refer to Scaling Parameters.
aDesc
,bDesc
,cDesc
Inputs. Handle to a previously initialized tensor descriptor.
A
,B
Inputs. Pointer to data of the tensors described by the
aDesc
andbDesc
descriptors, respectively.C
Input/Output. Pointer to data of the tensor described by the
cDesc
descriptor.
Returns
CUDNN_STATUS_SUCCESS
The function executed successfully.
CUDNN_STATUS_NOT_SUPPORTED
The function does not support the provided configuration. Some examples include:
The dimensions of the bias tensor and the output tensor dimensions are above 5.
opTensorCompType
is not set as stated above.
CUDNN_STATUS_BAD_PARAM
The data type of the destination tensor C is unrecognized, or the restrictions on the input and destination tensors, stated above, are not met.
CUDNN_STATUS_EXECUTION_FAILED
The function failed to launch on the GPU.
cudnnOpsVersionCheck()
Crosslibrary version checker. Each sublibrary has a version checker that checks whether its own version matches that of its dependencies.
cudnnStatus_t cudnnOpsVersionCheck(void)
Returns
CUDNN_STATUS_SUCCESS
The version check passed.
CUDNN_STATUS_SUBLIBRARY_VERSION_MISMATCH
The versions are inconsistent.
cudnnPoolingBackward()
This function has been deprecated in cuDNN 9.0.
This function computes the gradient of a pooling operation.
cudnnStatus_t cudnnPoolingBackward( cudnnHandle_t handle, const cudnnPoolingDescriptor_t poolingDesc, const void *alpha, const cudnnTensorDescriptor_t yDesc, const void *y, const cudnnTensorDescriptor_t dyDesc, const void *dy, const cudnnTensorDescriptor_t xDesc, const void *xData, const void *beta, const cudnnTensorDescriptor_t dxDesc, void *dx)
As of cuDNN version 6.0, a deterministic algorithm is implemented for max backwards pooling. This algorithm can be chosen via the pooling mode enum of poolingDesc
. The deterministic algorithm has been measured to be up to 50% slower than the legacy max backwards pooling algorithm, or up to 20% faster, depending upon the use case.
Tensor vectorization is not supported for any tensor descriptor arguments in this function. Best performance is expected when using HWpacked tensors. Only 2 and 3 spatial dimensions are supported.
Parameters
handle
Input. Handle to a previously created cuDNN context.
poolingDesc
Input. Handle to the previously initialized pooling descriptor.
alpha
,beta
Inputs. Pointers to scaling factors (in host memory) used to blend the computation result with prior value in the output layer as follows:
dstValue = alpha[0]*result + beta[0]*priorDstValue
For more information, refer to Scaling Parameters.
yDesc
Input. Handle to the previously initialized input tensor descriptor. Can be
NULL
for avg pooling.y
Input. Data pointer to GPU memory associated with the tensor descriptor
yDesc
. Can beNULL
for avg pooling.dyDesc
Input. Handle to the previously initialized input differential tensor descriptor. Must be of type
FLOAT
,DOUBLE
,HALF
, orBFLOAT16
. For more information, refer to cudnnDataType_t.dy
Input. Data pointer to GPU memory associated with the tensor descriptor
dyData
.xDesc
Input. Handle to the previously initialized output tensor descriptor. Can be
NULL
for avg pooling.x
Input. Data pointer to GPU memory associated with the output tensor descriptor
xDesc
. Can beNULL
for avg pooling.dxDesc
Input. Handle to the previously initialized output differential tensor descriptor. Must be of type
FLOAT
,DOUBLE
,HALF
, orBFLOAT16
. For more information, refer to cudnnDataType_t.dx
Output. Data pointer to GPU memory associated with the output tensor descriptor
dxDesc
.
Returns
CUDNN_STATUS_SUCCESS
The function launched successfully.
CUDNN_STATUS_BAD_PARAM
At least one of the following conditions are met:
The dimensions
n
,c
,h
,w
of theyDesc
anddyDesc
tensors differ.The strides
nStride
,cStride
,hStride
,wStride
of theyDesc
anddyDesc
tensors differ.The dimensions
n
,c
,h
,w
of thedxDesc
anddxDesc
tensors differ.The strides
nStride
,cStride
,hStride
,wStride
of thexDesc
anddxDesc
tensors differ.The datatype of the four tensors differ.
CUDNN_STATUS_NOT_SUPPORTED
The function does not support the provided configuration. Some examples includes:
The
wStride
of input tensor or output tensor is not1
.
CUDNN_STATUS_EXECUTION_FAILED
The function failed to launch on the GPU.
cudnnPoolingForward()
This function has been deprecated in cuDNN 9.0.
This function computes pooling of input values (meaning, the maximum or average of several adjacent values) to produce an output with smaller height and/or width.
cudnnStatus_t cudnnPoolingForward( cudnnHandle_t handle, const cudnnPoolingDescriptor_t poolingDesc, const void *alpha, const cudnnTensorDescriptor_t xDesc, const void *x, const void *beta, const cudnnTensorDescriptor_t yDesc, void *y)
All tensor formats are supported, best performance is expected when using HWpacked tensors. Only 2 and 3 spatial dimensions are allowed. Vectorized tensors are only supported if they have 2 spatial dimensions.
The dimensions of the output tensor yDesc
can be smaller or bigger than the dimensions advised by the routine cudnnGetPooling2dForwardOutputDim() or cudnnGetPoolingNdForwardOutputDim().
For average pooling, the compute type is float
even for integer input and output data type. Output round is nearesteven and clamp to the most negative or most positive value of type if out of range.
Parameters
handle
Input. Handle to a previously created cuDNN context.
poolingDesc
Input. Handle to a previously initialized pooling descriptor.
alpha
,beta
Inputs. Pointers to scaling factors (in host memory) used to blend the computation result with prior value in the output layer as follows:
dstValue = alpha[0]*result + beta[0]*priorDstValue
For more information, refer to Scaling Parameters.
xDesc
Input. Handle to the previously initialized input tensor descriptor. Must be of type
FLOAT
,DOUBLE
,HALF
,INT8
,INT8x4
,INT8x32
, orBFLOAT16
. For more information, refer to cudnnDataType_t.x
Input. Data pointer to GPU memory associated with the tensor descriptor
xDesc
.yDesc
Input. Handle to the previously initialized output tensor descriptor. Must be of type
FLOAT
,DOUBLE
,HALF
,INT8
,INT8x4
,INT8x32
, orBFLOAT16
. For more information, refer to cudnnDataType_t.y
Output. Data pointer to GPU memory associated with the output tensor descriptor
yDesc
.
Returns
CUDNN_STATUS_SUCCESS
The function launched successfully.
CUDNN_STATUS_BAD_PARAM
At least one of the following conditions are met:
The dimensions
n
,c
of the input tensor and output tensors differ.The datatype of the input tensor and output tensors differs.
CUDNN_STATUS_NOT_SUPPORTED
The function does not support the provided configuration.
CUDNN_STATUS_EXECUTION_FAILED
The function failed to launch on the GPU.
cudnnReduceTensor()
This function has been deprecated in cuDNN 9.0.
This function reduces tensor A by implementing the equation C = alpha * reduce op ( A ) + beta * C
, given tensors A and C and scaling factors alpha
and beta
. The reduction op to use is indicated by the descriptor reduceTensorDesc
. Currentlysupported ops are listed by the cudnnReduceTensorOp_t enum.
cudnnStatus_t cudnnReduceTensor( cudnnHandle_t handle, const cudnnReduceTensorDescriptor_t reduceTensorDesc, void *indices, size_t indicesSizeInBytes, void *workspace, size_t workspaceSizeInBytes, const void *alpha, const cudnnTensorDescriptor_t aDesc, const void *A, const void *beta, const cudnnTensorDescriptor_t cDesc, void *C)
Each dimension of the output tensor C must match the corresponding dimension of the input tensor A or must be equal to 1
. The dimensions equal to 1
indicate the dimensions of A to be reduced.
The implementation will generate indices for the min
and max
ops only, as indicated by the cudnnReduceTensorIndices_t enum of the reduceTensorDesc
. Requesting indices for the other reduction ops results in an error. The data type of the indices is indicated by the cudnnIndicesType_t enum; currently only the 32bit (unsigned int) type is supported.
The indices returned by the implementation are not absolute indices but relative to the dimensions being reduced. The indices are also flattened, meaning, not coordinate tuples.
The data types of the tensors A and C must match if of type double
. In this case, alpha
and beta
and the computation enum of reduceTensorDesc
are all assumed to be of type double
.
The HALF
and INT8
data types may be mixed with the FLOAT
data types. In these cases, the computation enum of reduceTensorDesc
is required to be of type FLOAT
.
Up to dimension 8, all tensor formats are supported. Beyond those dimensions, this routine is not supported.
Parameters
handle
Input. Handle to a previously created cuDNN context.
reduceTensorDesc
Input. Handle to a previously initialized reduce tensor descriptor.
indices
Output. Handle to a previously allocated space for writing indices.
indicesSizeInBytes
Input. Size of the above previously allocated space.
workspace
Input. Handle to a previously allocated space for the reduction implementation.
workspaceSizeInBytes
Input. Size of the above previously allocated space.
alpha
,beta
Inputs. Pointers to scaling factors (in host memory) used to blend the source value with prior value in the destination tensor as follows:
dstValue = alpha[0]*resultValue + beta[0]*priorDstValue
For more information, refer to Scaling Parameters.
aDesc
,cDesc
Inputs. Handle to a previously initialized tensor descriptor.
A
Input. Pointer to data of the tensor described by the
aDesc
descriptor.C
Input/Output. Pointer to data of the tensor described by the
cDesc
descriptor.
Returns
CUDNN_STATUS_SUCCESS
The function executed successfully.
CUDNN_STATUS_NOT_SUPPORTED
The function does not support the provided configuration. Some examples include:
The dimensions of the input tensor and the output tensor are above 8.
reduceTensorCompType
is not set as stated above.
CUDNN_STATUS_BAD_PARAM
The corresponding dimensions of the input and output tensors all match, or the conditions in the above paragraphs are unmet.
CUDNN_INVALID_VALUE
The allocations for the indices or workspace are insufficient.
CUDNN_STATUS_EXECUTION_FAILED
The function failed to launch on the GPU.
cudnnRestoreDropoutDescriptor()
This function restores a dropout descriptor to a previously savedoff state.
cudnnStatus_t cudnnRestoreDropoutDescriptor( cudnnDropoutDescriptor_t dropoutDesc, cudnnHandle_t handle, float dropout, void *states, size_t stateSizeInBytes, unsigned long long seed)
Parameters
dropoutDesc
Input/Output. Previously created dropout descriptor.
handle
Input. Handle to a previously created cuDNN context.
dropout
Input. Probability with which the value from an input tensor is set to 0 when performing dropout.
states
Input. Pointer to GPU memory that holds random number generator states initialized by a prior call to cudnnSetDropoutDescriptor().
stateSizeInBytes
Input. Size in bytes of buffer holding random number generator
states
.seed
Input. Seed used in prior calls to cudnnSetDropoutDescriptor() that initialized
states
buffer. Using a different seed from this has no effect. A change of seed, and subsequent update to random number generator states can be achieved by calling cudnnSetDropoutDescriptor().
Returns
CUDNN_STATUS_SUCCESS
The call was successful.
CUDNN_STATUS_INVALID_VALUE
The
states
buffer size (as indicated instateSizeInBytes
) is too small.
cudnnScaleTensor()
This function has been deprecated in cuDNN 9.0.
This function scales all the elements of a tensor by a given factor.
cudnnStatus_t cudnnScaleTensor( cudnnHandle_t handle, const cudnnTensorDescriptor_t yDesc, void *y, const void *alpha)
Parameters
handle
Input. Handle to a previously created cuDNN context.
yDesc
Input. Handle to a previously initialized tensor descriptor.
y
Input/Output. Pointer to data of the tensor described by the
yDesc
descriptor.alpha
Input. Pointer in Host memory to a single value that all elements of the tensor will be scaled with. For more information, refer to Scaling Parameters.
Returns
CUDNN_STATUS_SUCCESS
The function launched successfully.
CUDNN_STATUS_NOT_SUPPORTED
The function does not support the provided configuration.
CUDNN_STATUS_BAD_PARAM
One of the provided pointers is
NIL
.CUDNN_STATUS_EXECUTION_FAILED
The function failed to launch on the GPU.
cudnnSetActivationDescriptor()
This function initializes a previously created generic activation descriptor object.
cudnnStatus_t cudnnSetActivationDescriptor( cudnnActivationDescriptor_t activationDesc, cudnnActivationMode_t mode, cudnnNanPropagation_t reluNanOpt, double coef)
Parameters
activationDesc
Input/Output. Handle to a previously created activation descriptor.
mode
Input. Enumerant to specify the activation mode.
reluNanOpt
Input. Enumerant to specify the
Nan
propagation mode.coef
Input. Floating point number. When the activation mode (refer to cudnnActivationMode_t) is set to
CUDNN_ACTIVATION_CLIPPED_RELU
, this input specifies the clipping threshold; and when the activation mode is set toCUDNN_ACTIVATION_RELU
, this input specifies the upper bound.
Returns
CUDNN_STATUS_SUCCESS
The object was set successfully.
CUDNN_STATUS_BAD_PARAM
mode
orreluNanOpt
has an invalid enumerant value.
cudnnSetActivationDescriptorSwishBeta()
This function sets the beta parameter of the SWISH activation function to swish_beta
.
cudnnStatus_t cudnnSetActivationDescriptorSwishBeta(cudnnActivationDescriptor_t activationDesc, double swish_beta)
Parameters
activationDesc
Input/Output. Handle to a previously created activation descriptor.
swish_beta
Input. The value to set the SWISH activations’ beta parameter to.
Returns
CUDNN_STATUS_SUCCESS
The value was set successfully.
CUDNN_STATUS_BAD_PARAM
The activation descriptor is a
NULL
pointer.
cudnnSetDropoutDescriptor()
This function initializes a previously created dropout descriptor object. If the states
argument is equal to NULL
, then the random number generator states won’t be initialized, and only the dropout
value will be set. The user is expected not to change the memory pointed at by states
for the duration of the computation.
cudnnStatus_t cudnnSetDropoutDescriptor( cudnnDropoutDescriptor_t dropoutDesc, cudnnHandle_t handle, float dropout, void *states, size_t stateSizeInBytes, unsigned long long seed)
When the states
argument is not NULL
, a cuRAND initialization kernel is invoked by cudnnSetDropoutDescriptor()
. This kernel requires a substantial amount of GPU memory for the stack. Memory is released when the kernel finishes. The CUDNN_STATUS_ALLOC_FAILED
status is returned when no sufficient free memory is available for the GPU stack.
Parameters
dropoutDesc
Input/Output. Previously created dropout descriptor object.
handle
Input. Handle to a previously created cuDNN context.
dropout
Input. The probability with which the value from input is set to zero during the dropout layer.
states
Output. Pointer to userallocated GPU memory that will hold random number generator states.
stateSizeInBytes
Input. Specifies the size in bytes of the provided memory for the states.
seed
Input. Seed used to initialize random number generator states.
Returns
CUDNN_STATUS_SUCCESS
The call was successful.
CUDNN_STATUS_INVALID_VALUE
The
sizeInBytes
argument is less than the value returned by cudnnDropoutGetStatesSize().CUDNN_STATUS_ALLOC_FAILED
The function failed to temporarily extend the GPU stack.
CUDNN_STATUS_EXECUTION_FAILED
The function failed to launch on the GPU.
CUDNN_STATUS_INTERNAL_ERROR
Internally used CUDA functions returned an error status.
cudnnSetFilter4dDescriptor()
This function has been deprecated in cuDNN 9.0.
This function initializes a previously created filter descriptor object into a 4D filter. The layout of the filters must be contiguous in memory.
cudnnStatus_t cudnnSetFilter4dDescriptor( cudnnFilterDescriptor_t filterDesc, cudnnDataType_t dataType, cudnnTensorFormat_t format, int k, int c, int h, int w)
Tensor format CUDNN_TENSOR_NHWC
has limited support in cudnnConvolutionForward(), cudnnConvolutionBackwardData(), and cudnnConvolutionBackwardFilter().
Parameters
filterDesc
Input/Output. Handle to a previously created filter descriptor.
datatype
Input. Data type.
format
Input. Type of the filter layout format. If this input is set to
CUDNN_TENSOR_NCHW
, which is one of the enumerant values allowed by cudnnTensorFormat_t descriptor, then the layout of the filter is in the form ofKCRS
, where:K
represents the number of output feature mapsC
is the number of input feature mapsR
is the number of rows per filterS
is the number of columns per filter
If this input is set to
CUDNN_TENSOR_NHWC
, then the layout of the filter is in the form ofKRSC
.k
Input. Number of output feature maps.
c
Input. Number of input feature maps.
h
Input. Height of each filter.
w
Input. Width of each filter.
Returns
CUDNN_STATUS_SUCCESS
The object was set successfully.
CUDNN_STATUS_BAD_PARAM
At least one of the parameters
k
,c
,h
,w
is negative ordataType
orformat
has an invalid enumerant value.
cudnnSetFilterNdDescriptor()
This function has been deprecated in cuDNN 9.0.
This function initializes a previously created filter descriptor object. The layout of the filters must be contiguous in memory.
cudnnStatus_t cudnnSetFilterNdDescriptor( cudnnFilterDescriptor_t filterDesc, cudnnDataType_t dataType, cudnnTensorFormat_t format, int nbDims, const int filterDimA[])
The tensor format CUDNN_TENSOR_NHWC
has limited support in cudnnConvolutionForward(), cudnnConvolutionBackwardData(), and cudnnConvolutionBackwardFilter().
Parameters
filterDesc
Input/Output. Handle to a previously created filter descriptor.
datatype
Input. Data type.
format
Input.Type of the filter layout format. If this input is set to
CUDNN_TENSOR_NCHW
, which is one of the enumerant values allowed by cudnnTensorFormat_t descriptor, then the layout of the filter is as follows: For
N=4
, a 4D filter descriptor, the filter layout is in the form ofKCRS
: K
represents the number of output feature mapsC
is the number of input feature mapsR
is the number of rows per filterS
is the number of columns per filter
 For
For
N=3
, a 3D filter descriptor, the numberS
(number of columns per filter) is omitted.For
N=5
and greater, the layout of the higher dimensions immediately followsRS
.
On the other hand, if this input is set to
CUDNN_TENSOR_NHWC
, then the layout of the filter is as follows:For
N=4
, a 4D filter descriptor, the filter layout is in the form ofKRSC
.For
N=3
, a 3D filter descriptor, the numberS
(number of columns per filter) is omitted, and the layout ofC
immediately followsR
.For
N=5
and greater, the layout of the higher dimensions are inserted betweenS
andC
.
nbDims
Input. Dimension of the filter.
filterDimA
Input. Array of dimension
nbDims
containing the size of the filter for each dimension.
Returns
CUDNN_STATUS_SUCCESS
The object was set successfully.
CUDNN_STATUS_BAD_PARAM
At least one of the elements of the array
filterDimA
is negative ordataType
orformat
has an invalid enumerant value.CUDNN_STATUS_NOT_SUPPORTED
The parameter
nbDims
exceedsCUDNN_DIM_MAX
.
cudnnSetLRNDescriptor()
This function initializes a previously created LRN descriptor object.
cudnnStatus_t cudnnSetLRNDescriptor( cudnnLRNDescriptor_t normDesc, unsigned lrnN, double lrnAlpha, double lrnBeta, double lrnK)Note
Macros
CUDNN_LRN_MIN_N
,CUDNN_LRN_MAX_N
,CUDNN_LRN_MIN_K
,CUDNN_LRN_MIN_BETA
defined incudnn.h
specify valid ranges for parameters.Values of double parameters will be cast down to the tensor
datatype
during computation.
Parameters
normDesc
Output. Handle to a previously created LRN descriptor.
lrnN
Input. Normalization window width in elements. The LRN layer uses a window
[centerlookBehind, center+lookAhead]
, wherelookBehind = floor( (lrnN1)/2 )
,lookAhead = lrnNlookBehind1
. So forn=10
, the window is[k4...k...k+5]
with a total of 10 samples. For theDivisiveNormalization
layer, the window has the same extent as above in all spatial dimensions (dimA[2]
,dimA[3]
,dimA[4]
). By default,lrnN
is set to 5 in cudnnCreateLRNDescriptor().lrnAlpha
Input. Value of the alpha variance scaling parameter in the normalization formula. Inside the library code, this value is divided by the window width for LRN and by
(window width)^#spatialDimensions
forDivisiveNormalization
. By default, this value is set to 1e4 in cudnnCreateLRNDescriptor().lrnBeta
Input. Value of the beta power parameter in the normalization formula. By default, this value is set to
0.75
in cudnnCreateLRNDescriptor().lrnK
Input. Value of the
k
parameter in the normalization formula. By default, this value is set to2.0
.
Returns
CUDNN_STATUS_SUCCESS
The object was set successfully.
CUDNN_STATUS_BAD_PARAM
One of the input parameters was out of valid range as described above.
cudnnSetOpTensorDescriptor()
This function initializes a tensor pointwise math descriptor.
cudnnStatus_t cudnnSetOpTensorDescriptor( cudnnOpTensorDescriptor_t opTensorDesc, cudnnOpTensorOp_t opTensorOp, cudnnDataType_t opTensorCompType, cudnnNanPropagation_t opTensorNanOpt)
Parameters
opTensorDesc
Output. Pointer to the structure holding the description of the tensor pointwise math descriptor.
opTensorOp
Input. Tensor pointwise math operation for this tensor pointwise math descriptor.
opTensorCompType
Input. Computation datatype for this tensor pointwise math descriptor.
opTensorNanOpt
Input. NAN propagation policy.
Returns
CUDNN_STATUS_SUCCESS
The function returned successfully.
CUDNN_STATUS_BAD_PARAM
At least one of the input parameters passed is invalid.
cudnnSetPooling2dDescriptor()
This function initializes a previously created generic pooling descriptor object into a 2D description.
cudnnStatus_t cudnnSetPooling2dDescriptor( cudnnPoolingDescriptor_t poolingDesc, cudnnPoolingMode_t mode, cudnnNanPropagation_t maxpoolingNanOpt, int windowHeight, int windowWidth, int verticalPadding, int horizontalPadding, int verticalStride, int horizontalStride)
Parameters
poolingDesc
Input/Output. Handle to a previously created pooling descriptor.
mode
Input. Enumerant to specify the pooling mode.
maxpoolingNanOpt
Input. Enumerant to specify the Nan propagation mode.
windowHeight
Input. Height of the pooling window.
windowWidth
Input. Width of the pooling window.
verticalPadding
Input. Size of vertical padding.
horizontalPadding
Input. Size of horizontal padding
verticalStride
Input. Pooling vertical stride.
horizontalStride
Input. Pooling horizontal stride.
Returns
CUDNN_STATUS_SUCCESS
The object was set successfully.
CUDNN_STATUS_BAD_PARAM
At least one of the parameters
windowHeight
,windowWidth
,verticalStride
,horizontalStride
is negative or mode ormaxpoolingNanOpt
has an invalid enumerate value.
cudnnSetPoolingNdDescriptor()
This function initializes a previously created generic pooling descriptor object.
cudnnStatus_t cudnnSetPoolingNdDescriptor( cudnnPoolingDescriptor_t poolingDesc, const cudnnPoolingMode_t mode, const cudnnNanPropagation_t maxpoolingNanOpt, int nbDims, const int windowDimA[], const int paddingA[], const int strideA[])
Parameters
poolingDesc
Input/Output. Handle to a previously created pooling descriptor.
mode
Input. Enumerant to specify the pooling mode.
maxpoolingNanOpt
Input. Enumerant to specify the Nan propagation mode.
nbDims
Input. Dimension of the pooling operation. Must be greater than zero.
windowDimA
Input. Array of dimension
nbDims
containing the window size for each dimension. The value of array elements must be greater than zero.paddingA
Input. Array of dimension
nbDims
containing the padding size for each dimension. Negative padding is allowed.strideA
Input. Array of dimension
nbDims
containing the striding size for each dimension. The value of array elements must be greater than zero (meaning, negative striding size is not allowed).
Returns
CUDNN_STATUS_SUCCESS
The object was initialized successfully.
CUDNN_STATUS_NOT_SUPPORTED
If (
nbDims > CUDNN_DIM_MAX2
).CUDNN_STATUS_BAD_PARAM
Either
nbDims
, or at least one of the elements of the arrayswindowDimA
orstrideA
is negative, or mode ormaxpoolingNanOpt
has an invalid enumerate value.
cudnnSetReduceTensorDescriptor()
This function initializes a previously created reduce tensor descriptor object.
cudnnStatus_t cudnnSetReduceTensorDescriptor( cudnnReduceTensorDescriptor_t reduceTensorDesc, cudnnReduceTensorOp_t reduceTensorOp, cudnnDataType_t reduceTensorCompType, cudnnNanPropagation_t reduceTensorNanOpt, cudnnReduceTensorIndices_t reduceTensorIndices, cudnnIndicesType_t reduceTensorIndicesType)
Parameters
reduceTensorDesc
Input/Output. Handle to a previously created reduce tensor descriptor.
reduceTensorOp
Input. Enumerant to specify the reduced tensor operation.
reduceTensorCompType
Input. Enumerant to specify the computation datatype of the reduction.
reduceTensorNanOpt
Input. Enumerant to specify the Nan propagation mode.
reduceTensorIndices
Input. Enumerant to specify the reduced tensor indices.
reduceTensorIndicesType
Input. Enumerant to specify the reduced tensor indices type.
Returns
CUDNN_STATUS_SUCCESS
The object was set successfully.
CUDNN_STATUS_BAD_PARAM
reduceTensorDesc
isNULL
(reduceTensorOp
,reduceTensorCompType
,reduceTensorNanOpt
,reduceTensorIndices
orreduceTensorIndicesType
has an invalid enumerant value).
cudnnSetSpatialTransformerNdDescriptor()
This function initializes a previously created generic spatial transformer descriptor object.
cudnnStatus_t cudnnSetSpatialTransformerNdDescriptor( cudnnSpatialTransformerDescriptor_t stDesc, cudnnSamplerType_t samplerType, cudnnDataType_t dataType, const int nbDims, const int dimA[])
Parameters
stDesc
Input/Output. Previously created spatial transformer descriptor object.
samplerType
Input. Enumerant to specify the sampler type.
dataType
Input. Data type.
nbDims
Input. Dimension of the transformed tensor.
dimA
Input. Array of dimension
nbDims
containing the size of the transformed tensor for every dimension.
Returns
CUDNN_STATUS_SUCCESS
The call was successful.
CUDNN_STATUS_BAD_PARAM
At least one of the following conditions are met:
Either
stDesc
ordimA
isNULL
.Either
dataType
orsamplerType
has an invalid enumerant value.
cudnnSetTensor()
This function sets all the elements of a tensor to a given value.
cudnnStatus_t cudnnSetTensor( cudnnHandle_t handle, const cudnnTensorDescriptor_t yDesc, void *y, const void *valuePtr)
Parameters
handle
Input. Handle to a previously created cuDNN context.
yDesc
Input. Handle to a previously initialized tensor descriptor.
y
Input/Output. Pointer to data of the tensor described by the
yDesc
descriptor.valuePtr
Input. Pointer in host memory to a single value. All elements of the
y
tensor will be set tovalue[0]
. The data type of the element invalue[0]
has to match the data type of tensory
.
Returns
CUDNN_STATUS_SUCCESS
The function launched successfully.
CUDNN_STATUS_NOT_SUPPORTED
The function does not support the provided configuration.
CUDNN_STATUS_BAD_PARAM
One of the provided pointers is
NIL
.CUDNN_STATUS_EXECUTION_FAILED
The function failed to launch on the GPU.
cudnnSetTensor4dDescriptor()
This function initializes a previously created generic tensor descriptor object into a 4D tensor. The strides of the four dimensions are inferred from the format parameter and set in such a way that the data is contiguous in memory with no padding between dimensions.
cudnnStatus_t cudnnSetTensor4dDescriptor( cudnnTensorDescriptor_t tensorDesc, cudnnTensorFormat_t format, cudnnDataType_t dataType, int n, int c, int h, int w)
The total size of a tensor including the potential padding between dimensions is limited to 2 Gigaelements of type datatype
.
Parameters
tensorDesc
Input/Output. Handle to a previously created tensor descriptor.
format
Input. Type of format.
datatype
Input. Data type.
n
Input. Number of images.
c
Input. Number of feature maps per image.
h
Input. Height of each feature map.
w
Input. Width of each feature map.
Returns
CUDNN_STATUS_SUCCESS
The object was set successfully.
CUDNN_STATUS_BAD_PARAM
At least one of the parameters
n
,c
,h
,w
was negative orformat
has an invalid enumerant value ordataType
has an invalid enumerant value.CUDNN_STATUS_NOT_SUPPORTED
The total size of the tensor descriptor exceeds the maximum limit of 2 Gigaelements.
cudnnSetTensor4dDescriptorEx()
This function initializes a previously created generic tensor descriptor object into a 4D tensor, similarly to cudnnSetTensor4dDescriptor() but with the strides explicitly passed as parameters. This can be used to lay out the 4D tensor in any order or simply to define gaps between dimensions.
cudnnStatus_t cudnnSetTensor4dDescriptorEx( cudnnTensorDescriptor_t tensorDesc, cudnnDataType_t dataType, int n, int c, int h, int w, int nStride, int cStride, int hStride, int wStride)
At present, some cuDNN routines have limited support for strides. Those routines will return CUDNN_STATUS_NOT_SUPPORTED
if a 4D tensor object with an unsupported stride is used. cudnnTransformTensor() can be used to convert the data to a supported layout.
The total size of a tensor including the potential padding between dimensions is limited to 2 Gigaelements of type datatype
.
Parameters
tensorDesc
Input/Output. Handle to a previously created tensor descriptor.
datatype
Input. Data type.
n
Input. Number of images.
c
Input. Number of feature maps per image.
h
Input. Height of each feature map.
w
Input. Width of each feature map.
nStride
Input. Stride between two consecutive images.
cStride
Input. Stride between two consecutive feature maps.
hStride
Input. Stride between two consecutive rows.
wStride
Input. Stride between two consecutive columns.
Returns
CUDNN_STATUS_SUCCESS
The object was set successfully.
CUDNN_STATUS_BAD_PARAM
At least one of the parameters
n
,c
,h
,w
ornStride
,cStride
,hStride
,wStride
is negative ordataType
has an invalid enumerant value.CUDNN_STATUS_NOT_SUPPORTED
The total size of the tensor descriptor exceeds the maximum limit of 2 Gigaelements.
cudnnSetTensorNdDescriptor()
This function initializes a previously created generic tensor descriptor object.
cudnnStatus_t cudnnSetTensorNdDescriptor( cudnnTensorDescriptor_t tensorDesc, cudnnDataType_t dataType, int nbDims, const int dimA[], const int strideA[])
The total size of a tensor including the potential padding between dimensions is limited to 2 Gigaelements of type datatype
. Tensors are restricted to having at least 4 dimensions, and at most CUDNN_DIM_MAX
dimensions (defined in cudnn.h
). When working with lower dimensional data, it is recommended that the user create a 4D tensor, and set the size along unused dimensions to 1.
Parameters
tensorDesc
Input/Output. Handle to a previously created tensor descriptor.
datatype
Input. Data type.
nbDims
Input. Dimension of the tensor.
Do not use 2 dimensions. Due to historical reasons, the minimum number of dimensions in the filter descriptor is three.
dimA
Input. Array of dimension
nbDims
that contain the size of the tensor for every dimension. The size along unused dimensions should be set to1
. By convention, the ordering of dimensions in the array follows the format [N, C, D, H, W]
, withW
occupying the smallest index in the array.strideA
Input. Array of dimension
nbDims
that contain the stride of the tensor for every dimension. By convention, the ordering of the strides in the array follows the format [Nstride, Cstride, Dstride, Hstride, Wstride]
, withWstride
occupying the smallest index in the array.
Returns
CUDNN_STATUS_SUCCESS
The object was set successfully.
CUDNN_STATUS_BAD_PARAM
At least one of the elements of the array
dimA
was negative or zero, ordataType
has an invalid enumerant value.CUDNN_STATUS_NOT_SUPPORTED
The parameter
nbDims
is outside the range[4, CUDNN_DIM_MAX]
, or the total size of the tensor descriptor exceeds the maximum limit of 2 Gigaelements.
cudnnSetTensorNdDescriptorEx()
This function initializes an Nd
tensor descriptor.
cudnnStatus_t cudnnSetTensorNdDescriptorEx( cudnnTensorDescriptor_t tensorDesc, cudnnTensorFormat_t format, cudnnDataType_t dataType, int nbDims, const int dimA[])
Parameters
tensorDesc
Output. Pointer to the tensor descriptor struct to be initialized.
format
Input. Tensor format.
dataType
Input. Tensor data type.
nbDims
Input. Dimension of the tensor.
Do not use 2 dimensions. Due to historical reasons, the minimum number of dimensions in the filter descriptor is three.
dimA
Input. Array containing the size of each dimension.
Returns
CUDNN_STATUS_SUCCESS
The function was successful.
CUDNN_STATUS_BAD_PARAM
Tensor descriptor was not allocated properly; or input parameters are not set correctly.
CUDNN_STATUS_NOT_SUPPORTED
Dimension size requested is larger than maximum dimension size supported.
cudnnSetTensorTransformDescriptor()
This function has been deprecated in cuDNN 9.0.
This function initializes a tensor transform descriptor that was previously created using the cudnnCreateTensorTransformDescriptor() function.
cudnnStatus_t cudnnSetTensorTransformDescriptor( cudnnTensorTransformDescriptor_t transformDesc, const uint32_t nbDims, const cudnnTensorFormat_t destFormat, const int32_t padBeforeA[], const int32_t padAfterA[], const uint32_t foldA[], const cudnnFoldingDirection_t direction);
Parameters
transformDesc
Output. The tensor transform descriptor to be initialized.
nbDims
Input. The dimensionality of the transform operands. Must be greater than 2. For more information, refer to Tensor Descriptor.
destFormat
Input. The desired destination format.
padBeforeA[]
Input. An array that contains the amount of padding that should be added before each dimension. Set to
NULL
for no padding.padAfterA[]
Input. An array that contains the amount of padding that should be added after each dimension. Set to
NULL
for no padding.foldA[]
Input. An array that contains the folding parameters for each spatial dimension (dimensions 2 and up). Set to
NULL
for no folding.direction
Input. Selects folding or unfolding. This input has no effect when folding parameters are all <= 1. For more information, refer to cudnnFoldingDirection_t.
Returns
CUDNN_STATUS_SUCCESS
The function was launched successfully.
CUDNN_STATUS_BAD_PARAM
The parameter
transformDesc
isNULL
, or ifdirection
is invalid, ornbDims
is <= 2.CUDNN_STATUS_NOT_SUPPORTED
If the dimension size requested is larger than maximum dimension size supported (meaning, one of the
nbDims
is larger thanCUDNN_DIM_MAX
), or ifdestFromat
is something other than NCHW or NHWC.
cudnnSoftmaxBackward()
This routine computes the gradient of the softmax function.
cudnnStatus_t cudnnSoftmaxBackward( cudnnHandle_t handle, cudnnSoftmaxAlgorithm_t algorithm, cudnnSoftmaxMode_t mode, const void *alpha, const cudnnTensorDescriptor_t yDesc, const void *yData, const cudnnTensorDescriptor_t dyDesc, const void *dy, const void *beta, const cudnnTensorDescriptor_t dxDesc, void *dx)
Inplace operation is allowed for this routine; meaning, dy
and dx
pointers may be equal. However, this requires dyDesc
and dxDesc
descriptors to be identical (particularly, the strides of the input and output must match for inplace operation to be allowed).
All tensor formats are supported for all modes and algorithms with 4 and 5D tensors. Performance is expected to be highest with NCHW fullypacked tensors. For more than 5 dimensions tensors must be packed in their spatial dimensions.
Parameters
handle
Input. Handle to a previously created cuDNN context.
algorithm
Input. Enumerant to specify the softmax algorithm.
mode
Input. Enumerant to specify the softmax mode.
alpha
,beta
Inputs. Pointers to scaling factors (in host memory) used to blend the computation result with prior value in the output layer as follows:
dstValue = alpha[0]*result + beta[0]*priorDstValue
For more information, refer to Scaling Parameters.
yDesc
Input. Handle to the previously initialized input tensor descriptor.
y
Input. Data pointer to GPU memory associated with the tensor descriptor
yDesc
.dyDesc
Input. Handle to the previously initialized input differential tensor descriptor.
dy
Input. Data pointer to GPU memory associated with the tensor descriptor
dyData
.dxDesc
Input. Handle to the previously initialized output differential tensor descriptor.
dx
Output. Data pointer to GPU memory associated with the output tensor descriptor
dxDesc
.
Returns
CUDNN_STATUS_SUCCESS
The function launched successfully.
CUDNN_STATUS_NOT_SUPPORTED
The function does not support the provided configuration.
CUDNN_STATUS_BAD_PARAM
At least one of the following conditions are met:
The dimensions
n
,c
,h
,w
of theyDesc
,dyDesc
anddxDesc
tensors differ.The strides
nStride
,cStride
,hStride
,wStride
of theyDesc
anddyDesc
tensors differ.The
datatype
of the three tensors differs.
CUDNN_STATUS_EXECUTION_FAILED
The function failed to launch on the GPU.
cudnnSoftmaxForward()
This routine computes the softmax function.
cudnnStatus_t cudnnSoftmaxForward( cudnnHandle_t handle, cudnnSoftmaxAlgorithm_t algorithm, cudnnSoftmaxMode_t mode, const void *alpha, const cudnnTensorDescriptor_t xDesc, const void *x, const void *beta, const cudnnTensorDescriptor_t yDesc, void *y)
Inplace operation is allowed for this routine; meaning, x
and y
pointers may be equal. However, this requires xDesc
and yDesc
descriptors to be identical (particularly, the strides of the input and output must match for inplace operation to be allowed).
All tensor formats are supported for all modes and algorithms with 4 and 5D tensors. Performance is expected to be highest with NCHW fullypacked tensors. For more than 5 dimensions tensors must be packed in their spatial dimensions.
Parameters
handle
Input. Handle to a previously created cuDNN context.
algorithm
Input. Enumerant to specify the softmax algorithm.
mode
Input. Enumerant to specify the softmax mode.
alpha
,beta
Inputs. Pointers to scaling factors (in host memory) used to blend the computation result with prior value in the output layer as follows:
dstValue = alpha[0]*result + beta[0]*priorDstValue
For more information, refer to Scaling Parameters.
xDesc
Input. Handle to the previously initialized input tensor descriptor.
x
Input. Data pointer to GPU memory associated with the tensor descriptor
xDesc
.yDesc
Input. Handle to the previously initialized output tensor descriptor.
y
Output. Data pointer to GPU memory associated with the output tensor descriptor
yDesc
.
Returns
CUDNN_STATUS_SUCCESS
The function launched successfully.
CUDNN_STATUS_NOT_SUPPORTED
The function does not support the provided configuration.
CUDNN_STATUS_BAD_PARAM
At least one of the following conditions are met:
The dimensions
n
,c
,h
,w
of the input tensor and output tensors differ.The
datatype
of the input tensor and output tensors differ.The parameters
algorithm
ormode
have an invalid enumerant value.
CUDNN_STATUS_EXECUTION_FAILED
The function failed to launch on the GPU.
cudnnSpatialTfGridGeneratorBackward()
This function computes the gradient of a grid generation operation.
cudnnStatus_t cudnnSpatialTfGridGeneratorBackward( cudnnHandle_t handle, const cudnnSpatialTransformerDescriptor_t stDesc, const void *dgrid, void *dtheta)
Only 2D transformation is supported.
Parameters
handle
Input. Handle to a previously created cuDNN context.
stDesc
Input. Previously created spatial transformer descriptor object.
dgrid
Input. Data pointer to GPU memory contains the input differential data.
dtheta
Output. Data pointer to GPU memory contains the output differential data.
Returns
CUDNN_STATUS_SUCCESS
The call was successful.
CUDNN_STATUS_BAD_PARAM
At least one of the following conditions are met:
handle
isNULL
.One of the parameters
dgrid
ordtheta
isNULL
.
CUDNN_STATUS_NOT_SUPPORTED
The function does not support the provided configuration. Some examples incldue:
The dimension of the transformed tensor specified in
stDesc
> 4.
CUDNN_STATUS_EXECUTION_FAILED
The function failed to launch on the GPU.
cudnnSpatialTfGridGeneratorForward()
This function generates a grid of coordinates in the input tensor corresponding to each pixel from the output tensor.
cudnnStatus_t cudnnSpatialTfGridGeneratorForward( cudnnHandle_t handle, const cudnnSpatialTransformerDescriptor_t stDesc, const void *theta, void *grid)
Only 2D transformation is supported.
Parameters
handle
Input. Handle to a previously created cuDNN context.
stDesc
Input. Previously created spatial transformer descriptor object.
theta
Input. Affine transformation matrix. It should be of size
n*2*3
for a 2D transformation, wheren
is the number of images specified instDesc
.grid
Output. A grid of coordinates. It is of size
n*h*w*2
for a 2D transformation, wheren
,h
,w
is specified instDesc
. In the 4th dimension, the first coordinate isx
, and the second coordinate isy
.
Returns
CUDNN_STATUS_SUCCESS
The call was successful.
CUDNN_STATUS_BAD_PARAM
At least one of the following conditions are met:
handle
isNULL
.One of the parameters
grid
ortheta
isNULL
.
CUDNN_STATUS_NOT_SUPPORTED
The function does not support the provided configuration. Some examples include:
The dimension of the transformed tensor specified in
stDesc
> 4.
CUDNN_STATUS_EXECUTION_FAILED
The function failed to launch on the GPU.
cudnnSpatialTfSamplerBackward()
This function computes the gradient of a sampling operation.
cudnnStatus_t cudnnSpatialTfSamplerBackward( cudnnHandle_t handle, const cudnnSpatialTransformerDescriptor_t stDesc, const void *alpha, const cudnnTensorDescriptor_t xDesc, const void *x, const void *beta, const cudnnTensorDescriptor_t dxDesc, void *dx, const void *alphaDgrid, const cudnnTensorDescriptor_t dyDesc, const void *dy, const void *grid, const void *betaDgrid, void *dgrid)
Only 2D transformation is supported.
Parameters
handle
Input. Handle to a previously created cuDNN context.
stDesc
Input. Previously created spatial transformer descriptor object.
alpha
,beta
Inputs. Pointers to scaling factors (in host memory) used to blend the source value with prior value in the destination tensor as follows:
dstValue = alpha[0]*srcValue + beta[0]*priorDstValue
For more information, refer to Scaling Parameters.
xDesc
Input. Handle to the previously initialized input tensor descriptor.
x
Input. Data pointer to GPU memory associated with the tensor descriptor
xDesc
.dxDesc
Input. Handle to the previously initialized output differential tensor descriptor.
dx
Output. Data pointer to GPU memory associated with the output tensor descriptor
dxDesc
.alphaDgrid
,betaDgrid
Inputs. Pointers to scaling factors (in host memory) used to blend the gradient outputs dgrid with prior value in the destination pointer as follows:
dstValue = alpha[0]*srcValue + beta[0]*priorDstValue
For more information, refer to Scaling Parameters.
dyDesc
Input. Handle to the previously initialized input differential tensor descriptor.
dy
Input. Data pointer to GPU memory associated with the tensor descriptor
dyDesc
.grid
Input. A grid of coordinates generated by cudnnSpatialTfGridGeneratorForward().
dgrid
Output. Data pointer to GPU memory contains the output differential data.
Returns
CUDNN_STATUS_SUCCESS
The call was successful.
CUDNN_STATUS_BAD_PARAM
At least one of the following conditions are met:
handle
isNULL
.One of the parameters
x
,dx
,y
,dy
,grid
, anddgrid
isNULL
.The dimension of
dy
differs from those specified instDesc
.
CUDNN_STATUS_NOT_SUPPORTED
The function does not support the provided configuration. Some examples include:
The dimension of transformed tensor > 4.
CUDNN_STATUS_EXECUTION_FAILED
The function failed to launch on the GPU.
cudnnSpatialTfSamplerForward()
This function performs a sampler operation and generates the output tensor using the grid given by the grid generator.
cudnnStatus_t cudnnSpatialTfSamplerForward( cudnnHandle_t handle, const cudnnSpatialTransformerDescriptor_t stDesc, const void *alpha, const cudnnTensorDescriptor_t xDesc, const void *x, const void *grid, const void *beta, cudnnTensorDescriptor_t yDesc, void *y)
Only 2D transformation is supported.
Parameters
handle
Input. Handle to a previously created cuDNN context.
stDesc
Input. Previously created spatial transformer descriptor object.
alpha
,beta
Inputs. Pointers to scaling factors (in host memory) used to blend the source value with prior value in the destination tensor as follows:
dstValue = alpha[0]*srcValue + beta[0]*priorDstValue
For more information, refer to Scaling Parameters.
xDesc
Input. Handle to the previously initialized input tensor descriptor.
x
Input. Data pointer to GPU memory associated with the tensor descriptor
xDesc
.grid
Input. A grid of coordinates generated by cudnnSpatialTfGridGeneratorForward().
yDesc
Input. Handle to the previously initialized output tensor descriptor.
y
Output. Data pointer to GPU memory associated with the output tensor descriptor
yDesc
.
Returns
CUDNN_STATUS_SUCCESS
The call was successful.
CUDNN_STATUS_BAD_PARAM
At least one of the following conditions are met:
handle
isNULL
.One of the parameters
x
,y
, orgrid
isNULL
.The dimension of
dy
differs from those specified instDesc
.
CUDNN_STATUS_NOT_SUPPORTED
The function does not support the provided configuration. Some examples include:
The dimension of transformed tensor > 4.
CUDNN_STATUS_EXECUTION_FAILED
The function failed to launch on the GPU.
cudnnTransformFilter()
This function has been deprecated in cuDNN 9.0.
This function converts the filter between different formats, data types, or dimensions based on the described transformation. It can be used to convert a filter with an unsupported layout format to a filter with a supported layout format.
cudnnStatus_t cudnnTransformFilter( cudnnHandle_t handle, const cudnnTensorTransformDescriptor_t transDesc, const void *alpha, const cudnnFilterDescriptor_t srcDesc, const void *srcData, const void *beta, const cudnnFilterDescriptor_t destDesc, void *destData);
This function copies the scaled data from the input filter srcDesc
to the output tensor destDesc
with a different layout. If the filter descriptors of srcDesc
and destDesc
have different dimensions, they must be consistent with folding and padding amount and order specified in transDesc
.
The srcDesc
and destDesc
tensors must not overlap in any way (meaning, tensors cannot be transformed in place).
When performing a folding transform or a zeropadding transform, the scaling factors (alpha
, beta
) should be set to (1, 0). However, unfolding transforms support any (alpha
, beta
) values. This function is thread safe.
Parameters
handle
Input. Handle to a previously created cuDNN context. For more information, refer to cudnnHandle_t.
transDesc
Input. A descriptor containing the details of the requested filter transformation. For more information, refer to cudnnTensorTransformDescriptor_t.
alpha
,beta
Inputs. Pointers, in the host memory, to the scaling factors used to scale the data in the input tensor
srcDesc
.beta
is used to scale the destination tensor, whilealpha
is used to scale the source tensor. For more information, refer to Scaling Parameters.The beta scaling value is not honored in the folding and zeropadding cases. Unfolding supports any (
alpha
,beta
).srcDesc
,destDesc
Inputs. Handles to the previously initiated filter descriptors.
srcDesc
anddestDesc
must not overlap. For more information, refer to cudnnTensorDescriptor_t.srcData
,destData
Inputs. Pointers, in the host memory, to the data of the tensor described by
srcDesc
anddestDesc
respectively.
Returns
CUDNN_STATUS_SUCCESS
The function launched successfully.
CUDNN_STATUS_BAD_PARAM
A parameter is uninitialized or initialized incorrectly, or the number of dimensions is different between
srcDesc
anddestDesc
.CUDNN_STATUS_NOT_SUPPORTED
The function does not support the provided configuration. Also, in the folding and padding paths, any value other than
A=1
andB=0
will result in aCUDNN_STATUS_NOT_SUPPORTED
.CUDNN_STATUS_EXECUTION_FAILED
The function failed to launch on the GPU.
cudnnTransformTensor()
This function has been deprecated in cuDNN 9.0.
This function copies the scaled data from one tensor to another tensor with a different layout. Those descriptors need to have the same dimensions but not necessarily the same strides. The input and output tensors must not overlap in any way (meaning, tensors cannot be transformed in place). This function can be used to convert a tensor with an unsupported format to a supported one.
cudnnStatus_t cudnnTransformTensor( cudnnHandle_t handle, const void *alpha, const cudnnTensorDescriptor_t xDesc, const void *x, const void *beta, const cudnnTensorDescriptor_t yDesc, void *y)
Parameters
handle
Input. Handle to a previously created cuDNN context.
alpha
,beta
Inputs. Pointers to scaling factors (in host memory) used to blend the source value with prior value in the destination tensor as follows:
dstValue = alpha[0]*srcValue + beta[0]*priorDstValue
For more information, refer to Scaling Parameters.
xDesc
Input. Handle to a previously initialized tensor descriptor. For more information, refer to cudnnTensorDescriptor_t.
x
Input. Pointer to data of the tensor described by the
xDesc
descriptor.yDesc
Input. Handle to a previously initialized tensor descriptor. For more information, refer to cudnnTensorDescriptor_t.
y
Output. Pointer to data of the tensor described by the
yDesc
descriptor.
Returns
CUDNN_STATUS_SUCCESS
The function launched successfully.
CUDNN_STATUS_NOT_SUPPORTED
The function does not support the provided configuration.
CUDNN_STATUS_BAD_PARAM
The dimensions
n
,c
,h
,w
or thedataType
of the two tensor descriptors are different.CUDNN_STATUS_EXECUTION_FAILED
The function failed to launch on the GPU.
cudnnTransformTensorEx()
This function has been deprecated in cuDNN 9.0.
This function converts the tensor layouts between different formats. It can be used to convert a tensor with an unsupported layout format to a tensor with a supported layout format.
cudnnStatus_t cudnnTransformTensorEx( cudnnHandle_t handle, const cudnnTensorTransformDescriptor_t transDesc, const void *alpha, const cudnnTensorDescriptor_t srcDesc, const void *srcData, const void *beta, const cudnnTensorDescriptor_t destDesc, void *destData);
This function copies the scaled data from the input tensor srcDesc
to the output tensor destDesc
with a different layout. The tensor descriptors of srcDesc
and destDesc
should have the same dimensions but need not have the same strides.
The srcDesc
and destDesc
tensors must not overlap in any way (meaning, tensors cannot be transformed in place).
When performing a folding transform or a zeropadding transform, the scaling factors (alpha
, beta
) should be set to (1, 0). However, unfolding transforms support any (alpha
, beta
) values. This function is thread safe.
Parameters
handle
Input. Handle to a previously created cuDNN context. For more information, refer to cudnnHandle_t.
transDesc
Input. A descriptor containing the details of the requested tensor transformation. For more information, refer to cudnnTensorTransformDescriptor_t.
alpha
,beta
Inputs. Pointers, in the host memory, to the scaling factors used to scale the data in the input tensor
srcDesc
.beta
is used to scale the destination tensor, whilealpha
is used to scale the source tensor. For more information, refer to Scaling Parameters.The beta scaling value is not honored in the folding and zeropadding cases. Unfolding supports any (
alpha
,beta
).srcDesc
,destDesc
Inputs. Handles to the previously initiated tensor descriptors.
srcDesc
anddestDesc
must not overlap. For more information, refer to cudnnTensorDescriptor_t.srcData
,destData
Input. Pointers, in the host memory, to the data of the tensor described by
srcDesc
anddestDesc
respectively.
Returns
CUDNN_STATUS_SUCCESS
The function was launched successfully.
CUDNN_STATUS_BAD_PARAM
A parameter is uninitialized or initialized incorrectly, or the number of dimensions is different between
srcDesc
anddestDesc
.CUDNN_STATUS_NOT_SUPPORTED
Function does not support the provided configuration. Also, in the folding and padding paths, any value other than
A=1
andB=0
will result in aCUDNN_STATUS_NOT_SUPPORTED
.CUDNN_STATUS_EXECUTION_FAILED
Function failed to launch on the GPU.