cuDNN Release Notes :: NVIDIA Deep Learning SDK Documentation

cuDNN Release 7.6.2

This is the cuDNN 7.6.2 release notes. This release includes fixes from the previous cuDNN v7.x.x releases as well as the following additional changes.

For previous cuDNN release notes, see the cuDNN Archived Documentation.

Key Features and Enhancements

The following features and enhancements have been added to this release:

Enhanced the performance of 3D deconvolution using cudnnConvolutionBackwardData(), for the following configuration:
- 2x2x2 filter and 2x2x2 convolution stride.
- For FP16 for data input and output, and for accumulation.
- For FP32 for data input and output, and for accumulation.
Enhanced the performance of 3D convolution using cudnnConvolutionForward(), for the following configuration:
- Tensor Core for FP16 for data input and output and FP32 accumulation when CUDNN_TENSOR_OP_MATH is set.
- Tensor Core for FP32 for data input and output and FP32 accumulation when CUDNN_TENSOR_OP_MATH_ALLOW_CONVERSION is set.
Enhanced the functionality of the data type cudnnFusedOps_t by adding the below three enums:
- CUDNN_FUSED_CONV_SCALE_BIAS_ADD_ACTIVATION
- CUDNN_FUSED_SCALE_BIAS_ADD_ACTIVATION_GEN_BITMASK, and
- CUDNN_FUSED_DACTIVATION_FORK_DBATCHNORM

Fixed Issues

The following issues have been fixed in this release:

In cuDNN 7.6.1, on Volta architecture only, there may be a performance degradation when the function cudnnConvolutionBackwardFilter() is used for 3D convolutions with CUDNN_CONVOLUTION_BWD_FILTER_ALGO_1. This is fixed in cuDNN 7.6.2.
In cuDNN 7.6.1, on Turing and Pascal architectures, performance may be degraded for cudnnConvolutionBackwardData(), when used with the following conditions:
- CUDNN_CONVOLUTION_BWD_DATA_ALGO_0 for 3D convolutions
- wDesc, dyDesc and dxDesc are all in NCDHW
- Data type configuration is FLOAT_CONFIG (i.e., single precision data and compute)
This is fixed in cuDNN 7.6.2.
In cuDNN 7.6.1, in some cases the function cudnnConvolutionBackwardData() may fail with “disallowed mismatches” error on Turing (T4) and Volta(V100) architectures, when used with the configuration below:
- Algorithm is CUDNN_CONVOLUTION_BWD_DATA_ALGO_1
- Math type is CUDNN_TENSOR_OP_MATH or CUDNN_TENSOROP_MATH_ALLOW_CONVERSION
- Tensor format for filter is NCHW
- Input and outputs are in FP16 and computation is in FP32
This is fixed in cuDNN 7.6.2.