cuDNN Release Notes v7.6.2

The following features and enhancements have been added to this release:

• Enhanced the performance of 3D deconvolution using cudnnConvolutionBackwardData(), for the following configuration:
  • 2x2x2 filter and 2x2x2 convolution stride.
  • Tensor Core for FP16 for data input and output, and for accumulation.
  • Tensor Core for FP32 for data input and output, and for accumulation.
• Enhanced the performance of 3D convolution using cudnnConvolutionForward(), for the following configuration:
  • Tensor Core for FP16 for data input and output and FP32 accumulation when CUDNN_TENSOR_OP_MATH is set.
  • Tensor Core for FP32 for data input and output and FP32 accumulation when CUDNN_TENSOR_OP_MATH_ALLOW_CONVERSION is set.
• Enhanced the functionality of the data type cudnnFusedOps_t by adding the below three enums:
  • CUDNN_FUSED_CONV_SCALE_BIAS_ADD_ACTIVATION
  • CUDNN_FUSED_SCALE_BIAS_ADD_ACTIVATION_GEN_BITMASK, and
  • CUDNN_FUSED_DACTIVATION_FORK_DBATCHNORM

The following issues have been fixed in this release:

• In cuDNN 7.6.1, on Volta architecture only, there may be a performance degradation when the function cudnnConvolutionBackwardFilter() is used for 3D convolutions with CUDNN_CONVOLUTION_BWD_FILTER_ALGO_1. This is fixed in cuDNN 7.6.2.
• In cuDNN 7.6.1, on Turing and Pascal architectures, performance may be degraded for cudnnConvolutionBackwardData(), when used with the following conditions:
  • CUDNN_CONVOLUTION_BWD_DATA_ALGO_0 for 3D convolutions
  • wDesc, dyDesc and dxDesc are all in NCDHW
  • Data type configuration is FLOAT_CONFIG (i.e., single precision data and compute)

  This is fixed in cuDNN 7.6.2.
  

• In cuDNN 7.6.1, in some cases the function cudnnConvolutionBackwardData() may fail with “disallowed mismatches” error on Turing (T4) and Volta(V100) architectures, when used with the configuration below:
  • Algorithm is CUDNN_CONVOLUTION_BWD_DATA_ALGO_1
  • Math type is CUDNN_TENSOR_OP_MATH or CUDNN_TENSOROP_MATH_ALLOW_CONVERSION
  • Tensor format for filter is NCHW
  • Input and outputs are in FP16 and computation is in FP32

  This is fixed in cuDNN 7.6.2.