Backend API Overview
The NVIDIA CUDA Deep Neural Network (cuDNN) library offers a context-based API that allows for easy multithreading and (optional) interoperability with CUDA streams. This API Reference lists the data types and API functions per sub-library.
The cuDNN version 9 library is reorganized into several sub-libraries. This new library structure separates legacy functionality (an imperative API with a fixed set of operations and fusion patterns in cuDNN version 7 and older) from the graph API (a declarative API, introduced in cuDNN version 8), as well as from the engine implementation. The graph API is now grouped into a single library (libcudnn_graph.so
).
We recommend users to make API calls through the graph API, but the older shim layer (libcudnn.so
) and legacy libraries (libcudnn_cnn.so
, libcudnn_ops.so
, libcudnn_adv.so
) can still be directly used. The engine libraries (libcudnn_engines_precompiled.so
, libcudnn_heuristic.so
, and libcudnn_engines_runtime_compiled.so
) contain necessary internal functionality to support the aforementioned sub-libraries. This library split is done in a way to allow for more flexibility on the user side in loading libraries. The mechanism for more flexible configurability of libraries will be introduced in a later version of cuDNN.
The API is split into the following libraries, each having corresponding header files in the include directory:
This entity contains the API functions related to cuDNN context creation and destruction, a list of valid cuDNN backend descriptor types, a list of valid attributes, a subset of valid attribute values, and a full description of each backend descriptor type and their attributes. In addition, it is responsible for loading the correct engine libraries.
Engine and heuristics libraries (
cudnn_engines_precompiled
,cudnn_engines_runtime_compiled
, andcudnn_heuristics
) are loaded throughdlopen
bycudnn_graph
, and provide implementations for the API incudnn_graph
. The main idea of separating these engines from each other is to, in a future version of cuDNN, provide a way for the user to only load the functionality they need, and thus provide an opportunity to save memory and disk space.
This entity contains the legacy API functions related to tensor descriptor management, tensor utility API functions, and common machine learning algorithms such as batch normalization, softmax, dropout, and so on. This library depends on
cudnn_graph
, and indirectly oncudnn_engines_precompiled
, as kernels for some APIs reside in thecudnn_engines_precompiled
library.
This entity contains all legacy API functions related to convolutional neural networks. This library depends on
cudnn_ops
andcudnn_graph
. Thecudnn_engines_precompiled
library must be present forcudnn_cnn
to correctly load.
This entity contains all other features and algorithms. This includes RNNs, CTC loss, and multihead attention. This library depends on
cudnn_ops
andcudnn_graph
.
Added, Deprecated, and Removed APIs
API Changes for cuDNN 9.2.0
Added APIs:
Added Enums:
CUDNN_STATUS_SUBLIBRARY_LOADING_FAILED
CUDNN_ATTR_ENGINECFG_WORKSPACE_SIZE
CUDNN_ATTR_ENGINECFG_SHARED_MEMORY_USED
Deprecated APIs:
API Changes for cuDNN 9.1.0
Added Enums:
CUDNN_NUMERICAL_NOTE_STRICT_NAN_PROP
CUDNN_POINTWISE_ATAN2
CUDNN_POINTWISE_CBRT
CUDNN_POINTWISE_CLZ
CUDNN_POINTWISE_IS_FINITE
CUDNN_POINTWISE_POPC
CUDNN_POINTWISE_ROUND_NEAREST_AFZ
CUDNN_POINTWISE_ROUND_NEAREST_EVEN
CUDNN_POINTWISE_SHIFT_LEFT
CUDNN_POINTWISE_SHIFT_RIGHT_ARITHMETIC
CUDNN_POINTWISE_SHIFT_RIGHT_LOGICAL
CUDNN_POINTWISE_XOR
CUDNN_POINTWISE_AND
CUDNN_POINTWISE_OR
CUDNN_POINTWISE_NOT
API Changes for cuDNN 9.0.0
Added Environment Variables:
CUDNN_LOGLEVEL_DB
Added Type:
Added APIs:
Added Enums:
CUDNN_STATUS_SUBLIBRARY_VERSION_MISMATCH
CUDNN_STATUS_SERIALIZATION_VERSION_MISMATCH
CUDNN_STATUS_BAD_PARAM_NULL_POINTER
CUDNN_STATUS_BAD_PARAM_MISALIGNED_POINTER
CUDNN_STATUS_BAD_PARAM_NOT_FINALIZED
CUDNN_STATUS_BAD_PARAM_OUT_OF_BOUND
CUDNN_STATUS_BAD_PARAM_SIZE_INSUFFICIENT
CUDNN_STATUS_BAD_PARAM_STREAM_MISMATCH
CUDNN_STATUS_BAD_PARAM_SHAPE_MISMATCH
CUDNN_STATUS_BAD_PARAM_DUPLICATED_ENTRIES
CUDNN_STATUS_BAD_PARAM_ATTRIBUTE_TYPE
CUDNN_STATUS_NOT_SUPPORTED_GRAPH_PATTERN
CUDNN_STATUS_NOT_SUPPORTED_SHAPE
CUDNN_STATUS_NOT_SUPPORTED_DATA_TYPE
CUDNN_STATUS_NOT_SUPPORTED_LAYOUT
CUDNN_STATUS_NOT_SUPPORTED_INCOMPATIBLE_CUDA_DRIVER
CUDNN_STATUS_NOT_SUPPORTED_INCOMPATIBLE_CUDART
CUDNN_STATUS_NOT_SUPPORTED_ARCH_MISMATCH
CUDNN_STATUS_NOT_SUPPORTED_RUNTIME_PREREQUISITE_MISSING
CUDNN_STATUS_NOT_SUPPORTED_SUBLIBRARY_UNAVAILABLE
CUDNN_STATUS_NOT_SUPPORTED_SHARED_MEMORY_INSUFFICIENT
CUDNN_STATUS_NOT_SUPPORTED_PADDING
CUDNN_STATUS_NOT_SUPPORTED_BAD_LAUNCH_PARAM
CUDNN_STATUS_INTERNAL_ERROR_COMPILATION_FAILED
CUDNN_STATUS_INTERNAL_ERROR_UNEXPECTED_VALUE
CUDNN_STATUS_INTERNAL_ERROR_HOST_ALLOCATION_FAILED
CUDNN_STATUS_INTERNAL_ERROR_DEVICE_ALLOCATION_FAILED
CUDNN_STATUS_INTERNAL_ERROR_BAD_LAUNCH_PARAM
CUDNN_STATUS_INTERNAL_ERROR_TEXTURE_CREATION_FAILED
CUDNN_STATUS_EXECUTION_FAILED_CUDA_DRIVER
CUDNN_STATUS_EXECUTION_FAILED_CUBLAS
CUDNN_STATUS_EXECUTION_FAILED_CUDART
CUDNN_STATUS_EXECUTION_FAILED_CURAND
Deprecated Environment Variables:
CUDNN_LOGERR_DBG
CUDNN_LOGWARN_DBG
CUDNN_LOGINFO_DBG
Deprecated APIs:
Deprecated Enums:
CUDNN_ATTR_OPERATION_RESAMPLE_BWD_ALPHA
CUDNN_ATTR_OPERATION_RESAMPLE_BWD_BETA
CUDNN_ATTR_OPERATION_RESAMPLE_FWD_ALPHA
CUDNN_ATTR_OPERATION_RESAMPLE_FWD_BETA
CUDNN_DATA_INT8x32
CUDNN_DATA_INT8x4
CUDNN_DATA_UINT8x4
CUDNN_STATUS_ALLOC_FAILED
CUDNN_STATUS_ARCH_MISMATCH
CUDNN_STATUS_INVALID_VALUE
CUDNN_STATUS_MAPPING_ERROR
CUDNN_STATUS_RUNTIME_PREREQUISITE_MISSING
CUDNN_STATUS_VERSION_MISMATCH
cudnnBackendAttributeName_t:::CUDNN_ATTR_POINTWISE_NAN_PROPAGATION
cudnnBackendAttributeType_t::CUDNN_TYPE_NAN_PROPOGATION
CUDNN_TYPE_NAN_PROPOGATION
CUDNN_ATTR_POINTWISE_NAN_PROPAGATION
CUDNN_KNOB_TYPE_SPLIT_K
CUDNN_KNOB_TYPE_USE_TEX
CUDNN_KNOB_TYPE_KBLOCK
CUDNN_KNOB_TYPE_LDGA
CUDNN_KNOB_TYPE_LDGB
CUDNN_KNOB_TYPE_CHUNK_K
CUDNN_KNOB_TYPE_SPLIT_H
CUDNN_KNOB_TYPE_WINO_TILE
CUDNN_KNOB_TYPE_CTA_SPLIT_K_MODE
CUDNN_KNOB_TYPE_IDX_MODE
CUDNN_KNOB_TYPE_SLICED
CUDNN_KNOB_TYPE_SPLIT_RS
CUDNN_KNOB_TYPE_SINGLEBUFFER
CUDNN_KNOB_TYPE_LDGC
CUDNN_KNOB_TYPE_TILE_CGA
CUDNN_KNOB_TYPE_NUM_C_PER_BLOCK
Deprecated Types:
Removed APIs:
Functions Deprecated in v8, Removed in v9 |
Replacement Function |
---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Removed Types:
cudnnRNNPaddingMode_t
cudnnAttnQueryMap_t