Release 21.09

Release 21.09 (PDF)

This NVIDIA Optimized Deep Learning Framework, powered by Apache MXNet, container release is intended for use on the NVIDIA Ampere Architecture A100 GPU, as well as on previous generation GPUs like V100 and T4, and with the latest CUDA 11 and cuDNN 8 libraries. The container image is available on NGC.

Contents of the Optimized Deep Learning Framework container

This container image contains the complete source of the NVIDIA Optimized Deep Learning Framework, which is based upon Apache MXNet version 1.9.0.rc6. It is pre-built and installed to the Python path. The container also includes the following:

Driver Requirements

Release 21.09 is based on NVIDIA CUDA 11.4.2, which requires NVIDIA Driver release 470 or later. However, if you are running on Data Center GPUs (formerly Tesla), for example, T4, you may use NVIDIA driver release 418.40 (or later R418), 440.33 (or later R440), 450.51 (or later R450), or 460.27 (or later R460). The CUDA driver's compatibility package only supports particular drivers. For a complete list of supported drivers, see the CUDA Application Compatibility topic. For more information, see CUDA Compatibility and Upgrades and NVIDIA CUDA and Drivers Support.

GPU Requirements

Release 21.09 supports CUDA compute capability 6.0 and higher. This corresponds to GPUs in the NVIDIA Pascal, Volta, Turing, and Ampere Architecture GPU families. Specifically, for a list of GPUs that this compute capability corresponds to, see CUDA GPUs. For additional support details, see Deep Learning Frameworks Support Matrix.

Key Features and Enhancements

This Optimized Deep Learning Framework release includes the following key features and enhancements.

  • NVIDIA Optimized Deep Learning Framework, powered by Apache MXNet container image version 21.09 is based on 1.9.0.rc6. Apache MXNet is an effort undergoing incubation at The Apache Software Foundation (ASF).
  • As of the previous (21.08) release, the implementation of the Convolution and Deconvolution operators is based on a direct use of the cuDNN v8 API, rather than the cuDNN v7 legacy API. This approach can provide enhanced performance by autotuning over an expanded set of conv/deconv ‘engine configs’ as compared to the restricted set of ‘algo numbers’ in prior releases. Users should note that in this 21.09 release:
    • The time spent in autotuning may be longer than seen in 21.08 and earlier releases.
    • The potential issue of the 21.08 release, where a model’s GPU global memory use may have increased due to the new autotuning process and selections, has been corrected.


  • After release 21.09, future releases of the NVIDIA Optimized Deep Learning Framework, powered by Apache MXNet container will be paused for approximately 6 months.
  • Deep learning framework containers 19.11 and later include experimental support for Singularity v3.0.
  • The TensorCore example models are no longer provided in the core container (previously shipped in /workspace/nvidia-examples). Instead they can be obtained from Github or the NVIDIA GPU Cloud (NGC). Some python packages, included in previous containers to support these example models, have also been removed. Depending on their specific use cases, users may need to add some packages that were previously pre-installed.

NVIDIA Optimized Deep Learning Framework Container Versions

The following table shows what versions of Ubuntu, CUDA, Apache MXNet, and TensorRT are supported in each of the NVIDIA containers for the Optimized Deep Learning Framework. For older container versions, refer to the Frameworks Support Matrix.

Container Version Ubuntu CUDA Toolkit Apache MXNet TensorRT
21.09 20.04 NVIDIA CUDA 11.4.2 1.9.0.rc6 TensorRT 8.0.3
21.08 NVIDIA CUDA 11.4.1 TensorRT
21.07 NVIDIA CUDA 11.4.0 1.9.0.rc3
21.06 NVIDIA CUDA 11.3.1 1.9.0.rc2 TensorRT
21.05 NVIDIA CUDA 11.3.0 1.8.0
21.03 NVIDIA CUDA 11.2.1 TensorRT
21.02 NVIDIA CUDA 11.2.0 1.8.0.rc2
20.12 NVIDIA CUDA 11.1.1 1.8.0.rc0 TensorRT 7.2.2


NVIDIA CUDA 11.1.0 TensorRT 7.2.1
20.10 1.7.0
20.09 NVIDIA CUDA 11.0.3 TensorRT 7.1.3
20.08 1.6.0
20.07 NVIDIA CUDA 11.0.194
20.06 NVIDIA CUDA 11.0.167 TensorRT 7.1.2
20.03 NVIDIA CUDA 10.2.89 TensorRT 7.0.0
20.02 1.6.0.rc2


1.5.1 commit c98184806 from September 4, 2019



TensorRT 6.0.1
19.10 NVIDIA CUDA 10.1.243
19.09 1.5.0 commit 006486af3 from August 28, 2019
19.08 1.5.0 commit 75a9e187d from June 27, 2019 TensorRT 5.1.5

Tensor Core Examples

The tensor core examples provided in GitHub and NVIDIA GPU Cloud (NGC) focus on achieving the best performance and convergence from NVIDIA Volta tensor cores by using the latest deep learning example networks and model scripts for training. Each example model trains with mixed precision Tensor Cores on Volta and Turing, therefore you can get results much faster than training without tensor cores. This model is tested against each NGC monthly container release to ensure consistent accuracy and performance over time. This container includes the following tensor core examples.

Automatic Mixed Precision (AMP)

Training deep learning networks is a very computationally intensive task. Novel model architectures tend to have an increasing number of layers and parameters, which slows down training. Fortunately, new generations of training hardware as well as software optimizations make training these new models a feasible task.

Most of the hardware and software training optimization opportunities involve exploiting lower precision like FP16 in order to utilize the Tensor Cores available on new Volta and Turing GPUs. While training in FP16 showed great success in image classification tasks, other more complicated neural networks typically stayed in FP32 due to difficulties in applying the FP16 training guidelines that are needed to ensure proper model training.

That is where AMP (Automatic Mixed Precision) comes into play—it automatically applies the guidelines of FP16 training, using FP16 precision where it provides the most benefit, while conservatively keeping in full FP32 precision operations unsafe to do in FP16.

The NVIDIA Optimized Deep Learning Framework, powered by Apache MXNet AMP tutorial, located in /opt/mxnet/nvidia-examples/AMP/ inside this container, shows how to get started with mixed precision training using AMP for Apache MXNet, using by example the SSD network from GluonCV.

For more information about AMP, see the Training With Mixed Precision Guide.

Known Issues

  • The default setting of the environment variable MXNET_GPU_COPY_NTHREADS=1 in the container may not be optimal for all networks. Networks with a high ratio of parameters and computation, like AlexNet, may achieve greater multi-GPU training speeds with the setting MXNET_GPU_COPY_NTHREADS=2. Users are encouraged to try this setting for their own use case.
© Copyright 2023, NVIDIA. Last updated on Jul 26, 2023.