NVIDIA Optimized Frameworks

Release 24.02

This NVIDIA Optimized Deep Learning Framework, powered by Apache MXNet, container release is intended for use on the NVIDIA Ampere Architecture A100 GPU and on previous generation GPUs like V100 and T4, and with the latest NVIDIA CUDA® 11 and NVIDIA cuDNN 8 libraries. The container image is available on NGC.

Contents of the Optimized Deep Learning Framework container

This container image contains the complete source of the NVIDIA Optimized Deep Learning Framework, which is based upon Apache MXNet version 1.9.1. It is prebuilt and installed to the Python path. The container also includes the following:

Driver Requirements

Release 24.02 is based on CUDA 12.3.2, which requires NVIDIA Driver release 545 or later. However, if you are running on a data center GPU (for example, T4 or any other data center GPU), you can use NVIDIA driver release 470.57 (or later R470), 525.85 (or later R525), 535.86 (or later R535), or 545.23 (or later R545).

The CUDA driver's compatibility package only supports particular drivers. Thus, users should upgrade from all R418, R440, R450, R460, R510, and R520 drivers, which are not forward-compatible with CUDA 12.3. For a complete list of supported drivers, see the CUDA Application Compatibility topic. For more information, see CUDA Compatibility and Upgrades.

GPU Requirements

Release 24.02 supports CUDA compute capability 6.0 and later. This corresponds to GPUs in the NVIDIA Pascal™, NVIDIA Volta™, NVIDIA Turing™, NVIDIA Ampere architecture, NVIDIA Hopper™, and NVIDIA Ada Lovelace architecture families. Specifically, for a list of GPUs that this compute capability corresponds to, see CUDA GPUs. For additional support details, see Deep Learning Frameworks Support Matrix.

Key Features and Enhancements

This Optimized Deep Learning Framework release includes the following key features and enhancements.

  • NVIDIA Optimized Deep Learning Framework, powered by Apache MXNet container image version 24.02 is based on 1.9.1.

    Apache MXNet, formerly an effort undergoing incubation at The Apache Software Foundation (ASF), has graduated to become a full-status active project of the ASF.

Announcements

  • Starting with the 23.06 release, the NVIDIA Optimized Deep Learning Framework containers are no longer tested on Pascal GPU architectures.
  • Deep learning framework containers 19.11 and later include experimental support for Singularity v3.0.
  • The TensorCore example models are no longer provided in the core container (previously shipped in /workspace/nvidia-examples).

    The models can instead be obtained from Github or the NGC. Some python packages, included in previous containers to support these example models, have also been removed. Depending on their specific use cases, you might need to add some packages that were previously preinstalled.

NVIDIA Optimized Deep Learning Framework Container Versions

The following table shows what versions of Ubuntu, CUDA, Apache MXNet, and TensorRT are supported in each of the NVIDIA containers for the Optimized Deep Learning Framework. For older container versions, refer to the Frameworks Support Matrix.
Container Version Ubuntu CUDA Toolkit Apache MXNet TensorRT
24.02 22.04 NVIDIA CUDA 12.3.2 1.9.1 TensorRT 8.6.3
24.01 1.9.1 TensorRT 8.6.1.6
23.12
23.11 NVIDIA CUDA 12.3.0
23.10 NVIDIA CUDA 12.2.2
23.09 NVIDIA CUDA 12.2.1
23.08
23.07 NVIDIA CUDA 12.1.1
23.06
23.05 TensorRT 8.6.1.2
23.04 20.04 NVIDIA CUDA 12.1.0 TensorRT 8.6.1
23.03 TensorRT 8.5.3
23.02 NVIDIA CUDA 12.0.1
23.01 TensorRT 8.5.2.2
22.12 NVIDIA CUDA 11.8.0 TensorRT 8.5.1
22.11
22.10 TensorRT 8.5 EA
22.09
22.08 NVIDIA CUDA 11.7.1 TensorRT 8.4.2.4
22.07 NVIDIA CUDA 11.7 Update 1 Preview TensorRT 8.4.1
22.06 TensorRT 8.2.5
22.05 NVIDIA CUDA 11.7.0 1.9.0.rc6
22.04 NVIDIA CUDA 11.6.2 TensorRT 8.2.4.2
22.03 NVIDIA CUDA 11.6.1 TensorRT 8.2.3
21.09 NVIDIA CUDA 11.4.2 TensorRT 8.0.3
21.08 NVIDIA CUDA 11.4.1 TensorRT 8.0.1
21.07 NVIDIA CUDA 11.4.0 1.9.0.rc3
21.06 NVIDIA CUDA 11.3.1 1.9.0.rc2 TensorRT 7.2.3.4
21.05 NVIDIA CUDA 11.3.0 1.8.0
21.04
21.03 NVIDIA CUDA 11.2.1 TensorRT 7.2.2.3
21.02 NVIDIA CUDA 11.2.0 1.8.0.rc2 7.2.2.3+cuda11.1.0.024
20.12 NVIDIA CUDA 11.1.1 1.8.0.rc0 TensorRT 7.2.2
20.11

18.04

NVIDIA CUDA 11.1.0 TensorRT 7.2.1
20.10 1.7.0
20.09 NVIDIA CUDA 11.0.3 TensorRT 7.1.3
20.08 1.6.0
20.07 NVIDIA CUDA 11.0.194
20.06 NVIDIA CUDA 11.0.167 TensorRT 7.1.2
20.03 NVIDIA CUDA 10.2.89 TensorRT 7.0.0
20.02 1.6.0.rc2

20.01

1.5.1 commit c98184806 from September 4, 2019

19.12

19.11

TensorRT 6.0.1
19.10 NVIDIA CUDA 10.1.243
19.09 1.5.0 commit 006486af3 from August 28, 2019
19.08 1.5.0 commit 75a9e187d from June 27, 2019 TensorRT 5.1.5

Tensor Core Examples

The tensor core examples in GitHub and NGC focus on achieving the best performance and convergence from Volta tensor cores by using the latest deep learning example networks and model scripts for training. Each example model trains with mixed precision Tensor Cores on Volta and NVIDIA Turing, so you can get results much faster than training without tensor cores. This model is tested against each NGC monthly container release to ensure consistent accuracy and performance over time. This container includes the following tensor core examples:

Automatic Mixed Precision

Training deep learning networks is a computationally intensive task. Novel model architectures tend to have an increasing number of layers and parameters, which slows down training. Fortunately, new generations of training hardware and software optimizations make training these new models a feasible task.

Most of the hardware and software training optimization opportunities involve exploiting lower precision, such as FP16, to use the Tensor Cores that are available on new Volta and Turing GPUs. While training in FP16 showed great success in image classification tasks, other more complicated neural networks typically stayed in FP32 due to difficulties in applying the FP16 training guidelines that are needed to ensure proper model training.

Automatic Mixed Precision (AMP) automatically applies the FP16 training guidelines by using FP16 precision where it provides the most benefit, while conservatively keeping in full FP32 precision operations unsafe to do in FP16.

In the container, the NVIDIA Optimized Deep Learning Framework, powered by Apache MXNet AMP tutorial is located in the /opt/mxnet/nvidia-examples/AMP/AMP_tutorial.md directory, shows you how to get started with mixed precision training using AMP for Apache MXNet, by using the SSD network example from GluonCV.

For more information about AMP, see the Training With Mixed Precision Guide.

Known Issues

  • On H100 NVLink systems using 2 GPUs for training, certain communication patterns can trigger a corner-case bug that manifests either as a hang or as an "illegal instruction" exception. A workaround for this case is to set the environment variable NCCL_PROTO=^LL128. This issue will be addressed in an upcoming release.
  • This container has by default the environment variable setting MXNET_CUDNN_AUTOTUNE_DEFAULT=2, which enables the selection of the fastest convolution op implementation from a list of cuDNN-supplied candidates at runtime. This setting may lead in rare model training scripts to long start-up times or hangs, in which case the recommended remedy is to turn off auto-tuning via MXNET_CUDNN_AUTOTUNING_DEFAULT=0.
  • Users that run the pytest unittests found under /opt/mxnet/tests will notice some failing tests because the public download site data.mxnet.io for pre-trained models has moved.
© Copyright 2024, NVIDIA. Last updated on Oct 30, 2024.