The container image for NVIDIA Optimized Deep Learning Framework, powered by Apache MXNet, release 20.11, is available on NGC.
Contents of the Optimized Deep Learning Framework container
This container image contains the complete source of the NVIDIA Optimized Deep Learning Framework, which is based upon Apache MXNet version 1.8.0.rc0. It is pre-built and installed to the Python path. The container also includes the following:
- Ubuntu 18.04 including Python 3.6
- NVIDIA CUDA 11.1.0 including cuBLAS 11.2.1
- NVIDIA cuDNN 8.0.4
- NVIDIA NCCL 2.8.2 (optimized for NVLink™ )
- Amazon Labs Sockeye sequence-to-sequence framework 2.2.3 (for machine translation)
- MLNX_OFED 5.1
- OpenMPI 4.0.5
- Horovod 0.20.2 with enhancements
- Nsight Compute 2020.2.0.18
- Nsight Systems 2020.3.4.32
- GluonCV Toolkit 0.7
- GluonNLP Toolkit 0.8.1
- TensorRT 7.2.1
- DALI 0.27
- Tensor Core optimized example:
- Jupyter and JupyterLab:
Release 20.11 is based on NVIDIA CUDA 11.1.0, which requires NVIDIA Driver release 455 or later. However, if you are running on Tesla (for example, T4 or any other Tesla board), you may use NVIDIA driver release 418.xx, 440.30, or 450.xx. The CUDA driver's compatibility package only supports particular drivers. For a complete list of supported drivers, see the CUDA Application Compatibility topic. For more information, see CUDA Compatibility and Upgrades.
Release 20.11 supports CUDA compute capability 6.0 and higher. This corresponds to GPUs in the NVIDIA Pascal, Volta, Turing, and Ampere Architecture GPU families. Specifically, for a list of GPUs that this compute capability corresponds to, see CUDA GPUs. For additional support details, see Deep Learning Frameworks Support Matrix.
Key Features and Enhancements
- NVIDIA Optimized Deep Learning Framework, powered by Apache MXNet container image version 20.11 is based on 1.8.0.rc0. Apache MXNet is an effort undergoing incubation at The Apache Software Foundation (ASF).
- The environment variable controlling usage of CUDA graphs was changed to align with the upstream Apache MXNet 1.8.0 and is now MXNET_ENABLE_CUDA_GRAPHS (in the previous container releases it was MXNET_EXPERIMENTAL_ENABLE_CUDA_GRAPH).
- Support for CUDA graphs was added to MXNet-TensorRT integration.
- The speed of applying pointwise fusion to a graph was greatly improved.
- Support for CUDA enhanced compatibility was added.
- Latest version of NVIDIA NCCL 2.8.2
- Latest version of TensorRT 7.2.1
- Latest version of DALI 0.27
- Ubuntu 18.04 with October 2020 updates
- Deep learning framework containers 19.11 and later include experimental support for Singularity v3.0.
NVIDIA Optimized Deep Learning Framework Container Versions
The following table shows what versions of Ubuntu, CUDA, Apache MXNet, and TensorRT are supported in each of the NVIDIA containers for the Optimized Deep Learning Framework. For older container versions, refer to the Frameworks Support Matrix.
|Container Version||Ubuntu||CUDA Toolkit||Apache MXNet||TensorRT|
|NVIDIA CUDA 11.1.0||1.8.0.rc0||TensorRT 7.2.1|
|20.09||NVIDIA CUDA 11.0.3||TensorRT 7.1.3|
|20.07||NVIDIA CUDA 11.0.194|
|20.06||NVIDIA CUDA 11.0.167||TensorRT 7.1.2|
|20.03||NVIDIA CUDA 10.2.89||TensorRT 7.0.0|
|1.5.1 commit c98184806 from September 4, 2019|
|19.10||NVIDIA CUDA 10.1.243|
|19.09||1.5.0 commit 006486af3 from August 28, 2019|
|19.08||1.5.0 commit 75a9e187d from June 27, 2019||TensorRT 5.1.5|
Tensor Core Examples
The tensor core examples provided in GitHub and NVIDIA GPU Cloud (NGC) focus on achieving the best performance and convergence from NVIDIA Volta tensor cores by using the latest deep learning example networks and model scripts for training. Each example model trains with mixed precision Tensor Cores on Volta and Turing, therefore you can get results much faster than training without tensor cores. This model is tested against each NGC monthly container release to ensure consistent accuracy and performance over time. This container includes the following tensor core examples.
- The ResNet50 v1.5 model is a slightly modified version of the original ResNet50 v1 model that trains to a greater accuracy. This model script is available on GitHub as well as NVIDIA GPU Cloud (NGC).
Automatic Mixed Precision (AMP)
Training deep learning networks is a very computationally intensive task. Novel model architectures tend to have an increasing number of layers and parameters, which slows down training. Fortunately, new generations of training hardware as well as software optimizations make training these new models a feasible task.
Most of the hardware and software training optimization opportunities involve exploiting lower precision like FP16 in order to utilize the Tensor Cores available on new Volta and Turing GPUs. While training in FP16 showed great success in image classification tasks, other more complicated neural networks typically stayed in FP32 due to difficulties in applying the FP16 training guidelines that are needed to ensure proper model training.
That is where AMP (Automatic Mixed Precision) comes into play—it automatically applies the guidelines of FP16 training, using FP16 precision where it provides the most benefit, while conservatively keeping in full FP32 precision operations unsafe to do in FP16.
The NVIDIA Optimized Deep Learning Framework, powered by Apache MXNet AMP tutorial, located in
/opt/mxnet/nvidia-examples/AMP/AMP_tutorial.md inside this container, shows how to get started with mixed precision training using AMP for Apache MXNet, using by example the SSD network from GluonCV.
For more information about AMP, see the Training With Mixed Precision Guide.
The Apache MXNet KVStore GPU peer-to-peer communication tree discovery, as of release 18.09, is not compatible with DGX-1V. Only users that set the environment variable
MXNET_KVSTORE_USETREE=1will experience issues, which will be resolved in a subsequent release. Issue tracked under 13341.
The default setting of the environment variable
MXNET_GPU_COPY_NTHREADS=1in the container may not be optimal for all networks. Networks with a high ratio of parameters and computation, like AlexNet, may achieve greater multi-GPU training speeds with the setting
MXNET_GPU_COPY_NTHREADS=2. Users are encouraged to try this setting for their own use case.
- The 1D NCHW Deconvolution operator may produce incorrect output under these very narrow constraints: input_features=1, output_features=1, input_size=output_size, stride=1, cross-correlation mode, no bias and an output fed into an Addoperator. Also, the 1D NCHW Convolution operator may produce an incorrect data gradient under the constraints: input_features=1, output_features=1, input_size=output_size, stride=1, cross-correlation mode, no bias, an input fed into some other operator and MXNET_EXEC_ENABLE_ADDTO=1 set in the environment. This will be fixed in a future release.
- Performance regression (up to 40%) was noticed for ResNet50 mixed precision training with small batch sizes, such as 16 or 32 on V100 compared to 20.03. This will be addressed in a future release.
- Performance regression (up to 6%) was noticed for ResNet50 mixed precision training with batch size 128 and 256 on T4 compared to the 20.03. This will be addressed in a future release.