MXNet Release 18.11

The NVIDIA container image for MXNet, release 18.11, is available.

Contents of MXNet

This container image contains the complete source of the version of MXNet in /opt/mxnet. It is pre-built and installed to the Python path.

The container also includes the following:

Driver Requirements

Release 18.11 is based on CUDA 10, which requires NVIDIA Driver release 410.xx. However, if you are running on Tesla (Tesla V100, Tesla P4, Tesla P40, or Tesla P100), you may use NVIDIA driver release 384. For more information, see CUDA Compatibility and Upgrades.

Key Features and Enhancements

This MXNet release includes the following key features and enhancements.
  • MXNet container image version 18.11 is based on 1.3.0, with all upstream changes from the MXNet master branch up to and including PR 12537.
  • Added fused BatchNormAddRelu operator to the MXNet Symbol package (accessible via mx.sym.BatchNormAddRelu), which performs BatchNorm operation on data, sums the result with a tensor and performs Relu activation on the result of the sum. Currently it is limited to FP16 data type and NHWC data layout.
  • Added MXNET_EXEC_ENABLE_ADDTO environment variable, which when set to 1 increases performance for some networks.
  • Increased performance of Batchnorm and Batchnorm+Relu operators in FP16 and NHWC data format.
  • Added support for multi-node via Horovod integration. Currently you can use it by specifying horovod type of KVStore.
  • Added MXNET_UPDATE_ON_KVSTORE environment variable, which controls whether to update parameters using KVStore (default is 1 for KVStore device and 0 for KVStore horovod).
  • Added aggregation of SGD updates which increases performance when update on KVStore is disabled.
  • Increased performance when training with small batch sizes.
  • Fixed a bug that prevented matrix multiplications to overlap with other computation, which increases performance for some networks.
  • Fixed an issue which prevented score function to respect not-full batches of data.
  • Added resnet-v1b as possible network in the train_imagenet_runner script.
  • Latest version of NCCL 2.3.7.
  • Latest version of NVIDIA cuDNN 7.4.1.
  • Latest version of TensorRT 5.0.2
  • Latest version of DALI 0.4.1 Beta.
  • Ubuntu 16.04 with October 2018 updates.

Known Issues

  • The MXNet KVStore GPU peer-to-peer communication tree discovery, as of release 18.09, is not compatible with DGX-1V. Only users that set the environment variable MXNET_KVSTORE_USETREE=1 will experience issues, which will be resolved in a subsequent release.

  • MXNet ResNet50 regresses in FP32 performance. This issue should be fixed in a later release.