TensorRT Release 21.04
The NVIDIA container image for TensorRT, release 21.04, is available on NGC.
Contents of the TensorRT container
- The TensorRT C++ samples and C++ API documentation. The samples can be built by running make in the /workspace/tensorrt/samples directory. The resulting executables are in the /workspace/tensorrt/bin directory. The C++ API documentation can be found in the /workspace/tensorrt/doc/cpp directory.
- The TensorRT Python samples and Python API documentation. The Python samples
can be found in the /workspace/tensorrt/samples/python
directory. Many Python samples can be run using python
<script.py> -d /workspace/tensorrt/data. For example:
python caffe_resnet50.py -d /workspace/tensorrt/data
The Python API documentation can be found in the /workspace/tensorrt/doc/python directory. - TensorRT 7.2.3.4. Note that the ONNX
parser and plugin libraries bundled with this container are built from
TensorRT Open Source Software: https://github.com/NVIDIA/TensorRT/releases/tag/21.04Prominent updates to the 21.04 TensorRT Open Source Software release are:
- Addition of TensorRT Python API bindings.
- Addition of TensorRT Python samples.
- Plugin enhancements - FP16 support in batchedNMSPlugin, configurable input sizes for TLT MaskRCNN plugin.
- ONNX opset13 updates, ResNet example, and documentation updates to PyTorch Quantization toolkit.
- BERT demo updated to work with Tensorflow 2.x.
- Ubuntu 20.04Note: Container image 21.04-py3 contains Python 3.8.
- NVIDIA CUDA 11.3.0
- cuBLAS 11.5.1.101
- NVIDIA cuDNN 8.2.0.41
- NVIDIA NCCL 2.9.6
(optimized for NVLink™
)
Note: Although NCCL is packaged in the container, it does not affect TensorRT nor inferencing in any way.
- rdma-core 32.1
- OpenMPI 4.1.1rc1
- OpenUCX 1.10.0
- GDRCopy 2.2
- NVIDIA HPC-X 2.8.2rc3
- Nsight Compute 2021.1.0.18
- Nsight Systems 2021.1.3.14
- TensorRT 7.2.3.4
Driver Requirements
Release 21.04 is based on NVIDIA CUDA 11.3.0, which requires NVIDIA Driver release 465.19.01 or later. However, if you are running on Data Center GPUs (formerly Tesla), for example, T4, you may use NVIDIA driver release 418.40 (or later R418), 440.33 (or later R440), 450.51 (or later R450), or 460.27 (or later R460). The CUDA driver's compatibility package only supports particular drivers. For a complete list of supported drivers, see the CUDA Application Compatibility topic. For more information, see CUDA Compatibility and Upgrades and NVIDIA CUDA and Drivers Support.
GPU Requirements
Release 21.04 supports CUDA compute capability 3.5 and higher. This corresponds to GPUs in the Kepler, Maxwell, Pascal, Volta, Turing, and NVIDIA Ampere Architecture GPU families. Specifically, for a list of GPUs that this compute capability corresponds to, see CUDA GPUs. For additional support details, see Deep Learning Frameworks Support Matrix.
Key Features and Enhancements
- TensorRT container image version 21.04 is based on TensorRT 7.2.3.4. For a list of the new features and enhancements introduced in TensorRT 7.2.3.4 refer to the TensorRT 7.2.3 release notes.
- Ubuntu 20.04 with March 2021 updates
Obtaining Missing Data Files
Some samples require data files that are not included within the TensorRT container either due to licensing restrictions or because they are too large. Samples which do not include all the required data files include a README.md file in the corresponding source directory informing you how to obtain the necessary data files.
Installing Required Python Modules
You may need to first run the Python setup script in order to complete some of the samples. The following script has been added to the container to install the missing Python modules and their dependencies if desired: /opt/tensorrt/python/python_setup.sh
Installing Open Source Components
A script has been added to clone, build and replace the provided plugin, Caffe parser, and ONNX parser libraries with the open source ones based off the 21.04 tag on the official TensorRT open source repository.
To install the open source components inside the container, run the following script:
/opt/tensorrt/install_opensource.sh
For more information see GitHub: TensorRT 21.04.
Limitations
- Accelerating Inference In TensorFlow with TensorRT (TF-TRT) is not supported in the TensorRT containers.
NVIDIA TensorRT Container Versions
The following table shows what versions of Ubuntu, CUDA, and TensorRT are supported in each of the NVIDIA containers for TensorRT. For older container versions, refer to the Frameworks Support Matrix.
Known Issues
There are no known issues in this release.