NVIDIA Deep Learning TensorRT Documentation
Container Release Notes (PDF) - Last updated February 2, 2023

TensorRT Container Release Notes

The TensorRT container is an easy to use container for TensorRT development. The container allows you to build, modify, and execute TensorRT samples. These release notes provide a list of key features, packaged software in the container, software enhancements and improvements, and known issues for the 23.01 and earlier releases. The TensorRT container is released monthly to provide you with the latest NVIDIA deep learning software libraries and GitHub code contributions that have been sent upstream. The libraries and contributions have all been tested, tuned, and optimized.

For a complete view of the supported software and specific versions that are packaged with the frameworks based on the container image, see the Frameworks Support Matrix.

1. TensorRT Overview

The core of NVIDIA® TensorRT™ is a C++ library that facilitates high-performance inference on NVIDIA graphics processing units (GPUs). TensorRT takes a trained network, which consists of a network definition and a set of trained parameters, and produces a highly optimized runtime engine that performs inference for that network.

You can describe a TensorRT network by using a C++ or Python API, or you can import an existing Caffe, ONNX, or TensorFlow model by using one of the provided parsers.

TensorRT provides APIs through C++ and Python that help express deep learning models by using the Network Definition API or load a predefined model by using parsers that allows TensorRT to optimize and run them on an NVIDIA GPU. TensorRT applies graph optimizations, layer fusion, and other optimizations, while also finding the fastest implementation of that model by leveraging a diverse collection of highly optimized kernels. TensorRT also supplies a runtime that you can use to execute this network on all NVIDIA’s GPUs from the NVIDIA Kepler™ generation onwards.

TensorRT also includes optional high-speed, mixed precision capabilities that were introduced in Tegra X1 and were extended with the NVIDIA Pascal, NVIDIA Volta™, and NVIDIA Turing™ architectures.

The TensorRT container allows TensorRT samples to be built, modified, and executed. For more information about the TensorRT samples, see the TensorRT Sample Support Guide.

For a complete list of installation options and instructions, refer to Installing TensorRT.

2. Pulling A Container

Before you can pull a container from the NGC container registry:

The deep learning frameworks, the NGC Docker containers, and the deep learning framework containers are stored in the nvcr.io/nvidia repository.

3. Running TensorRT

Before you can run an NGC deep learning framework container, your Docker environment must support NVIDIA GPUs. To run a container, issue the appropriate command as explained in Running A Container and specify the registry, repository, and tags.

On a system with GPU support for NGC containers, when you run a container, the following occurs:
  • The Docker engine loads the image into a container which runs the software.
  • You define the runtime resources of the container by including the additional flags and settings that are used with the command.

    These flags and settings are described in Running A Container.

  • The GPUs are explicitly defined for the Docker container, which defaults to all GPUs, but can be specified by using the NVIDIA_VISIBLE_DEVICES environment variable.

    For more information, refer to the nvidia-docker documentation.

    Note: Starting in Docker 19.03, complete the steps below.

The method implemented in your system depends on the DGX OS version that you installed (for DGX systems), the NGC Cloud Image that was provided by a Cloud Service Provider, or the software that you installed to prepare to run NGC containers on TITAN PCs, Quadro PCs, or NVIDIA Virtual GPUs (vGPUs).

  1. Issue the command for the applicable release of the container that you want.

    The following command assumes that you want to pull the latest container.

    docker pull nvcr.io/nvidia/tensorrt:23.01-py3
  2. Open a command prompt and paste the pull command.

    Ensure that the pull process successfully completes before you proceed to step 3.

  3. Run the container image.
    • If you have Docker 19.03 or later, a typical command to launch the container is:
      docker run --gpus all -it --rm -v local_dir:container_dir nvcr.io/nvidia/tensorrt:<xx.xx>-py<x>
    • If you have Docker 19.02 or earlier, a typical command to launch the container is:
      nvidia-docker run -it --rm -v local_dir:container_dir nvcr.io/nvidia/tensorrt:<xx.xx>-py<x>
  4. To extend the TensorRT container, select one of the following options:
    • Add to or modify the source code in this container and run your customized version.
    • To add additional packages, use docker build to add your customizations on top of this container.
      Note: NVIDIA recommends using the docker build option for ease of migration to later versions of the TensorRT container.

4. TensorRT Release 23.01

The NVIDIA container image for TensorRT, release 23.01, is available on NGC.

Contents of the TensorRT container

This container includes the following:
  • The TensorRT C++ samples and C++ API documentation.
    • Build the samples can be by running make in the /workspace/tensorrt/samples directory.
    • The resulting executables are in the /workspace/tensorrt/bin directory.
    • The C++ API documentation is in the /workspace/tensorrt/doc/cpp directory.
  • The TensorRT Python samples and Python API documentation.
    • The Python samples are in the /workspace/tensorrt/samples/python directory.

      Refer to the respective README documents for more samples.

    • Many Python samples can be run by using python <script.py> -d /workspace/tensorrt/data.
      For example:
      python onnx_resnet50.py -d /workspace/tensorrt/data
    • The Python API documentation is in the /workspace/tensorrt/doc/python directory.
  • TensorRT 8.5.2.2.

    The ONNX parser and plug-in libraries that are bundled with this container are built from TensorRT Open Source Software.

The container also includes the following:

Driver Requirements

Release 23.01 is based on CUDA 12.0.1, which requires NVIDIA Driver release 525 or later. However, if you are running on a data center GPU (for example, T4 or any other data center GPU), you can use NVIDIA driver release 450.51 (or later R450), 470.57 (or later R470), 510.47 (or later R510), 515.65 (or later R515), or 525.85 (or later R525).

The CUDA driver's compatibility package only supports particular drivers. Thus, users should upgrade from all R418, R440, and R460 drivers, which are not forward-compatible with CUDA 12.0. For a complete list of supported drivers, see the CUDA Application Compatibility topic. For more information, see CUDA Compatibility and Upgrades.

GPU Requirements

Release 23.01 supports CUDA compute capability 3.5 and later. This corresponds to GPUs in the NVIDIA Kepler, Maxwell, NVIDIA Pascal, NVIDIA Volta™, NVIDIA Turing™, NVIDIA Ampere architecture, and NVIDIA Hopper™ architecture families. For a list of GPUs to which this compute capability corresponds, see CUDA GPUs. For additional support details, see the Deep Learning Frameworks Support Matrix.

Key Features and Enhancements

This TensorRT container release includes the following key features and enhancements.
  • TensorRT container image version 23.01 is based on TensorRT 8.5.2.2.

    For a list of the features and enhancements that were introduced in TensorRT 8.5.2, refer to the TensorRT 8.5 release notes.

  • Ubuntu 20.04 with December 2022 updates.

Announcements

  • Starting with the 22.05 release, the TensorRT container is available for the Arm SBSA platform.

    For example, when you pull the nvcr.io/nvidia/tensorrt:22.05-py3 Docker image on an Arm SBSA machine, the Arm-specific image is automatically fetched.

  • NVIDIA Deep Learning Profiler (DLProf) v1.8, which was included in the 21.12 container, was the last release of DLProf.

    Starting with the 22.01 container, DLProf is longer included. It can still be manually installed by using a pip wheel on the nvidia-pyindex.

Obtaining Missing Data Files

Some samples require data files that are not included in the TensorRT container because of licensing restrictions, or because they are too large. Samples that do not include the required data files include a README.md file in the corresponding source directory that provides information about how to obtain the necessary data files.

Installing Required Python Modules

  • To complete some of the samples, you might want to first run the Python setup script.
  • If you need to install the missing Python modules and their dependencies, run the /opt/tensorrt/python/python_setup.sh script.

Installing Open Source Components

A script has been added to clone, build, and replace the provided plug-in, the Caffe parser, and the ONNX parser libraries with the open source ones that are based on the 22.05 tag on the official TensorRT open source repository.

To install the open source components in the container, run the following script:

/opt/tensorrt/install_opensource.sh -b main
Note: Since the 22.09 release is based on an early access version of TensorRT 8.5, which is not accompanied by the publication of a corresponding TensorRT Open Source Software (OSS) release to GitHub, please specify building from the main branch in install_opensource.sh until the TensorRT OSS 8.5.1 release is posted.

For more information, see GitHub: TensorRT.

Limitations

Known Issues

None.

5. TensorRT Release 22.12

The NVIDIA container image for TensorRT, release 22.12, is available on NGC.

Contents of the TensorRT container

This container includes the following:
  • The TensorRT C++ samples and C++ API documentation.
    • Build the samples can be by running make in the /workspace/tensorrt/samples directory.
    • The resulting executables are in the /workspace/tensorrt/bin directory.
    • The C++ API documentation is in the /workspace/tensorrt/doc/cpp directory.
  • The TensorRT Python samples and Python API documentation.
    • The Python samples are in the /workspace/tensorrt/samples/python directory.

      Refer to the respective README documents for more samples.

    • Many Python samples can be run by using python <script.py> -d /workspace/tensorrt/data.
      For example:
      python onnx_resnet50.py -d /workspace/tensorrt/data
    • The Python API documentation is in the /workspace/tensorrt/doc/python directory.
  • TensorRT 8.5.1.

    The ONNX parser and plug-in libraries that are bundled with this container are built from TensorRT Open Source Software.

The container also includes the following:

Driver Requirements

Release 22.12 is based on CUDA 11.8.0, which requires NVIDIA Driver release 520 or later. However, if you are running on a data center GPU (for example, T4 or any other data center GPU), you can use NVIDIA driver release 450.51 (or later R450), 470.57 (or later R470), 510.47 (or later R510), or 515.65 (or later R515).

The CUDA driver's compatibility package only supports particular drivers. Thus, users should upgrade from all R418, R440, and R460 drivers, which are not forward-compatible with CUDA 11.8. For a complete list of supported drivers, see the CUDA Application Compatibility topic. For more information, see CUDA Compatibility and Upgrades.

GPU Requirements

Release 22.12 supports CUDA compute capability 3.5 and later. This corresponds to GPUs in the NVIDIA Kepler, Maxwell, NVIDIA Pascal, NVIDIA Volta™, NVIDIA Turing™, NVIDIA Ampere architecture, and NVIDIA Hopper™ architecture families. For a list of GPUs to which this compute capability corresponds, see CUDA GPUs. For additional support details, see the Deep Learning Frameworks Support Matrix.

Key Features and Enhancements

This TensorRT container release includes the following key features and enhancements.
  • TensorRT container image version 22.12 is based on TensorRT 8.5.1.

    For a list of the features and enhancements that were introduced in TensorRT 8.5.1, refer to the TensorRT 8.5 release notes.

  • Ubuntu 20.04 with November 2022 updates.

Announcements

  • Starting with the 22.05 release, the TensorRT container is available for the Arm SBSA platform.

    For example, when you pull the nvcr.io/nvidia/tensorrt:22.05-py3 Docker image on an Arm SBSA machine, the Arm-specific image is automatically fetched.

  • NVIDIA Deep Learning Profiler (DLProf) v1.8, which was included in the 21.12 container, was the last release of DLProf.

    Starting with the 22.01 container, DLProf is longer included. It can still be manually installed by using a pip wheel on the nvidia-pyindex.

Obtaining Missing Data Files

Some samples require data files that are not included in the TensorRT container because of licensing restrictions, or because they are too large. Samples that do not include the required data files include a README.md file in the corresponding source directory that provides information about how to obtain the necessary data files.

Installing Required Python Modules

  • To complete some of the samples, you might want to first run the Python setup script.
  • If you need to install the missing Python modules and their dependencies, run the /opt/tensorrt/python/python_setup.sh script.

Installing Open Source Components

A script has been added to clone, build, and replace the provided plug-in, the Caffe parser, and the ONNX parser libraries with the open source ones that are based on the 22.05 tag on the official TensorRT open source repository.

To install the open source components in the container, run the following script:

/opt/tensorrt/install_opensource.sh -b main
Note: Since the 22.09 release is based on an early access version of TensorRT 8.5, which is not accompanied by the publication of a corresponding TensorRT Open Source Software (OSS) release to GitHub, please specify building from the main branch in install_opensource.sh until the TensorRT OSS 8.5.1 release is posted.

For more information, see GitHub: TensorRT.

Limitations

Known Issues

None.

6. TensorRT Release 22.11

The NVIDIA container image for TensorRT, release 22.10, is available on NGC.

Contents of the TensorRT container

This container includes the following:
  • The TensorRT C++ samples and C++ API documentation.
    • Build the samples can be by running make in the /workspace/tensorrt/samples directory.
    • The resulting executables are in the /workspace/tensorrt/bin directory.
    • The C++ API documentation is in the /workspace/tensorrt/doc/cpp directory.
  • The TensorRT Python samples and Python API documentation.
    • The Python samples are in the /workspace/tensorrt/samples/python directory.

      Refer to the respective README documents for more samples.

    • Many Python samples can be run by using python <script.py> -d /workspace/tensorrt/data.
      For example:
      python onnx_resnet50.py -d /workspace/tensorrt/data
    • The Python API documentation is in the /workspace/tensorrt/doc/python directory.
  • TensorRT 8.5.1.

    The ONNX parser and plug-in libraries that are bundled with this container are built from TensorRT Open Source Software.

The container also includes the following:

Driver Requirements

Release 22.11 is based on CUDA 11.8.0, which requires NVIDIA Driver release 520 or later. However, if you are running on a data center GPU (for example, T4 or any other data center GPU), you can use NVIDIA driver release 450.51 (or later R450), 470.57 (or later R470), 510.47 (or later R510), or 515.65 (or later R515).

The CUDA driver's compatibility package only supports particular drivers. Thus, users should upgrade from all R418, R440, and R460 drivers, which are not forward-compatible with CUDA 11.8. For a complete list of supported drivers, see the CUDA Application Compatibility topic. For more information, see CUDA Compatibility and Upgrades.

GPU Requirements

Release 22.11 supports CUDA compute capability 3.5 and later. This corresponds to GPUs in the NVIDIA Kepler, Maxwell, NVIDIA Pascal, NVIDIA Volta™, NVIDIA Turing™, NVIDIA Ampere architecture, and NVIDIA Hopper™ architecture families. For a list of GPUs to which this compute capability corresponds, see CUDA GPUs. For additional support details, see the Deep Learning Frameworks Support Matrix.

Key Features and Enhancements

This TensorRT container release includes the following key features and enhancements.
  • TensorRT container image version 22.11 is based on TensorRT 8.5.1.

    For a list of the features and enhancements that were introduced in TensorRT 8.5.1, refer to the TensorRT 8.5 release notes.

  • Ubuntu 20.04 with October 2022 updates.

Announcements

  • Starting with the 22.05 release, the TensorRT container is available for the Arm SBSA platform.

    For example, when you pull the nvcr.io/nvidia/tensorrt:22.05-py3 Docker image on an Arm SBSA machine, the Arm-specific image is automatically fetched.

  • NVIDIA Deep Learning Profiler (DLProf) v1.8, which was included in the 21.12 container, was the last release of DLProf.

    Starting with the 22.01 container, DLProf is longer included. It can still be manually installed by using a pip wheel on the nvidia-pyindex.

Obtaining Missing Data Files

Some samples require data files that are not included in the TensorRT container because of licensing restrictions, or because they are too large. Samples that do not include the required data files include a README.md file in the corresponding source directory that provides information about how to obtain the necessary data files.

Installing Required Python Modules

  • To complete some of the samples, you might want to first run the Python setup script.
  • If you need to install the missing Python modules and their dependencies, run the /opt/tensorrt/python/python_setup.sh script.

Installing Open Source Components

A script has been added to clone, build, and replace the provided plug-in, the Caffe parser, and the ONNX parser libraries with the open source ones that are based on the 22.05 tag on the official TensorRT open source repository.

To install the open source components in the container, run the following script:

/opt/tensorrt/install_opensource.sh -b main
Note: Since the 22.09 release is based on an early access version of TensorRT 8.5, which is not accompanied by the publication of a corresponding TensorRT Open Source Software (OSS) release to GitHub, please specify building from the main branch in install_opensource.sh until the TensorRT OSS 8.5.1 release is posted.

For more information, see GitHub: TensorRT.

Limitations

Known Issues

None.

7. TensorRT Release 22.10

The NVIDIA container image for TensorRT, release 22.10, is available on NGC.

Contents of the TensorRT container

This container includes the following:
  • The TensorRT C++ samples and C++ API documentation.
    • Build the samples can be by running make in the /workspace/tensorrt/samples directory.
    • The resulting executables are in the /workspace/tensorrt/bin directory.
    • The C++ API documentation is in the /workspace/tensorrt/doc/cpp directory.
  • The TensorRT Python samples and Python API documentation.
    • The Python samples are in the /workspace/tensorrt/samples/python directory.

      Refer to the respective README documents for more samples.

    • Many Python samples can be run by using python <script.py> -d /workspace/tensorrt/data.
      For example:
      python onnx_resnet50.py -d /workspace/tensorrt/data
    • The Python API documentation is in the /workspace/tensorrt/doc/python directory.
  • TensorRT 8.5 EA.

    The ONNX parser and plug-in libraries that are bundled with this container are built from TensorRT Open Source Software.

The container also includes the following:

Driver Requirements

Release 22.10 is based on CUDA 11.8.0, which requires NVIDIA Driver release 520 or later. However, if you are running on a data center GPU (for example, T4 or any other data center GPU), you can use NVIDIA driver release 450.51 (or later R450), 470.57 (or later R470), 510.47 (or later R510), or 515.65 (or later R515).

The CUDA driver's compatibility package only supports particular drivers. Thus, users should upgrade from all R418, R440, and R460 drivers, which are not forward-compatible with CUDA 11.8. For a complete list of supported drivers, see the CUDA Application Compatibility topic. For more information, see CUDA Compatibility and Upgrades.

GPU Requirements

Release 22.10 supports CUDA compute capability 3.5 and later. This corresponds to GPUs in the NVIDIA Kepler, Maxwell, NVIDIA Pascal, NVIDIA Volta™, NVIDIA Turing™, NVIDIA Ampere architecture, and NVIDIA Hopper™ architecture families. For a list of GPUs to which this compute capability corresponds, see CUDA GPUs. For additional support details, see the Deep Learning Frameworks Support Matrix.

Key Features and Enhancements

This TensorRT container release includes the following key features and enhancements.
  • TensorRT container image version 22.10 is based on TensorRT 8.5 EA.

    For a list of the features and enhancements that were introduced in TensorRT 8.5.0.12, refer to the TensorRT 8.5 release notes.

  • Ubuntu 20.04 with September 2022 updates.

Announcements

  • Starting with the 22.05 release, the TensorRT container is available for the Arm SBSA platform.

    For example, when you pull the nvcr.io/nvidia/tensorrt:22.05-py3 Docker image on an Arm SBSA machine, the Arm-specific image is automatically fetched.

  • NVIDIA Deep Learning Profiler (DLProf) v1.8, which was included in the 21.12 container, was the last release of DLProf.

    Starting with the 22.01 container, DLProf is longer included. It can still be manually installed by using a pip wheel on the nvidia-pyindex.

Obtaining Missing Data Files

Some samples require data files that are not included in the TensorRT container because of licensing restrictions, or because they are too large. Samples that do not include the required data files include a README.md file in the corresponding source directory that provides information about how to obtain the necessary data files.

Installing Required Python Modules

  • To complete some of the samples, you might want to first run the Python setup script.
  • If you need to install the missing Python modules and their dependencies, run the /opt/tensorrt/python/python_setup.sh script.

Installing Open Source Components

A script has been added to clone, build, and replace the provided plug-in, the Caffe parser, and the ONNX parser libraries with the open source ones that are based on the 22.05 tag on the official TensorRT open source repository.

To install the open source components in the container, run the following script:

/opt/tensorrt/install_opensource.sh -b main
Note: Since the 22.09 release is based on an early access version of TensorRT 8.5, which is not accompanied by the publication of a corresponding TensorRT Open Source Software (OSS) release to GitHub, please specify building from the main branch in install_opensource.sh until the TensorRT OSS 8.5.1 release is posted.

For more information, see GitHub: TensorRT.

Limitations

Known Issues

None.

8. TensorRT Release 22.09

The NVIDIA container image for TensorRT, release 22.09, is available on NGC.

Contents of the TensorRT container

This container includes the following:
  • The TensorRT C++ samples and C++ API documentation.
    • Build the samples can be by running make in the /workspace/tensorrt/samples directory.
    • The resulting executables are in the /workspace/tensorrt/bin directory.
    • The C++ API documentation is in the /workspace/tensorrt/doc/cpp directory.
  • The TensorRT Python samples and Python API documentation.
    • The Python samples are in the /workspace/tensorrt/samples/python directory.

      Refer to the respective README documents for more samples.

    • Many Python samples can be run by using python <script.py> -d /workspace/tensorrt/data.
      For example:
      python onnx_resnet50.py -d /workspace/tensorrt/data
    • The Python API documentation is in the /workspace/tensorrt/doc/python directory.
  • TensorRT 8.5 EA.

    The ONNX parser and plug-in libraries that are bundled with this container are built from TensorRT Open Source Software.

The container also includes the following:

Driver Requirements

Release 22.09 is based on CUDA 11.8.0, which requires NVIDIA Driver release 520 or later. However, if you are running on a data center GPU (for example, T4 or any other data center GPU), you can use NVIDIA driver release 450.51 (or later R450), 470.57 (or later R470), 510.47 (or later R510), or 515.65 (or later R515).

The CUDA driver's compatibility package only supports particular drivers. Thus, users should upgrade from all R418, R440, and R460 drivers, which are not forward-compatible with CUDA 11.8. For a complete list of supported drivers, see the CUDA Application Compatibility topic. For more information, see CUDA Compatibility and Upgrades.

GPU Requirements

Release 22.09 supports CUDA compute capability 3.5 and later. This corresponds to GPUs in the NVIDIA Kepler, Maxwell, NVIDIA Pascal, NVIDIA Volta™, NVIDIA Turing™, NVIDIA Ampere architecture, and NVIDIA Hopper™ architecture families. For a list of GPUs to which this compute capability corresponds, see CUDA GPUs. For additional support details, see the Deep Learning Frameworks Support Matrix.

Key Features and Enhancements

This TensorRT container release includes the following key features and enhancements.
  • TensorRT container image version 22.09 is based on TensorRT 8.5 EA.

    For a list of the features and enhancements that were introduced in TensorRT 8.5.0.12, refer to the TensorRT 8.5 release notes.

  • Ubuntu 20.04 with August 2022 updates.

Announcements

  • Starting with the 22.05 release, the TensorRT container is available for the Arm SBSA platform.

    For example, when you pull the nvcr.io/nvidia/tensorrt:22.05-py3 Docker image on an Arm SBSA machine, the Arm-specific image is automatically fetched.

  • NVIDIA Deep Learning Profiler (DLProf) v1.8, which was included in the 21.12 container, was the last release of DLProf.

    Starting with the 22.01 container, DLProf is longer included. It can still be manually installed by using a pip wheel on the nvidia-pyindex.

Obtaining Missing Data Files

Some samples require data files that are not included in the TensorRT container because of licensing restrictions, or because they are too large. Samples that do not include the required data files include a README.md file in the corresponding source directory that provides information about how to obtain the necessary data files.

Installing Required Python Modules

  • To complete some of the samples, you might want to first run the Python setup script.
  • If you need to install the missing Python modules and their dependencies, run the /opt/tensorrt/python/python_setup.sh script.

Installing Open Source Components

A script has been added to clone, build, and replace the provided plug-in, the Caffe parser, and the ONNX parser libraries with the open source ones that are based on the 22.05 tag on the official TensorRT open source repository.

To install the open source components in the container, run the following script:

/opt/tensorrt/install_opensource.sh -b main
Note: Since the 22.09 release is based on an early access version of TensorRT 8.5, which is not accompanied by the publication of a corresponding TensorRT Open Source Software (OSS) release to GitHub, please specify building from the main branch in install_opensource.sh until the TensorRT OSS 8.5.1 release is posted.

For more information, see GitHub: TensorRT.

Limitations

Known Issues

None.

9. TensorRT Release 22.08

The NVIDIA container image for TensorRT, release 22.08, is available on NGC.

Contents of the TensorRT container

This container includes the following:
  • The TensorRT C++ samples and C++ API documentation.
    • Build the samples can be by running make in the /workspace/tensorrt/samples directory.
    • The resulting executables are in the /workspace/tensorrt/bin directory.
    • The C++ API documentation is in the /workspace/tensorrt/doc/cpp directory.
  • The TensorRT Python samples and Python API documentation.
    • The Python samples are in the /workspace/tensorrt/samples/python directory.

      Refer to the respective README documents for more samples.

    • Many Python samples can be run by using python <script.py> -d /workspace/tensorrt/data.
      For example:
      python onnx_resnet50.py -d /workspace/tensorrt/data
    • The Python API documentation is in the /workspace/tensorrt/doc/python directory.
  • TensorRT 8.4.2.4.
    The ONNX parser and plug-in libraries that are bundled with this container are built from TensorRT Open Source Software:
The container also includes the following:

Driver Requirements

Release 22.08 is based on CUDA 11.7.1, which requires NVIDIA Driver release 515 or later. However, if you are running on a data center GPU (for example, T4 or any other data center GPU), you can use NVIDIA driver release 450.51 (or later R450), 470.57 (or later R470), or 510.47 (or later R510).

The CUDA driver's compatibility package only supports particular drivers. Thus, users should upgrade from all R418, R440, and R460 drivers, which are not forward-compatible with CUDA 11.7. For a complete list of supported drivers, see the CUDA Application Compatibility topic. For more information, see CUDA Compatibility and Upgrades.

GPU Requirements

Release 22.08 supports CUDA compute capability 3.5 and later. This corresponds to GPUs in the Kepler, Maxwell, Pascal, Volta, Turing, and NVIDIA Ampere Architecture GPU families. For a list of GPUs to which this compute capability corresponds, see CUDA GPUs. For additional support details, see the Deep Learning Frameworks Support Matrix.

Key Features and Enhancements

This TensorRT container release includes the following key features and enhancements.
  • TensorRT container image version 22.08 is based on TensorRT 8.4.2.4.

    For a list of the features and enhancements that were introduced in TensorRT 8.4.2.4, refer to the TensorRT 8.4.2 release notes.

  • Ubuntu 20.04 with July 2022 updates.

Announcements

  • Starting with the 22.05 release, the TensorRT container is available for the Arm SBSA platform.

    For example, when you pull the nvcr.io/nvidia/tensorrt:22.05-py3 Docker image on an Arm SBSA machine, the Arm-specific image is automatically fetched.

  • NVIDIA Deep Learning Profiler (DLProf) v1.8, which was included in the 21.12 container, was the last release of DLProf.

    Starting with the 22.01 container, DLProf is longer included. It can still be manually installed by using a pip wheel on the nvidia-pyindex.

Obtaining Missing Data Files

Some samples require data files that are not included in the TensorRT container because of licensing restrictions, or because they are too large. Samples that do not include the required data files include a README.md file in the corresponding source directory that provides information about how to obtain the necessary data files.

Installing Required Python Modules

  • To complete some of the samples, you might want to first run the Python setup script.
  • If you need to install the missing Python modules and their dependencies, run the /opt/tensorrt/python/python_setup.sh script.

Installing Open Source Components

A script has been added to clone, build, and replace the provided plug-in, the Caffe parser, and the ONNX parser libraries with the open source ones that are based on the 22.05 tag on the official TensorRT open source repository.

To install the open source components in the container, run the following script:

/opt/tensorrt/install_opensource.sh

For more information, see GitHub: TensorRT 22.08.

Limitations

Known Issues

None.

10. TensorRT Release 22.07

The NVIDIA container image for TensorRT, release 22.07, is available on NGC.

Contents of the TensorRT container

This container includes the following:
  • The TensorRT C++ samples and C++ API documentation.
    • Build the samples can be by running make in the /workspace/tensorrt/samples directory.
    • The resulting executables are in the /workspace/tensorrt/bin directory.
    • The C++ API documentation is in the /workspace/tensorrt/doc/cpp directory.
  • The TensorRT Python samples and Python API documentation.
    • The Python samples are in the /workspace/tensorrt/samples/python directory.

      Refer to the respective README documents for more samples.

    • Many Python samples can be run by using python <script.py> -d /workspace/tensorrt/data.
      For example:
      python onnx_resnet50.py -d /workspace/tensorrt/data
    • The Python API documentation is in the /workspace/tensorrt/doc/python directory.
  • TensorRT 8.4.1.
    The ONNX parser and plug-in libraries that are bundled with this container are built from TensorRT Open Source Software:
The container also includes the following:

Driver Requirements

Release 22.07 is based on CUDA 11.7 Update 1 Preview, which requires NVIDIA Driver release 515 or later. However, if you are running on a data center GPU (for example, T4 or any other data center GPU), you can use NVIDIA driver release 450.51 (or later R450), 470.57 (or later R470), or 510.47 (or later R510).

The CUDA driver's compatibility package only supports particular drivers. Thus, users should upgrade from all R418, R440, and R460 drivers, which are not forward-compatible with CUDA 11.7. For a complete list of supported drivers, see the CUDA Application Compatibility topic. For more information, see CUDA Compatibility and Upgrades.

GPU Requirements

Release 22.07 supports CUDA compute capability 3.5 and later. This corresponds to GPUs in the Kepler, Maxwell, Pascal, Volta, Turing, and NVIDIA Ampere Architecture GPU families. For a list of GPUs to which this compute capability corresponds, see CUDA GPUs. For additional support details, see the Deep Learning Frameworks Support Matrix.

Key Features and Enhancements

This TensorRT container release includes the following key features and enhancements.
  • TensorRT container image version 22.07 is based on TensorRT 8.4.1.

    For a list of the features and enhancements that were introduced in TensorRT 8.4.1, refer to the TensorRT 8.4.1 release notes.

  • Ubuntu 20.04 with June 2022 updates.

Announcements

  • Starting with the 22.05 release, the TensorRT container is available for the Arm SBSA platform.

    For example, when you pull the nvcr.io/nvidia/tensorrt:22.05-py3 Docker image on an Arm SBSA machine, the Arm-specific image is automatically fetched.

  • NVIDIA Deep Learning Profiler (DLProf) v1.8, which was included in the 21.12 container, was the last release of DLProf.

    Starting with the 22.01 container, DLProf is longer included. It can still be manually installed by using a pip wheel on the nvidia-pyindex.

Obtaining Missing Data Files

Some samples require data files that are not included in the TensorRT container because of licensing restrictions, or because they are too large. Samples that do not include the required data files include a README.md file in the corresponding source directory that provides information about how to obtain the necessary data files.

Installing Required Python Modules

  • To complete some of the samples, you might want to first run the Python setup script.
  • If you need to install the missing Python modules and their dependencies, run the /opt/tensorrt/python/python_setup.sh script.

Installing Open Source Components

A script has been added to clone, build, and replace the provided plug-in, the Caffe parser, and the ONNX parser libraries with the open source ones that are based on the 22.05 tag on the official TensorRT open source repository.

To install the open source components in the container, run the following script:

/opt/tensorrt/install_opensource.sh

For more information, see GitHub: TensorRT 22.07.

Limitations

Known Issues

None.

11. TensorRT Release 22.06

The NVIDIA container image for TensorRT, release 22.06, is available on NGC.

Contents of the TensorRT container

This container includes the following:
  • The TensorRT C++ samples and C++ API documentation.
    • Build the samples can be by running make in the /workspace/tensorrt/samples directory.
    • The resulting executables are in the /workspace/tensorrt/bin directory.
    • The C++ API documentation is in the /workspace/tensorrt/doc/cpp directory.
  • The TensorRT Python samples and Python API documentation.
    • The Python samples are in the /workspace/tensorrt/samples/python directory.

      Refer to the respective README documents for more samples.

    • Many Python samples can be run by using python <script.py> -d /workspace/tensorrt/data.
      For example:
      python onnx_resnet50.py -d /workspace/tensorrt/data
    • The Python API documentation is in the /workspace/tensorrt/doc/python directory.
  • TensorRT 8.2.5.
    The ONNX parser and plug-in libraries that are bundled with this container are built from TensorRT Open Source Software:
The container also includes the following:

Driver Requirements

Release 22.06 is based on CUDA 11.7 Update 1 Preview, which requires NVIDIA Driver release 515 or later. However, if you are running on a data center GPU (for example, T4 or any other data center GPU), you can use NVIDIA driver release 450.51 (or later R450), 470.57 (or later R470), or 510.47 (or later R510).

The CUDA driver's compatibility package only supports particular drivers. Thus, users should upgrade from all R418, R440, and R460 drivers, which are not forward-compatible with CUDA 11.7. For a complete list of supported drivers, see the CUDA Application Compatibility topic. For more information, see CUDA Compatibility and Upgrades.

GPU Requirements

Release 22.06 supports CUDA compute capability 3.5 and later. This corresponds to GPUs in the Kepler, Maxwell, Pascal, Volta, Turing, and NVIDIA Ampere Architecture GPU families. For a list of GPUs to which this compute capability corresponds, see CUDA GPUs. For additional support details, see the Deep Learning Frameworks Support Matrix.

Key Features and Enhancements

This TensorRT container release includes the following key features and enhancements.
  • TensorRT container image version 22.06 is based on TensorRT 8.2.5.

    For a list of the features and enhancements that were introduced in TensorRT 8.2.5, refer to the TensorRT 8.2.5 release notes.

  • Ubuntu 20.04 with May 2022 updates.

Announcements

  • Starting with the 22.05 release, the TensorRT container is available for the Arm SBSA platform.

    For example, when you pull the nvcr.io/nvidia/tensorrt:22.05-py3 Docker image on an Arm SBSA machine, the Arm-specific image is automatically fetched.

  • NVIDIA Deep Learning Profiler (DLProf) v1.8, which was included in the 21.12 container, was the last release of DLProf.

    Starting with the 22.01 container, DLProf is longer included. It can still be manually installed by using a pip wheel on the nvidia-pyindex.

Obtaining Missing Data Files

Some samples require data files that are not included in the TensorRT container because of licensing restrictions, or because they are too large. Samples that do not include the required data files include a README.md file in the corresponding source directory that provides information about how to obtain the necessary data files.

Installing Required Python Modules

  • To complete some of the samples, you might want to first run the Python setup script.
  • If you need to install the missing Python modules and their dependencies, run the /opt/tensorrt/python/python_setup.sh script.

Installing Open Source Components

A script has been added to clone, build, and replace the provided plug-in, the Caffe parser, and the ONNX parser libraries with the open source ones that are based on the 22.05 tag on the official TensorRT open source repository.

To install the open source components in the container, run the following script:

/opt/tensorrt/install_opensource.sh

For more information, see GitHub: TensorRT 22.06.

Limitations

Known Issues

None.

12. TensorRT Release 22.05

The NVIDIA container image for TensorRT, release 22.05, is available on NGC.

Contents of the TensorRT container

This container includes the following:
  • The TensorRT C++ samples and C++ API documentation.
    • Build the samples can be by running make in the /workspace/tensorrt/samples directory.
    • The resulting executables are in the /workspace/tensorrt/bin directory.
    • The C++ API documentation is in the /workspace/tensorrt/doc/cpp directory.
  • The TensorRT Python samples and Python API documentation.
    • The Python samples are in the /workspace/tensorrt/samples/python directory.

      Refer to the respective README documents for more samples.

    • Many Python samples can be run by using python <script.py> -d /workspace/tensorrt/data.
      For example:
      python onnx_resnet50.py -d /workspace/tensorrt/data
    • The Python API documentation is in the /workspace/tensorrt/doc/python directory.
  • TensorRT 8.2.5.1.
    The ONNX parser and plug-in libraries that are bundled with this container are built from TensorRT Open Source Software:

The container also includes the following:

Driver Requirements

Release 22.05 is based on CUDA 11.7, which requires NVIDIA Driver release 515 or later. However, if you are running on a data center GPU (for example, T4 or any other data center GPU), you can use NVIDIA driver release 450.51 (or later R450), 470.57 (or later R470), or 510.47 (or later R510).

The CUDA driver's compatibility package only supports particular drivers. Thus, users should upgrade from all R418, R440, and R460 drivers, which are not forward-compatible with CUDA 11.7. For a complete list of supported drivers, see the CUDA Application Compatibility topic. For more information, see CUDA Compatibility and Upgrades.

GPU Requirements

Release 22.05 supports CUDA compute capability 3.5 and later. This corresponds to GPUs in the Kepler, Maxwell, Pascal, Volta, Turing, and NVIDIA Ampere Architecture GPU families. For a list of GPUs to which this compute capability corresponds, see CUDA GPUs. For additional support details, see the Deep Learning Frameworks Support Matrix.

Key Features and Enhancements

This TensorRT container release includes the following key features and enhancements.
  • TensorRT container image version 22.05 is based on TensorRT 8.2.5.

    For a list of the features and enhancements that were introduced in TensorRT 8.2.5, refer to the TensorRT 8.2.5 release notes.

  • Ubuntu 20.04 with April 2022 updates.
  • Added Disentangled attention plugin for DeBERTa.
  • Added DMHA (multiscaleDeformableAttnPlugin) plugin for DDETR.
  • Added fp16 support for pillarScatterPlugin.
  • Removed usage of deprecated TensorRT APIs in samples.

Announcements

  • Starting with the 22.05 release, the TensorRT container is available for the Arm SBSA platform.

    For example, when you pull the nvcr.io/nvidia/tensorrt:22.05-py3 Docker image on an Arm SBSA machine, the Arm-specific image is automatically fetched.

  • NVIDIA Deep Learning Profiler (DLProf) v1.8, which was included in the 21.12 container, was the last release of DLProf.

    Starting with the 22.01 container, DLProf is longer included. It can still be manually installed by using a pip wheel on the nvidia-pyindex.

Obtaining Missing Data Files

Some samples require data files that are not included in the TensorRT container because of licensing restrictions, or because they are too large. Samples that do not include the required data files include a README.md file in the corresponding source directory that provides information about how to obtain the necessary data files.

Installing Required Python Modules

  • To complete some of the samples, you might want to first run the Python setup script.
  • If you need to install the missing Python modules and their dependencies, run the /opt/tensorrt/python/python_setup.sh script.

Installing Open Source Components

A script has been added to clone, build, and replace the provided plug-in, the Caffe parser, and the ONNX parser libraries with the open source ones that are based on the 22.05 tag on the official TensorRT open source repository.

To install the open source components in the container, run the following script:

/opt/tensorrt/install_opensource.sh

For more information, see GitHub: TensorRT 22.05.

Limitations

Known Issues

None.

13. TensorRT Release 22.04

The NVIDIA container image for TensorRT, release 22.04, is available on NGC.

Contents of the TensorRT container

This container includes the following:
  • The TensorRT C++ samples and C++ API documentation.
    • Build the samples can be by running make in the /workspace/tensorrt/samples directory.
    • The resulting executables are in the /workspace/tensorrt/bin directory.
    • The C++ API documentation is in the /workspace/tensorrt/doc/cpp directory.
  • The TensorRT Python samples and Python API documentation.
    • The Python samples are in the /workspace/tensorrt/samples/python directory.

      Refer to the respective README documents for more samples.

    • Many Python samples can be run by using python <script.py> -d /workspace/tensorrt/data.
      For example:
      python onnx_resnet50.py -d /workspace/tensorrt/data
    • The Python API documentation is in the /workspace/tensorrt/doc/python directory.
  • TensorRT 8.2.4.2.
    The ONNX parser and plug-in libraries that are bundled with this container are built from TensorRT Open Source Software:
Here are the major updates to the 22.04 TensorRT Open Source Software release:
  • Bug fixes and refactored the PyramidROIAlign plugin.
  • Fixed the MultilevelCropAndResize plugin crashes on Windows.
  • Added a Detectron2 Mask R-CNN R50-FPN Python sample.
  • Removed sampleNMT.
The container also includes the following:

Driver Requirements

Release 22.04 is based on CUDA 11.6.2, which requires NVIDIA Driver release 510 or later. However, if you are running on a Data Center GPU (for example, T4 or any other Tesla board), use NVIDIA driver release 418.40 (or later R418), 440.33 (or later R440), 450.51 (or later R450), 460.27 (or later R460), or 470.57 (or later R470). The CUDA driver's compatibility package only supports particular drivers. For a complete list of supported drivers, see CUDA Application Compatibility. For more information, see CUDA Compatibility and Upgrades and NVIDIA CUDA and Drivers Support.

GPU Requirements

Release 22.04 supports CUDA compute capability 3.5 and later. This corresponds to GPUs in the Kepler, Maxwell, Pascal, Volta, Turing, and NVIDIA Ampere Architecture GPU families. For a list of GPUs to which this compute capability corresponds, see CUDA GPUs. For additional support details, see the Deep Learning Frameworks Support Matrix.

Key Features and Enhancements

This TensorRT container release includes the following key features and enhancements.
  • TensorRT container image version 22.04 is based on TensorRT 8.2.4.2.

    For a list of the features and enhancements that were introduced in TensorRT 8.2.4.2, refer to the TensorRT 8.2.4.2 release notes.

  • Ubuntu 20.04 with March 2022 updates.

Announcements

  • Starting with the 21.12 release, a beta version of the TensorRT container is available for the Arm SBSA platform.

    For example, when you pull the nvcr.io/nvidia/tensorrt:22.04-py3 Docker image on an Arm SBSA machine, the Arm-specific image is automatically fetched.

  • NVIDIA Deep Learning Profiler (DLProf) v1.8, which was included in the 21.12 container, was the last release of DLProf.

    Starting with the 22.01 container, DLProf is longer included. It can still be manually installed by using a pip wheel on the nvidia-pyindex.

Obtaining Missing Data Files

Some samples require data files that are not included in the TensorRT container because of licensing restrictions, or because they are too large. Samples that do not include the required data files include a README.md file in the corresponding source directory that provides information about how to obtain the necessary data files.

Installing Required Python Modules

  • To complete some of the samples, you might want to first run the Python setup script.
  • If you need to install the missing Python modules and their dependencies, run the /opt/tensorrt/python/python_setup.sh script.

Installing Open Source Components

A script has been added to clone, build, and replace the provided plug-in, the Caffe parser, and the ONNX parser libraries with the open source ones that are based on the 22.04 tag on the official TensorRT open source repository.

To install the open source components in the container, run the following script:

/opt/tensorrt/install_opensource.sh

For more information, see GitHub: TensorRT 22.04.

Limitations

Known Issues

None.

14. TensorRT Release 22.03

The NVIDIA container image for TensorRT, release 22.03, is available on NGC.

Contents of the TensorRT container

This container includes the following:
  • The TensorRT C++ samples and C++ API documentation.
    • Build the samples can be by running make in the /workspace/tensorrt/samples directory.
    • The resulting executables are in the /workspace/tensorrt/bin directory.
    • The C++ API documentation is in the /workspace/tensorrt/doc/cpp directory.
  • The TensorRT Python samples and Python API documentation.
    • The Python samples are in the /workspace/tensorrt/samples/python directory.

      Refer to the respective README documents for more samples.

    • Many Python samples can be run by using python <script.py> -d /workspace/tensorrt/data.
      For example:
      python onnx_resnet50.py -d /workspace/tensorrt/data
    • The Python API documentation is in the /workspace/tensorrt/doc/python directory.
  • TensorRT 8.2.3.
    The ONNX parser and plug-in libraries that are bundled with this container are built from TensorRT Open Source Software:
The container also includes the following:

Driver Requirements

Release 22.03 is based on CUDA 11.6.1, which requires NVIDIA Driver release 510 or later. However, if you are running on a Data Center GPU (for example, T4 or any other Tesla board), use NVIDIA driver release 418.40 (or later R418), 440.33 (or later R440), 450.51 (or later R450), 460.27 (or later R460), or 470.57 (or later R470). The CUDA driver's compatibility package only supports particular drivers. For a complete list of supported drivers, see CUDA Application Compatibility. For more information, see CUDA Compatibility and Upgrades and NVIDIA CUDA and Drivers Support.

GPU Requirements

Release 22.03 supports CUDA compute capability 3.5 and later. This corresponds to GPUs in the Kepler, Maxwell, Pascal, Volta, Turing, and NVIDIA Ampere Architecture GPU families. For a list of GPUs to which this compute capability corresponds, see CUDA GPUs. For additional support details, see the Deep Learning Frameworks Support Matrix.

Key Features and Enhancements

This TensorRT container release includes the following key features and enhancements.
  • TensorRT container image version 22.03 is based on TensorRT 8.2.3.

    For a list of the features and enhancements that were introduced in TensorRT 8.2.3, refer to the TensorRT 8.2.3 release notes.

  • Ubuntu 20.04 with February 2202 updates.

Announcements

  • Starting with the 21.12 release, a beta version of the TensorRT container is available for the Arm SBSA platform.

    For example, when you pull the Docker image nvcr.io/nvidia/tensorrt:22.03-py3 on an Arm SBSA machine will automatically fetch the Arm-specific image.

  • NVIDIA Deep Learning Profiler (DLProf) v1.8, which was included in the 21.12 container, was the last release of DLProf.

    Starting with the 22.01 container, DLProf is longer included. It can still be manually installed by using a pip wheel on the nvidia-pyindex.

Obtaining Missing Data Files

Some samples require data files that are not included in the TensorRT container because of licensing restrictions, or because they are too large. Samples that do not include the required data files include a README.md file in the corresponding source directory that provides information about how to obtain the necessary data files.

Installing Required Python Modules

  • To complete some of the samples, you might want to first run the Python setup script.
  • If you need to install the missing Python modules and their dependencies, run the /opt/tensorrt/python/python_setup.sh script.

Installing Open Source Components

A script has been added to clone, build, and replace the provided plug-in, the Caffe parser, and the ONNX parser libraries with the open source ones that are based on the 22.03 tag on the official TensorRT open source repository.

To install the open source components in the container, run the following script:

/opt/tensorrt/install_opensource.sh

For more information, see GitHub: TensorRT 22.03.

Limitations

Known Issues

None.

15. TensorRT Release 22.02

The NVIDIA container image for TensorRT, release 22.02, is available on NGC.

Contents of the TensorRT container

This container includes the following:
  • The TensorRT C++ samples and C++ API documentation. The samples can be built by running make in the /workspace/tensorrt/samples directory. The resulting executables are in the /workspace/tensorrt/bin directory. The C++ API documentation can be found in the /workspace/tensorrt/doc/cpp directory.
  • The TensorRT Python samples and Python API documentation. The Python samples can be found in the /workspace/tensorrt/samples/python directory. Refer to the respective README documents for more samples. Many Python samples can be run using python <script.py> -d /workspace/tensorrt/data. For example:
    python onnx_resnet50.py -d /workspace/tensorrt/data
    The Python API documentation can be found in the /workspace/tensorrt/doc/python directory.
  • TensorRT 8.2.3. Note that the ONNX parser and plugin libraries bundled with this container are built from TensorRT Open Source Software: https://github.com/NVIDIA/TensorRT/releases/tag/22.02https://github.com/NVIDIA/TensorRT/releases/tag/22.02.
The container also includes the following:

Driver Requirements

Release 22.02 is based on NVIDIA CUDA 11.6.0, which requires NVIDIA Driver release 510 or later. However, if you are running on a Data Center GPU (for example, T4 or any other Tesla board), you may use NVIDIA driver release 418.40 (or later R418), 440.33 (or later R440), 450.51 (or later R450), 460.27 (or later R460), or 470.57 (or later R470). The CUDA driver's compatibility package only supports particular drivers. For a complete list of supported drivers, see the CUDA Application Compatibility topic. For more information, see CUDA Compatibility and Upgrades and NVIDIA CUDA and Drivers Support.

GPU Requirements

Release 22.02 supports CUDA compute capability 3.5 and higher. This corresponds to GPUs in the Kepler, Maxwell, Pascal, Volta, Turing, and NVIDIA Ampere Architecture GPU families. Specifically, for a list of GPUs that this compute capability corresponds to, see CUDA GPUs. For additional support details, see Deep Learning Frameworks Support Matrix.

Key Features and Enhancements

This TensorRT container release includes the following key features and enhancements.
  • TensorRT container image version 22.02 is based on TensorRT 8.2.3. For a list of the new features and enhancements introduced in TensorRT 8.2.3 refer to the TensorRT 8.2.3 release notes.
  • Ubuntu 20.04 with January 2022 updates.

Announcements

  • Starting with the 21.12 release, a beta version of the TensorRT container is available for the ARM SBSA platform. For example, pulling the Docker image nvcr.io/nvidia/tensorrt:22.02-py3 on an ARM SBSA machine will automatically fetch the ARM-specific image.
  • DLProf v1.8, which was included in the 21.12 container, was the last release of DLProf. Starting with the 22.01 container, DLProf is longer included. It can still be manually installed via a pip wheel on the nvidia-pyindex.

Obtaining Missing Data Files

Some samples require data files that are not included within the TensorRT container either due to licensing restrictions or because they are too large. Samples which do not include all the required data files include a README.md file in the corresponding source directory informing you how to obtain the necessary data files.

Installing Required Python Modules

You may need to first run the Python setup script in order to complete some of the samples. The following script has been added to the container to install the missing Python modules and their dependencies if desired: /opt/tensorrt/python/python_setup.sh

Installing Open Source Components

A script has been added to clone, build and replace the provided plugin, Caffe parser, and ONNX parser libraries with the open source ones based off the 21.10 tag on the official TensorRT open source repository.

To install the open source components inside the container, run the following script:

/opt/tensorrt/install_opensource.sh

For more information see GitHub: TensorRT 22.02.

Limitations

Known Issues

  • None.

16. TensorRT Release 22.01

The NVIDIA container image for TensorRT, release 22.01, is available on NGC.

Contents of the TensorRT container

This container includes the following:
  • The TensorRT C++ samples and C++ API documentation. The samples can be built by running make in the /workspace/tensorrt/samples directory. The resulting executables are in the /workspace/tensorrt/bin directory. The C++ API documentation can be found in the /workspace/tensorrt/doc/cpp directory.
  • The TensorRT Python samples and Python API documentation. The Python samples can be found in the /workspace/tensorrt/samples/python directory. Refer to the respective README documents for more samples. Many Python samples can be run using python <script.py> -d /workspace/tensorrt/data. For example:
    python onnx_resnet50.py -d /workspace/tensorrt/data
    The Python API documentation can be found in the /workspace/tensorrt/doc/python directory.
  • TensorRT 8.2.2. Note that the ONNX parser and plugin libraries bundled with this container are built from TensorRT Open Source Software: https://github.com/NVIDIA/TensorRT/releases/tag/22.01.
The container also includes the following:

Driver Requirements

Release 22.01 is based on NVIDIA CUDA 11.6.0, which requires NVIDIA Driver release 510 or later. However, if you are running on a Data Center GPU (for example, T4 or any other Tesla board), you may use NVIDIA driver release 418.40 (or later R418), 440.33 (or later R440), 450.51 (or later R450), 460.27 (or later R460), or 470.57 (or later R470). The CUDA driver's compatibility package only supports particular drivers. For a complete list of supported drivers, see the CUDA Application Compatibility topic. For more information, see CUDA Compatibility and Upgrades and NVIDIA CUDA and Drivers Support.

GPU Requirements

Release 22.01 supports CUDA compute capability 3.5 and higher. This corresponds to GPUs in the Kepler, Maxwell, Pascal, Volta, Turing, and NVIDIA Ampere Architecture GPU families. Specifically, for a list of GPUs that this compute capability corresponds to, see CUDA GPUs. For additional support details, see Deep Learning Frameworks Support Matrix.

Key Features and Enhancements

This TensorRT container release includes the following key features and enhancements.
  • TensorRT container image version 22.01 is based on TensorRT 8.2.2. For a list of the new features and enhancements introduced in TensorRT 8.2.2 refer to the TensorRT 8.2.2 release notes.
  • Ubuntu 20.04 with December 2021 updates.

Announcements

  • Starting with the 21.12 release, a beta version of the TensorRT container is available for the ARM SBSA platform. For example, pulling the Docker image nvcr.io/nvidia/tensorrt:22.01-py3 on an ARM SBSA machine will automatically fetch the ARM-specific image.
  • DLProf v1.8, which was included in the 21.12 container, was the last release of DLProf. Starting with the 22.01 container, DLProf is longer included. It can still be manually installed via a pip wheel on the nvidia-pyindex.

Obtaining Missing Data Files

Some samples require data files that are not included within the TensorRT container either due to licensing restrictions or because they are too large. Samples which do not include all the required data files include a README.md file in the corresponding source directory informing you how to obtain the necessary data files.

Installing Required Python Modules

You may need to first run the Python setup script in order to complete some of the samples. The following script has been added to the container to install the missing Python modules and their dependencies if desired: /opt/tensorrt/python/python_setup.sh

Installing Open Source Components

A script has been added to clone, build and replace the provided plugin, Caffe parser, and ONNX parser libraries with the open source ones based off the 21.10 tag on the official TensorRT open source repository.

To install the open source components inside the container, run the following script:

/opt/tensorrt/install_opensource.sh

For more information see GitHub: TensorRT 22.01.

Limitations

Known Issues

  • None.

17. TensorRT Release 21.12

The NVIDIA container image for TensorRT, release 21.12, is available on NGC.

Contents of the TensorRT container

This container includes the following:
  • The TensorRT C++ samples and C++ API documentation. The samples can be built by running make in the /workspace/tensorrt/samples directory. The resulting executables are in the /workspace/tensorrt/bin directory. The C++ API documentation can be found in the /workspace/tensorrt/doc/cpp directory.
  • The TensorRT Python samples and Python API documentation. The Python samples can be found in the /workspace/tensorrt/samples/python directory. Refer to the respective README documents for more samples. Many Python samples can be run using python <script.py> -d /workspace/tensorrt/data. For example:
    python onnx_resnet50.py -d /workspace/tensorrt/data
    The Python API documentation can be found in the /workspace/tensorrt/doc/python directory.
  • TensorRT 8.2.1.8. Note that the ONNX parser and plugin libraries bundled with this container are built from TensorRT Open Source Software: https://github.com/NVIDIA/TensorRT/releases/.
The container also includes the following:

Driver Requirements

Release 21.12 is based on NVIDIA CUDA 11.5.0, which requires NVIDIA Driver release 495 or later. However, if you are running on a Data Center GPU (for example, T4 or any other Tesla board), you may use NVIDIA driver release 418.40 (or later R418), 440.33 (or later R440), 450.51 (or later R450), 460.27 (or later R460), or 470.57 (or later R470). The CUDA driver's compatibility package only supports particular drivers. For a complete list of supported drivers, see the CUDA Application Compatibility topic. For more information, see CUDA Compatibility and Upgrades and NVIDIA CUDA and Drivers Support.

GPU Requirements

Release 21.12 supports CUDA compute capability 3.5 and higher. This corresponds to GPUs in the Kepler, Maxwell, Pascal, Volta, Turing, and NVIDIA Ampere Architecture GPU families. Specifically, for a list of GPUs that this compute capability corresponds to, see CUDA GPUs. For additional support details, see Deep Learning Frameworks Support Matrix.

Key Features and Enhancements

This TensorRT container release includes the following key features and enhancements.
  • TensorRT container image version 21.12 is based on TensorRT 8.2.1.8. For a list of the new features and enhancements introduced in TensorRT 8.2.1 refer to the TensorRT 8.2.1 release notes.
  • Ubuntu 20.04 with November 2021 updates.

Announcements

  • Starting with the 21.12 release, a beta version of the TensorRT container is available for the ARM SBSA platform. Pulling the Docker image nvcr.io/nvidia/tensorrt:21.12-py3 on an ARM SBSA machine will automatically fetch the ARM-specific image.
  • DLProf v1.8, which is included in the 21.12 container, will be the last release of DLProf. Starting with the 22.01 container, DLProf will no longer be included. It can still be manually installed via a pip wheel on the nvidia-pyindex.

Obtaining Missing Data Files

Some samples require data files that are not included within the TensorRT container either due to licensing restrictions or because they are too large. Samples which do not include all the required data files include a README.md file in the corresponding source directory informing you how to obtain the necessary data files.

Installing Required Python Modules

You may need to first run the Python setup script in order to complete some of the samples. The following script has been added to the container to install the missing Python modules and their dependencies if desired: /opt/tensorrt/python/python_setup.sh

Installing Open Source Components

A script has been added to clone, build and replace the provided plugin, Caffe parser, and ONNX parser libraries with the open source ones based off the 21.10 tag on the official TensorRT open source repository.

To install the open source components inside the container, run the following script:

/opt/tensorrt/install_opensource.sh

For more information see GitHub: TensorRT 21.12.

Limitations

Known Issues

  • None.

18. TensorRT Release 21.11

The NVIDIA container image for TensorRT, release 21.11, is available on NGC.

Contents of the TensorRT container

This container includes the following:
  • The TensorRT C++ samples and C++ API documentation. The samples can be built by running make in the /workspace/tensorrt/samples directory. The resulting executables are in the /workspace/tensorrt/bin directory. The C++ API documentation can be found in the /workspace/tensorrt/doc/cpp directory.
  • The TensorRT Python samples and Python API documentation. The Python samples can be found in the /workspace/tensorrt/samples/python directory. Refer to the respective README documents for more samples. Many Python samples can be run using python <script.py> -d /workspace/tensorrt/data. For example:
    python onnx_resnet50.py -d /workspace/tensorrt/data
    The Python API documentation can be found in the /workspace/tensorrt/doc/python directory.
  • TensorRT 8.0.3.4. Note that the ONNX parser and plugin libraries bundled with this container are built from TensorRT Open Source Software: https://github.com/NVIDIA/TensorRT/releases/.
The container also includes the following:

Driver Requirements

Release 21.11 is based on NVIDIA CUDA 11.5.0, which requires NVIDIA Driver release 495 or later. However, if you are running on a Data Center GPU (for example, T4 or any other Tesla board), you may use NVIDIA driver release 418.40 (or later R418), 440.33 (or later R440), 450.51 (or later R450), 460.27 (or later R460), or 470.57 (or later R470). The CUDA driver's compatibility package only supports particular drivers. For a complete list of supported drivers, see the CUDA Application Compatibility topic. For more information, see CUDA Compatibility and Upgrades and NVIDIA CUDA and Drivers Support.

GPU Requirements

Release 21.11 supports CUDA compute capability 3.5 and higher. This corresponds to GPUs in the Kepler, Maxwell, Pascal, Volta, Turing, and NVIDIA Ampere Architecture GPU families. Specifically, for a list of GPUs that this compute capability corresponds to, see CUDA GPUs. For additional support details, see Deep Learning Frameworks Support Matrix.

Key Features and Enhancements

This TensorRT container release includes the following key features and enhancements.
  • TensorRT container image version 21.11 is based on TensorRT 8.0.3.4. For a list of the new features and enhancements introduced in TensorRT 8.0.3 refer to the TensorRT 8.0.3 release notes.
  • Ubuntu 20.04 with October 2021 updates.

Announcements

DLProf v1.8, which will be included in the 21.12 container, will be the last release of DLProf. Starting with the 22.01 container, DLProf will no longer be included. It can still be manually installed via a pip wheel on the nvidia-pyindex.

Obtaining Missing Data Files

Some samples require data files that are not included within the TensorRT container either due to licensing restrictions or because they are too large. Samples which do not include all the required data files include a README.md file in the corresponding source directory informing you how to obtain the necessary data files.

Installing Required Python Modules

You may need to first run the Python setup script in order to complete some of the samples. The following script has been added to the container to install the missing Python modules and their dependencies if desired: /opt/tensorrt/python/python_setup.sh

Installing Open Source Components

A script has been added to clone, build and replace the provided plugin, Caffe parser, and ONNX parser libraries with the open source ones based off the 21.10 tag on the official TensorRT open source repository.

To install the open source components inside the container, run the following script:

/opt/tensorrt/install_opensource.sh

For more information see GitHub: TensorRT 21.11.

Limitations

Known Issues

  • None.

19. TensorRT Release 21.10

The NVIDIA container image for TensorRT, release 21.10, is available on NGC.

Contents of the TensorRT container

This container includes the following:
  • The TensorRT C++ samples and C++ API documentation. The samples can be built by running make in the /workspace/tensorrt/samples directory. The resulting executables are in the /workspace/tensorrt/bin directory. The C++ API documentation can be found in the /workspace/tensorrt/doc/cpp directory.
  • The TensorRT Python samples and Python API documentation. The Python samples can be found in the /workspace/tensorrt/samples/python directory. Refer to the respective README documents for more samples. Many Python samples can be run using python <script.py> -d /workspace/tensorrt/data. For example:
    python onnx_resnet50.py -d /workspace/tensorrt/data
    The Python API documentation can be found in the /workspace/tensorrt/doc/python directory.
  • TensorRT 8.0.3.4. Note that the ONNX parser and plugin libraries bundled with this container are built from TensorRT Open Source Software: https://github.com/NVIDIA/TensorRT/releases/tag/21.10. Prominent updates to the 21.10 TensorRT Open Source Software release are:
    • Bump TensorRT version to 8.0.3.4
    • demo/BERT enhancements:
      • Added benchmark script for demoBERT-Megatron
      • Use static shape for single batch single sequence inputs
      • Revert to using native FC layer and FCPlugin only for older GPUs
    • Plugin enhancements:
      • Dynamic Input Shape support for EfficientNMS plugin
    • ONNX support enhancements:
      • Update ONNX submodule to v1.8.0
      • Support empty dimensions in ONNX
      • Several bugfixes and documentation updates

    • Updates to TensorRT developer tools:
      • Polygraphy v0.33.0
        • Added various examples, a CLI User Guide and how-to guides.
        • Added experimental support for DLA
        • Added a PluginRefRunner which provides CPU reference implementations for TensorRT plugins
    • Bugfixes and documentation updates in pytorch-quantization toolkit.
The container also includes the following:

Driver Requirements

Release 21.10 is based on NVIDIA CUDA 11.4.2 with cuBLAS 11.6.5.2, which requires NVIDIA Driver release 470 or later. However, if you are running on Data Center GPUs (formerly Tesla), for example, T4, you may use NVIDIA driver release 418.40 (or later R418), 440.33 (or later R440), 450.51 (or later R450), or 460.27 (or later R460). The CUDA driver's compatibility package only supports particular drivers. For a complete list of supported drivers, see the CUDA Application Compatibility topic. For more information, see CUDA Compatibility and Upgrades and NVIDIA CUDA and Drivers Support.

GPU Requirements

Release 21.10 supports CUDA compute capability 3.5 and higher. This corresponds to GPUs in the Kepler, Maxwell, Pascal, Volta, Turing, and NVIDIA Ampere Architecture GPU families. Specifically, for a list of GPUs that this compute capability corresponds to, see CUDA GPUs. For additional support details, see Deep Learning Frameworks Support Matrix.

Key Features and Enhancements

This TensorRT container release includes the following key features and enhancements.
  • TensorRT container image version 21.10 is based on TensorRT 8.0.3.4. For a list of the new features and enhancements introduced in TensorRT 8.0.3 refer to the TensorRT 8.0.3 release notes.
  • Ubuntu 20.04 with September 2021 updates.

Obtaining Missing Data Files

Some samples require data files that are not included within the TensorRT container either due to licensing restrictions or because they are too large. Samples which do not include all the required data files include a README.md file in the corresponding source directory informing you how to obtain the necessary data files.

Installing Required Python Modules

You may need to first run the Python setup script in order to complete some of the samples. The following script has been added to the container to install the missing Python modules and their dependencies if desired: /opt/tensorrt/python/python_setup.sh

Installing Open Source Components

A script has been added to clone, build and replace the provided plugin, Caffe parser, and ONNX parser libraries with the open source ones based off the 21.10 tag on the official TensorRT open source repository.

To install the open source components inside the container, run the following script:

/opt/tensorrt/install_opensource.sh

For more information see GitHub: TensorRT 21.10.

Limitations

Known Issues

  • None.

20. TensorRT Release 21.09

The NVIDIA container image for TensorRT, release 21.09, is available on NGC.

Contents of the TensorRT container

This container includes the following:
  • The TensorRT C++ samples and C++ API documentation. The samples can be built by running make in the /workspace/tensorrt/samples directory. The resulting executables are in the /workspace/tensorrt/bin directory. The C++ API documentation can be found in the /workspace/tensorrt/doc/cpp directory.
  • The TensorRT Python samples and Python API documentation. The Python samples can be found in the /workspace/tensorrt/samples/python directory. Refer to the respective README documents for more samples. Many Python samples can be run using python <script.py> -d /workspace/tensorrt/data. For example:
    python onnx_resnet50.py -d /workspace/tensorrt/data
    The Python API documentation can be found in the /workspace/tensorrt/doc/python directory.
  • TensorRT 8.0.3. Note that the ONNX parser and plugin libraries bundled with this container are built from TensorRT Open Source Software: https://github.com/NVIDIA/TensorRT/releases/tag/21.09. Prominent updates to the 21.08 TensorRT Open Source Software release are:
The container also includes the following:

Driver Requirements

Release 21.09 is based on NVIDIA CUDA 11.4.2, which requires NVIDIA Driver release 470 or later. However, if you are running on Data Center GPUs (formerly Tesla), for example, T4, you may use NVIDIA driver release 418.40 (or later R418), 440.33 (or later R440), 450.51 (or later R450), or 460.27 (or later R460). The CUDA driver's compatibility package only supports particular drivers. For a complete list of supported drivers, see the CUDA Application Compatibility topic. For more information, see CUDA Compatibility and Upgrades and NVIDIA CUDA and Drivers Support.

GPU Requirements

Release 21.09 supports CUDA compute capability 3.5 and higher. This corresponds to GPUs in the Kepler, Maxwell, Pascal, Volta, Turing, and NVIDIA Ampere Architecture GPU families. Specifically, for a list of GPUs that this compute capability corresponds to, see CUDA GPUs. For additional support details, see Deep Learning Frameworks Support Matrix.

Key Features and Enhancements

This TensorRT container release includes the following key features and enhancements.
  • TensorRT container image version 21.09 is based on TensorRT 8.0.3. For a list of the new features and enhancements introduced in TensorRT 8.0.3 refer to the TensorRT 8.0.3 release notes.
  • Ubuntu 20.04 with August 2021 updates.

Obtaining Missing Data Files

Some samples require data files that are not included within the TensorRT container either due to licensing restrictions or because they are too large. Samples which do not include all the required data files include a README.md file in the corresponding source directory informing you how to obtain the necessary data files.

Installing Required Python Modules

You may need to first run the Python setup script in order to complete some of the samples. The following script has been added to the container to install the missing Python modules and their dependencies if desired: /opt/tensorrt/python/python_setup.sh

Installing Open Source Components

A script has been added to clone, build and replace the provided plugin, Caffe parser, and ONNX parser libraries with the open source ones based off the 21.09 tag on the official TensorRT open source repository.

To install the open source components inside the container, run the following script:

/opt/tensorrt/install_opensource.sh

For more information see GitHub: TensorRT 21.09.

Limitations

Known Issues

  • None.

21. TensorRT Release 21.08

The NVIDIA container image for TensorRT, release 21.08, is available on NGC.

Contents of the TensorRT container

This container includes the following:
  • The TensorRT C++ samples and C++ API documentation. The samples can be built by running make in the /workspace/tensorrt/samples directory. The resulting executables are in the /workspace/tensorrt/bin directory. The C++ API documentation can be found in the /workspace/tensorrt/doc/cpp directory.
  • The TensorRT Python samples and Python API documentation. The Python samples can be found in the /workspace/tensorrt/samples/python directory. Refer to the respective README documents for more samples. Many Python samples can be run using python <script.py> -d /workspace/tensorrt/data. For example:
    python onnx_resnet50.py -d /workspace/tensorrt/data
    The Python API documentation can be found in the /workspace/tensorrt/doc/python directory.
  • TensorRT 8.0.1.6. Note that the ONNX parser and plugin libraries bundled with this container are built from TensorRT Open Source Software: https://github.com/NVIDIA/TensorRT/releases/tag/21.08. Prominent updates to the 21.08 TensorRT Open Source Software release are:
The container also includes the following:

Driver Requirements

Release 21.08 is based on NVIDIA CUDA 11.4.1, which requires NVIDIA Driver release 470 or later. However, if you are running on Data Center GPUs (formerly Tesla), for example, T4, you may use NVIDIA driver release 418.40 (or later R418), 440.33 (or later R440), 450.51 (or later R450), or 460.27 (or later R460). The CUDA driver's compatibility package only supports particular drivers. For a complete list of supported drivers, see the CUDA Application Compatibility topic. For more information, see CUDA Compatibility and Upgrades and NVIDIA CUDA and Drivers Support.

GPU Requirements

Release 21.08 supports CUDA compute capability 3.5 and higher. This corresponds to GPUs in the Kepler, Maxwell, Pascal, Volta, Turing, and NVIDIA Ampere Architecture GPU families. Specifically, for a list of GPUs that this compute capability corresponds to, see CUDA GPUs. For additional support details, see Deep Learning Frameworks Support Matrix.

Key Features and Enhancements

This TensorRT container release includes the following key features and enhancements.
  • TensorRT container image version 21.08 is based on TensorRT 8.0.1.6. For a list of the new features and enhancements introduced in TensorRT 8.0.1.6 refer to the TensorRT 8.0.1 release notes.
  • Ubuntu 20.04 with July 2021 updates.

Obtaining Missing Data Files

Some samples require data files that are not included within the TensorRT container either due to licensing restrictions or because they are too large. Samples which do not include all the required data files include a README.md file in the corresponding source directory informing you how to obtain the necessary data files.

Installing Required Python Modules

You may need to first run the Python setup script in order to complete some of the samples. The following script has been added to the container to install the missing Python modules and their dependencies if desired: /opt/tensorrt/python/python_setup.sh

Installing Open Source Components

A script has been added to clone, build and replace the provided plugin, Caffe parser, and ONNX parser libraries with the open source ones based off the 21.08 tag on the official TensorRT open source repository.

To install the open source components inside the container, run the following script:

/opt/tensorrt/install_opensource.sh

For more information see GitHub: TensorRT 21.08.

Limitations

Known Issues

  • None.

22. TensorRT Release 21.07

The NVIDIA container image for TensorRT, release 21.07, is available on NGC.

Contents of the TensorRT container

This container includes the following:
  • The TensorRT C++ samples and C++ API documentation. The samples can be built by running make in the /workspace/tensorrt/samples directory. The resulting executables are in the /workspace/tensorrt/bin directory. The C++ API documentation can be found in the /workspace/tensorrt/doc/cpp directory.
  • The TensorRT Python samples and Python API documentation. The Python samples can be found in the /workspace/tensorrt/samples/python directory. Many Python samples can be run using python <script.py> -d /workspace/tensorrt/data. For example:
    python caffe_resnet50.py -d /workspace/tensorrt/data
    The Python API documentation can be found in the /workspace/tensorrt/doc/python directory.
  • TensorRT 8.0.1.6. Note that the ONNX parser and plugin libraries bundled with this container are built from TensorRT Open Source Software: https://github.com/NVIDIA/TensorRT/releases/tag/21.07. Prominent updates to the 21.07 TensorRT Open Source Software release are:
    • Major upgrade to TensorRT 8.0.1.6 GA.
    • Added support for ONNX operators: Celu, CumSum, EyeLike, GatherElements, GlobalLpPool, GreaterOrEqual, LessOrEqual, LpNormalization, LpPool, ReverseSequence, and SoftmaxCrossEntropyLoss.
    • Enhanced support for ONNX operators: Resize, ConvTranspose, InstanceNormalization, QuantizeLinear, DequantizeLinear, Pad.
    • Added new plugins: EfficientNMS_TRT, EfficientNMS_ONNX_TRT, ScatterND.
    • Added new samples: engine_refit_onnx_bidaf, efficientdet, efficientnet.
    • Added docker build support for Ubuntu20.04 and RedHat/CentOS 8.3.
    • Added Python 3.9 support.
    • Updates to ONNX tools: Polygraphy v0.30.3, ONNX-GraphSurgeon v0.3.10, Pytorch Quantization toolkit v2.1.0.
    • Removed IPlugin and IPluginFactory interfaces.
    • Removed samples: samplePlugin, sampleMovieLens, sampleMovieLensMPS.
    • Removed docker build support for Ubuntu 16.04, and PowerPC.
The container also includes the following:

Driver Requirements

Release 21.07 is based on NVIDIA CUDA 11.4.0, which requires NVIDIA Driver release 470 or later. However, if you are running on Data Center GPUs (formerly Tesla), for example, T4, you may use NVIDIA driver release 418.40 (or later R418), 440.33 (or later R440), 450.51 (or later R450), or 460.27 (or later R460). The CUDA driver's compatibility package only supports particular drivers. For a complete list of supported drivers, see the CUDA Application Compatibility topic. For more information, see CUDA Compatibility and Upgrades and NVIDIA CUDA and Drivers Support.

GPU Requirements

Release 21.07 supports CUDA compute capability 3.5 and higher. This corresponds to GPUs in the Kepler, Maxwell, Pascal, Volta, Turing, and NVIDIA Ampere Architecture GPU families. Specifically, for a list of GPUs that this compute capability corresponds to, see CUDA GPUs. For additional support details, see Deep Learning Frameworks Support Matrix.

Key Features and Enhancements

This TensorRT container release includes the following key features and enhancements.
  • TensorRT container image version 21.07 is based on TensorRT 8.0.1.6. For a list of the new features and enhancements introduced in TensorRT 8.0.1.6 refer to the TensorRT 8.0.1 release notes.
  • Ubuntu 20.04 with June 2021 updates.

Obtaining Missing Data Files

Some samples require data files that are not included within the TensorRT container either due to licensing restrictions or because they are too large. Samples which do not include all the required data files include a README.md file in the corresponding source directory informing you how to obtain the necessary data files.

Installing Required Python Modules

You may need to first run the Python setup script in order to complete some of the samples. The following script has been added to the container to install the missing Python modules and their dependencies if desired: /opt/tensorrt/python/python_setup.sh

Installing Open Source Components

A script has been added to clone, build and replace the provided plugin, Caffe parser, and ONNX parser libraries with the open source ones based off the 21.07 tag on the official TensorRT open source repository.

To install the open source components inside the container, run the following script:

/opt/tensorrt/install_opensource.sh

For more information see GitHub: TensorRT 21.07.

Limitations

Known Issues

  • The 21.07 release includes libsystemd and libudev versions that have a known vulnerability that was discovered late in our QA process. See CVE-2021-33910 for details. This will be fixed in the next release.

23. TensorRT Release 21.06

The NVIDIA container image for TensorRT, release 21.06, is available on NGC.

Contents of the TensorRT container

This container includes the following:
  • The TensorRT C++ samples and C++ API documentation. The samples can be built by running make in the /workspace/tensorrt/samples directory. The resulting executables are in the /workspace/tensorrt/bin directory. The C++ API documentation can be found in the /workspace/tensorrt/doc/cpp directory.
  • The TensorRT Python samples and Python API documentation. The Python samples can be found in the /workspace/tensorrt/samples/python directory. Many Python samples can be run using python <script.py> -d /workspace/tensorrt/data. For example:
    python caffe_resnet50.py -d /workspace/tensorrt/data
    The Python API documentation can be found in the /workspace/tensorrt/doc/python directory.
  • TensorRT 7.2.3.4. Note that the ONNX parser and plugin libraries bundled with this container are built from TensorRT Open Source Software: https://github.com/NVIDIA/TensorRT/releases/tag/21.06
The container also includes the following:

Driver Requirements

Release 21.06 is based on NVIDIA CUDA 11.3.1, which requires NVIDIA Driver release 465.19.01 or later. However, if you are running on Data Center GPUs (formerly Tesla), for example, T4, you may use NVIDIA driver release 418.40 (or later R418), 440.33 (or later R440), 450.51 (or later R450), or 460.27 (or later R460). The CUDA driver's compatibility package only supports particular drivers. For a complete list of supported drivers, see the CUDA Application Compatibility topic. For more information, see CUDA Compatibility and Upgrades and NVIDIA CUDA and Drivers Support.

GPU Requirements

Release 21.06 supports CUDA compute capability 3.5 and higher. This corresponds to GPUs in the Kepler, Maxwell, Pascal, Volta, Turing, and NVIDIA Ampere Architecture GPU families. Specifically, for a list of GPUs that this compute capability corresponds to, see CUDA GPUs. For additional support details, see Deep Learning Frameworks Support Matrix.

Key Features and Enhancements

This TensorRT container release includes the following key features and enhancements.
  • TensorRT container image version 21.06 is based on TensorRT 7.2.3.4. For a list of the new features and enhancements introduced in TensorRT 7.2.3.4 refer to the TensorRT 7.2.3 release notes.
  • Added missing model.py in uff_custom_plugin sample.
  • Fixed numerical errors for float type in NMS/batchedNMS plugins.
  • Removed fcplugin from demoBERT to improve latency.
  • Ubuntu 20.04 with May 2021 updates.

Obtaining Missing Data Files

Some samples require data files that are not included within the TensorRT container either due to licensing restrictions or because they are too large. Samples which do not include all the required data files include a README.md file in the corresponding source directory informing you how to obtain the necessary data files.

Installing Required Python Modules

You may need to first run the Python setup script in order to complete some of the samples. The following script has been added to the container to install the missing Python modules and their dependencies if desired: /opt/tensorrt/python/python_setup.sh

Installing Open Source Components

A script has been added to clone, build and replace the provided plugin, Caffe parser, and ONNX parser libraries with the open source ones based off the 21.06 tag on the official TensorRT open source repository.

To install the open source components inside the container, run the following script:

/opt/tensorrt/install_opensource.sh

For more information see GitHub: TensorRT 21.06.

Limitations

Known Issues

There are no known issues in this release.

24. TensorRT Release 21.05

The NVIDIA container image for TensorRT, release 21.05, is available on NGC.

Contents of the TensorRT container

This container includes the following:
  • The TensorRT C++ samples and C++ API documentation. The samples can be built by running make in the /workspace/tensorrt/samples directory. The resulting executables are in the /workspace/tensorrt/bin directory. The C++ API documentation can be found in the /workspace/tensorrt/doc/cpp directory.
  • The TensorRT Python samples and Python API documentation. The Python samples can be found in the /workspace/tensorrt/samples/python directory. Many Python samples can be run using python <script.py> -d /workspace/tensorrt/data. For example:
    python caffe_resnet50.py -d /workspace/tensorrt/data
    The Python API documentation can be found in the /workspace/tensorrt/doc/python directory.
  • TensorRT 7.2.3.4. Note that the ONNX parser and plugin libraries bundled with this container are built from TensorRT Open Source Software: https://github.com/NVIDIA/TensorRT/releases/tag/21.05
    Prominent updates to the 21.04 TensorRT Open Source Software release are:
    • Addition of TensorRT Python API bindings.
    • Addition of TensorRT Python samples.
    • Plugin enhancements - FP16 support in batchedNMSPlugin, configurable input sizes for TLT MaskRCNN plugin.
    • ONNX opset13 updates, ResNet example, and documentation updates to PyTorch Quantization toolkit.
    • BERT demo updated to work with Tensorflow 2.x.
The container also includes the following:

Driver Requirements

Release 21.05 is based on NVIDIA CUDA 11.3.0, which requires NVIDIA Driver release 465.19.01 or later. However, if you are running on Data Center GPUs (formerly Tesla), for example, T4, you may use NVIDIA driver release 418.40 (or later R418), 440.33 (or later R440), 450.51 (or later R450), or 460.27 (or later R460). The CUDA driver's compatibility package only supports particular drivers. For a complete list of supported drivers, see the CUDA Application Compatibility topic. For more information, see CUDA Compatibility and Upgrades and NVIDIA CUDA and Drivers Support.

GPU Requirements

Release 21.05 supports CUDA compute capability 3.5 and higher. This corresponds to GPUs in the Kepler, Maxwell, Pascal, Volta, Turing, and NVIDIA Ampere Architecture GPU families. Specifically, for a list of GPUs that this compute capability corresponds to, see CUDA GPUs. For additional support details, see Deep Learning Frameworks Support Matrix.

Key Features and Enhancements

This TensorRT container release includes the following key features and enhancements.
  • TensorRT container image version 21.05 is based on TensorRT 7.2.3.4. For a list of the new features and enhancements introduced in TensorRT 7.2.3.4 refer to the TensorRT 7.2.3 release notes.
  • Ubuntu 20.04 with April 2021 updates

Obtaining Missing Data Files

Some samples require data files that are not included within the TensorRT container either due to licensing restrictions or because they are too large. Samples which do not include all the required data files include a README.md file in the corresponding source directory informing you how to obtain the necessary data files.

Installing Required Python Modules

You may need to first run the Python setup script in order to complete some of the samples. The following script has been added to the container to install the missing Python modules and their dependencies if desired: /opt/tensorrt/python/python_setup.sh

Installing Open Source Components

A script has been added to clone, build and replace the provided plugin, Caffe parser, and ONNX parser libraries with the open source ones based off the 21.05 tag on the official TensorRT open source repository.

To install the open source components inside the container, run the following script:

/opt/tensorrt/install_opensource.sh

For more information see GitHub: TensorRT 21.05.

Limitations

NVIDIA TensorRT Container Versions

The following table shows what versions of Ubuntu, CUDA, and TensorRT are supported in each of the NVIDIA containers for TensorRT. For older container versions, refer to the Frameworks Support Matrix.

Known Issues

There are no known issues in this release.

25. TensorRT Release 21.04

The NVIDIA container image for TensorRT, release 21.04, is available on NGC.

Contents of the TensorRT container

This container includes the following:
  • The TensorRT C++ samples and C++ API documentation. The samples can be built by running make in the /workspace/tensorrt/samples directory. The resulting executables are in the /workspace/tensorrt/bin directory. The C++ API documentation can be found in the /workspace/tensorrt/doc/cpp directory.
  • The TensorRT Python samples and Python API documentation. The Python samples can be found in the /workspace/tensorrt/samples/python directory. Many Python samples can be run using python <script.py> -d /workspace/tensorrt/data. For example:
    python caffe_resnet50.py -d /workspace/tensorrt/data
    The Python API documentation can be found in the /workspace/tensorrt/doc/python directory.
  • TensorRT 7.2.3.4. Note that the ONNX parser and plugin libraries bundled with this container are built from TensorRT Open Source Software: https://github.com/NVIDIA/TensorRT/releases/tag/21.04
    Prominent updates to the 21.04 TensorRT Open Source Software release are:
    • Addition of TensorRT Python API bindings.
    • Addition of TensorRT Python samples.
    • Plugin enhancements - FP16 support in batchedNMSPlugin, configurable input sizes for TLT MaskRCNN plugin.
    • ONNX opset13 updates, ResNet example, and documentation updates to PyTorch Quantization toolkit.
    • BERT demo updated to work with Tensorflow 2.x.
The container also includes the following:

Driver Requirements

Release 21.04 is based on NVIDIA CUDA 11.3.0, which requires NVIDIA Driver release 465.19.01 or later. However, if you are running on Data Center GPUs (formerly Tesla), for example, T4, you may use NVIDIA driver release 418.40 (or later R418), 440.33 (or later R440), 450.51 (or later R450), or 460.27 (or later R460). The CUDA driver's compatibility package only supports particular drivers. For a complete list of supported drivers, see the CUDA Application Compatibility topic. For more information, see CUDA Compatibility and Upgrades and NVIDIA CUDA and Drivers Support.

GPU Requirements

Release 21.04 supports CUDA compute capability 3.5 and higher. This corresponds to GPUs in the Kepler, Maxwell, Pascal, Volta, Turing, and NVIDIA Ampere Architecture GPU families. Specifically, for a list of GPUs that this compute capability corresponds to, see CUDA GPUs. For additional support details, see Deep Learning Frameworks Support Matrix.

Key Features and Enhancements

This TensorRT container release includes the following key features and enhancements.
  • TensorRT container image version 21.04 is based on TensorRT 7.2.3.4. For a list of the new features and enhancements introduced in TensorRT 7.2.3.4 refer to the TensorRT 7.2.3 release notes.
  • Ubuntu 20.04 with March 2021 updates

Obtaining Missing Data Files

Some samples require data files that are not included within the TensorRT container either due to licensing restrictions or because they are too large. Samples which do not include all the required data files include a README.md file in the corresponding source directory informing you how to obtain the necessary data files.

Installing Required Python Modules

You may need to first run the Python setup script in order to complete some of the samples. The following script has been added to the container to install the missing Python modules and their dependencies if desired: /opt/tensorrt/python/python_setup.sh

Installing Open Source Components

A script has been added to clone, build and replace the provided plugin, Caffe parser, and ONNX parser libraries with the open source ones based off the 21.04 tag on the official TensorRT open source repository.

To install the open source components inside the container, run the following script:

/opt/tensorrt/install_opensource.sh

For more information see GitHub: TensorRT 21.04.

Limitations

NVIDIA TensorRT Container Versions

The following table shows what versions of Ubuntu, CUDA, and TensorRT are supported in each of the NVIDIA containers for TensorRT. For older container versions, refer to the Frameworks Support Matrix.

Known Issues

There are no known issues in this release.

26. TensorRT Release 21.03

The NVIDIA container image for TensorRT, release 21.03, is available on NGC.

Contents of the TensorRT container

This container includes the following:
  • The TensorRT C++ samples and C++ API documentation. The samples can be built by running make in the /workspace/tensorrt/samples directory. The resulting executables are in the /workspace/tensorrt/bin directory. The C++ API documentation can be found in the /workspace/tensorrt/doc/cpp directory.
  • The TensorRT Python samples and Python API documentation. The Python samples can be found in the /workspace/tensorrt/samples/python directory. Many Python samples can be run using python <script.py> -d /workspace/tensorrt/data. For example:
    python caffe_resnet50.py -d /workspace/tensorrt/data
    The Python API documentation can be found in the /workspace/tensorrt/doc/python directory.
  • TensorRT 7.2.2.3. Note that the ONNX parser and plugin libraries bundled with this container are built from TensorRT Open Source Software: https://github.com/NVIDIA/TensorRT/releases/tag/21.03
    Prominent updates to the 21.03 TensorRT Open Source Software release are:
    • Addition of TensorRT Python API bindings.
    • Addition of TensorRT Python samples.
    • Plugin enhancements - FP16 support in batchedNMSPlugin, configurable input sizes for TLT MaskRCNN plugin.
    • ONNX opset13 updates, ResNet example, and documentation updates to PyTorch Quantization toolkit.
    • BERT demo updated to work with Tensorflow 2.x.
The container also includes the following:

Driver Requirements

Release 21.03 is based on NVIDIA CUDA 11.2.0, which requires NVIDIA Driver release 460.32.03 or later. However, if you are running on Data Center GPUs (formerly Tesla), for example, T4, you may use NVIDIA driver release 418.40 (or later R418), 440.33 (or later R440), 450.51(or later R450). The CUDA driver's compatibility package only supports particular drivers. For a complete list of supported drivers, see the CUDA Application Compatibility topic. For more information, see CUDA Compatibility and Upgrades and NVIDIA CUDA and Drivers Support.

GPU Requirements

Release 21.03 supports CUDA compute capability 3.5 and higher. This corresponds to GPUs in the Kepler, Maxwell, Pascal, Volta, Turing, and NVIDIA Ampere Architecture GPU families. Specifically, for a list of GPUs that this compute capability corresponds to, see CUDA GPUs. For additional support details, see Deep Learning Frameworks Support Matrix.

Key Features and Enhancements

This TensorRT container release includes the following key features and enhancements.

Announcements

  • Python 2.7 is no longer supported in this TensorRT container release.

Obtaining Missing Data Files

Some samples require data files that are not included within the TensorRT container either due to licensing restrictions or because they are too large. Samples which do not include all the required data files include a README.md file in the corresponding source directory informing you how to obtain the necessary data files.

Installing Required Python Modules

You may need to first run the Python setup script in order to complete some of the samples. The following script has been added to the container to install the missing Python modules and their dependencies if desired: /opt/tensorrt/python/python_setup.sh

Installing Open Source Components

A script has been added to clone, build and replace the provided plugin, Caffe parser, and ONNX parser libraries with the open source ones based off the 21.03 tag on the official TensorRT open source repository.

To install the open source components inside the container, run the following script:

/opt/tensorrt/install_opensource.sh

For more information see GitHub: TensorRT 21.03.

Limitations

NVIDIA TensorRT Container Versions

The following table shows what versions of Ubuntu, CUDA, and TensorRT are supported in each of the NVIDIA containers for TensorRT. For older container versions, refer to the Frameworks Support Matrix.

Known Issues

There are no known issues in this release.

27. TensorRT Release 21.02

The NVIDIA container image for TensorRT, release 21.02, is available on NGC.

Contents of the TensorRT container

This container includes the following:
  • The TensorRT C++ samples and C++ API documentation. The samples can be built by running make in the /workspace/tensorrt/samples directory. The resulting executables are in the /workspace/tensorrt/bin directory. The C++ API documentation can be found in the /workspace/tensorrt/doc/cpp directory.
  • The TensorRT Python samples and Python API documentation. The Python samples can be found in the /workspace/tensorrt/samples/python directory. Many Python samples can be run using python <script.py> -d /workspace/tensorrt/data. For example:
    python caffe_resnet50.py -d /workspace/tensorrt/data
    The Python API documentation can be found in the /workspace/tensorrt/doc/python directory.
  • 7.2.2.3+cuda11.1.0.024. Note that the ONNX parser and plugin libraries bundled with this container are built from TensorRT Open Source Software: https://github.com/NVIDIA/TensorRT/releases/tag/21.02
    Prominent updates to the 21.02 TensorRT Open Source Software release are:
    • Addition of TensorRT Python API bindings.
    • Addition of TensorRT Python samples.
    • Plugin enhancements - FP16 support in batchedNMSPlugin, configurable input sizes for TLT MaskRCNN plugin.
    • ONNX opset13 updates, ResNet example, and documentation updates to PyTorch Quantization toolkit.
    • BERT demo updated to work with Tensorflow 2.x.
The container also includes the following:

Driver Requirements

Release 21.02 is based on NVIDIA CUDA 11.2.0, which requires NVIDIA Driver release 460.27.04 or later. However, if you are running on Data Center GPUs (formerly Tesla), for example, T4, you may use NVIDIA driver release 418.40 (or later R418), 440.33 (or later R440), 450.51(or later R450). The CUDA driver's compatibility package only supports particular drivers. For a complete list of supported drivers, see the CUDA Application Compatibility topic. For more information, see CUDA Compatibility and Upgrades and NVIDIA CUDA and Drivers Support.

GPU Requirements

Release 21.02 supports CUDA compute capability 3.5 and higher. This corresponds to GPUs in the Kepler, Maxwell, Pascal, Volta, Turing, and NVIDIA Ampere Architecture GPU families. Specifically, for a list of GPUs that this compute capability corresponds to, see CUDA GPUs. For additional support details, see Deep Learning Frameworks Support Matrix.

Key Features and Enhancements

This TensorRT container release includes the following key features and enhancements.

Announcements

  • Python 2.7 is no longer supported in this TensorRT container release.

Obtaining Missing Data Files

Some samples require data files that are not included within the TensorRT container either due to licensing restrictions or because they are too large. Samples which do not include all the required data files include a README.md file in the corresponding source directory informing you how to obtain the necessary data files.

Installing Required Python Modules

You may need to first run the Python setup script in order to complete some of the samples. The following script has been added to the container to install the missing Python modules and their dependencies if desired: /opt/tensorrt/python/python_setup.sh

Installing Open Source Components

A script has been added to clone, build and replace the provided plugin, Caffe parser, and ONNX parser libraries with the open source ones based off the 21.02 tag on the official TensorRT open source repository.

To install the open source components inside the container, run the following script:

/opt/tensorrt/install_opensource.sh

For more information see GitHub: TensorRT 21.02.

Limitations

NVIDIA TensorRT Container Versions

The following table shows what versions of Ubuntu, CUDA, and TensorRT are supported in each of the NVIDIA containers for TensorRT. For older container versions, refer to the Frameworks Support Matrix.

Known Issues

There are no known issues in this release.

28. TensorRT Release 21.01

The NVIDIA container image release for TensorRT 21.01 has been canceled. The next release will be the 21.02 release which is expected to be released at the end of February.

29. TensorRT Release 20.12

The NVIDIA container image for TensorRT, release 20.12, is available on NGC.

Contents of the TensorRT container

This container includes the following:
  • The TensorRT C++ samples and C++ API documentation. The samples can be built by running make in the /workspace/tensorrt/samples directory. The resulting executables are in the /workspace/tensorrt/bin directory. The C++ API documentation can be found in the /workspace/tensorrt/doc/cpp directory.
  • The TensorRT Python samples and Python API documentation. The Python samples can be found in the /workspace/tensorrt/samples/python directory. Many Python samples can be run using python <script.py> -d /workspace/tensorrt/data. For example:
    python caffe_resnet50.py -d /workspace/tensorrt/data
    The Python API documentation can be found in the /workspace/tensorrt/doc/python directory.
  • TensorRT 7.2.2. Note that the ONNX parser and plugin libraries bundled with this container are built from TensorRT Open Source Software: https://github.com/NVIDIA/TensorRT/releases/tag/20.12
The container also includes the following:

Driver Requirements

Release 20.12 is based on NVIDIA CUDA 11.1.1, which requires NVIDIA Driver release 455 or later. However, if you are running on Tesla (for example, T4 or any other Tesla board), you may use NVIDIA driver release 418.xx, 440.30, or 450.xx. The CUDA driver's compatibility package only supports particular drivers. For a complete list of supported drivers, see the CUDA Application Compatibility topic. For more information, see CUDA Compatibility and Upgrades.

GPU Requirements

Release 20.12 supports CUDA compute capability 3.5 and higher. This corresponds to GPUs in the Kepler, Maxwell, Pascal, Volta, Turing, and Ampere Architecture GPU families. Specifically, for a list of GPUs that this compute capability corresponds to, see CUDA GPUs. For additional support details, see Deep Learning Frameworks Support Matrix.

Key Features and Enhancements

This TensorRT container release includes the following key features and enhancements.

Announcements

  • Python 2.7 is no longer supported in this TensorRT container release.

Obtaining Missing Data Files

Some samples require data files that are not included within the TensorRT container either due to licensing restrictions or because they are too large. Samples which do not include all the required data files include a README.md file in the corresponding source directory informing you how to obtain the necessary data files.

Installing Required Python Modules

You may need to first run the Python setup script in order to complete some of the samples. The following script has been added to the container to install the missing Python modules and their dependencies if desired: /opt/tensorrt/python/python_setup.sh

Installing Open Source Components

A script has been added to clone, build and replace the provided plugin, Caffe parser, and ONNX parser libraries with the open source ones based off the 20.12 tag on the official TensorRT open source repository.

To install the open source components inside the container, run the following script:

/opt/tensorrt/install_opensource.sh

For more information see GitHub: TensorRT 20.12.

Limitations

NVIDIA TensorRT Container Versions

The following table shows what versions of Ubuntu, CUDA, and TensorRT are supported in each of the NVIDIA containers for TensorRT. For older container versions, refer to the Frameworks Support Matrix.

Known Issues

There are no known issues in this release.

30. TensorRT Release 20.11

The NVIDIA container image for TensorRT, release 20.11, is available on NGC.

Contents of the TensorRT container

This container includes the following:
  • The TensorRT C++ samples and C++ API documentation. The samples can be built by running make in the /workspace/tensorrt/samples directory. The resulting executables are in the /workspace/tensorrt/bin directory. The C++ API documentation can be found in the /workspace/tensorrt/doc/cpp directory.
  • The TensorRT Python samples and Python API documentation. The Python samples can be found in the /workspace/tensorrt/samples/python directory. Many Python samples can be run using python <script.py> -d /workspace/tensorrt/data. For example:
    python caffe_resnet50.py -d /workspace/tensorrt/data
    The Python API documentation can be found in the /workspace/tensorrt/doc/python directory.
  • TensorRT 7.2.1. Note that the ONNX parser and plugin libraries bundled with this container are built from TensorRT Open Source Software: https://github.com/NVIDIA/TensorRT/releases/tag/20.11
The container also includes the following:

Driver Requirements

Release 20.11 is based on NVIDIA CUDA 11.1.0, which requires NVIDIA Driver release 455 or later. However, if you are running on Tesla (for example, T4 or any other Tesla board), you may use NVIDIA driver release 418.xx, 440.30, or 450.xx. The CUDA driver's compatibility package only supports particular drivers. For a complete list of supported drivers, see the CUDA Application Compatibility topic. For more information, see CUDA Compatibility and Upgrades.

GPU Requirements

Release 20.11 supports CUDA compute capability 3.5 and higher. This corresponds to GPUs in the Kepler, Maxwell, Pascal, Volta, Turing, and Ampere Architecture GPU families. Specifically, for a list of GPUs that this compute capability corresponds to, see CUDA GPUs. For additional support details, see Deep Learning Frameworks Support Matrix.

Key Features and Enhancements

This TensorRT container release includes the following key features and enhancements.
  • TensorRT container image version 20.11 is based on TensorRT 7.2.1. For a list of the new features and enhancements introduced in TensorRT 7.2.1 refer to the TensorRT 7.2.1 release notes.
  • The latest version of NVIDIA NCCL 2.8.2
  • Ubuntu 18.04 with October 2020 updates

Announcements

  • Python 2.7 is no longer supported in this TensorRT container release.

Obtaining Missing Data Files

Some samples require data files that are not included within the TensorRT container either due to licensing restrictions or because they are too large. Samples which do not include all the required data files include a README.md file in the corresponding source directory informing you how to obtain the necessary data files.

Installing Required Python Modules

You may need to first run the Python setup script in order to complete some of the samples. The following script has been added to the container to install the missing Python modules and their dependencies if desired: /opt/tensorrt/python/python_setup.sh

Installing Open Source Components

A script has been added to clone, build and replace the provided plugin, Caffe parser, and ONNX parser libraries with the open source ones based off the 20.11 tag on the official TensorRT open source repository.

To install the open source components inside the container, run the following script:

/opt/tensorrt/install_opensource.sh

For more information see GitHub: TensorRT 20.11.

Limitations

NVIDIA TensorRT Container Versions

The following table shows what versions of Ubuntu, CUDA, and TensorRT are supported in each of the NVIDIA containers for TensorRT. For older container versions, refer to the Frameworks Support Matrix.

Known Issues

There are no known issues in this release.

31. TensorRT Release 20.10

The NVIDIA container image for TensorRT, release 20.10, is available on NGC.

Contents of the TensorRT container

This container includes the following:
  • The TensorRT C++ samples and C++ API documentation. The samples can be built by running make in the /workspace/tensorrt/samples directory. The resulting executables are in the /workspace/tensorrt/bin directory. The C++ API documentation can be found in the /workspace/tensorrt/doc/cpp directory.
  • The TensorRT Python samples and Python API documentation. The Python samples can be found in the /workspace/tensorrt/samples/python directory. Many Python samples can be run using python <script.py> -d /workspace/tensorrt/data. For example:
    python caffe_resnet50.py -d /workspace/tensorrt/data
    The Python API documentation can be found in the /workspace/tensorrt/doc/python directory.
  • TensorRT 7.2.1. Note that the ONNX parser and plugin libraries bundled with this container are built from TensorRT Open Source Software: https://github.com/NVIDIA/TensorRT/releases/tag/20.10
The container also includes the following:

Driver Requirements

Release 20.10 is based on NVIDIA CUDA 11.1.0, which requires NVIDIA Driver release 455 or later. However, if you are running on Tesla (for example, T4 or any other Tesla board), you may use NVIDIA driver release 418.xx, 440.30, or 450.xx. The CUDA driver's compatibility package only supports particular drivers. For a complete list of supported drivers, see the CUDA Application Compatibility topic. For more information, see CUDA Compatibility and Upgrades.

GPU Requirements

Release 20.10 supports CUDA compute capability 3.5 and higher. This corresponds to GPUs in the Kepler, Maxwell, Pascal, Volta, Turing, and Ampere Architecture GPU families. Specifically, for a list of GPUs that this compute capability corresponds to, see CUDA GPUs. For additional support details, see Deep Learning Frameworks Support Matrix.

Key Features and Enhancements

This TensorRT container release includes the following key features and enhancements.

Announcements

  • Python 2.7 is no longer supported in this TensorRT container release.

Obtaining Missing Data Files

Some samples require data files that are not included within the TensorRT container either due to licensing restrictions or because they are too large. Samples which do not include all the required data files include a README.md file in the corresponding source directory informing you how to obtain the necessary data files.

Installing Required Python Modules

You may need to first run the Python setup script in order to complete some of the samples. The following script has been added to the container to install the missing Python modules and their dependencies if desired: /opt/tensorrt/python/python_setup.sh

Installing Open Source Components

A script has been added to clone, build and replace the provided plugin, Caffe parser, and ONNX parser libraries with the open source ones based off the 20.10 tag on the official TensorRT open source repository.

To install the open source components inside the container, run the following script:

/opt/tensorrt/install_opensource.sh

For more information see GitHub: TensorRT 20.10.

Limitations

NVIDIA TensorRT Container Versions

The following table shows what versions of Ubuntu, CUDA, and TensorRT are supported in each of the NVIDIA containers for TensorRT. For older container versions, refer to the