(PDF) - Last updated ,

TensorRT Container Release Notes

The TensorRT container is an easy to use container for TensorRT development. The container allows you to build, modify, and execute TensorRT samples. These release notes provide a list of key features, packaged software in the container, software enhancements and improvements, and known issues for the 24.02 and earlier releases. The TensorRT container is released monthly to provide you with the latest NVIDIA deep learning software libraries and GitHub code contributions that have been sent upstream. The libraries and contributions have all been tested, tuned, and optimized.

For a complete view of the supported software and specific versions that are packaged with the frameworks based on the container image, see the Frameworks Support Matrix.

1. TensorRT Overview

The core of NVIDIA® TensorRT™ is a C++ library that facilitates high-performance inference on NVIDIA graphics processing units (GPUs). TensorRT takes a trained network, which consists of a network definition and a set of trained parameters, and produces a highly optimized runtime engine that performs inference for that network.

You can describe a TensorRT network by using a C++ or Python API, or you can import an existing Caffe, ONNX, or TensorFlow model by using one of the provided parsers.

TensorRT provides APIs through C++ and Python that help express deep learning models by using the Network Definition API or load a predefined model by using parsers that allows TensorRT to optimize and run them on an NVIDIA GPU. TensorRT applies graph optimizations, layer fusion, and other optimizations, while also finding the fastest implementation of that model by leveraging a diverse collection of highly optimized kernels. TensorRT also supplies a runtime that you can use to execute this network on all NVIDIA’s GPUs from the NVIDIA Pascal™ generation onwards.

TensorRT also includes optional high-speed, mixed precision capabilities that were introduced in Tegra X1 and were extended with the NVIDIA Pascal, NVIDIA Volta™, and NVIDIA Turing™ architectures.

The TensorRT container allows TensorRT samples to be built, modified, and executed. For more information about the TensorRT samples, see the TensorRT Sample Support Guide.

For a complete list of installation options and instructions, refer to Installing TensorRT.

2. Pulling A Container

Before you can pull a container from the NGC container registry:

The deep learning frameworks, the NGC Docker containers, and the deep learning framework containers are stored in the nvcr.io/nvidia repository.

3. Running TensorRT

Before you can run an NGC deep learning framework container, your Docker environment must support NVIDIA GPUs. To run a container, issue the appropriate command as explained in Running A Container and specify the registry, repository, and tags.

On a system with GPU support for NGC containers, when you run a container, the following occurs:
  • The Docker engine loads the image into a container which runs the software.
  • You define the runtime resources of the container by including the additional flags and settings that are used with the command.

    These flags and settings are described in Running A Container.

  • The GPUs are explicitly defined for the Docker container, which defaults to all GPUs, but can be specified by using the NVIDIA_VISIBLE_DEVICES environment variable.

    For more information, refer to the nvidia-docker documentation.

    Note: Starting in Docker 19.03, complete the steps below.

The method implemented in your system depends on the DGX OS version that you installed (for DGX systems), the NGC Cloud Image that was provided by a Cloud Service Provider, or the software that you installed to prepare to run NGC containers on TITAN PCs, Quadro PCs, or NVIDIA Virtual GPUs (vGPUs).

  1. Issue the command for the applicable release of the container that you want.

    The following command assumes that you want to pull the latest container.

    docker pull nvcr.io/nvidia/tensorrt:24.02-py3
  2. Open a command prompt and paste the pull command.

    Ensure that the pull process successfully completes before you proceed to step 3.

  3. Run the container image.
    • If you have Docker 19.03 or later, a typical command to launch the container is:
      docker run --gpus all -it --rm -v local_dir:container_dir nvcr.io/nvidia/tensorrt:<xx.xx>-py<x>
    • If you have Docker 19.02 or earlier, a typical command to launch the container is:
      nvidia-docker run -it --rm -v local_dir:container_dir nvcr.io/nvidia/tensorrt:<xx.xx>-py<x>
  4. To extend the TensorRT container, select one of the following options:
    • Add to or modify the source code in this container and run your customized version.
    • To add additional packages, use docker build to add your customizations on top of this container.
      Note: NVIDIA recommends using the docker build option for ease of migration to later versions of the TensorRT container.

4. TensorRT Release 24.02

The NVIDIA container image for TensorRT, release 24.02, is available on NGC.

Contents of the TensorRT container

This container includes the following:
  • The TensorRT C++ samples and C++ API documentation.
    • Build the samples can be by running make in the /workspace/tensorrt/samples directory.
    • The resulting executables are in the /workspace/tensorrt/bin directory.
    • The C++ API documentation is in the /workspace/tensorrt/doc/cpp directory.
  • The TensorRT Python samples and Python API documentation.
    • The Python samples are in the /workspace/tensorrt/samples/python directory.

      Refer to the respective README documents for more samples.

    • Many Python samples can be run by using python <script.py> -d /workspace/tensorrt/data.
      For example:
      python onnx_resnet50.py -d /workspace/tensorrt/data
    • The Python API documentation is in the /workspace/tensorrt/doc/python directory.
  • TensorRT 8.6.3.

    The ONNX parser and plug-in libraries that are bundled with this container are built from TensorRT Open Source Software.

The container also includes the following:

Driver Requirements

Release 24.02 is based on CUDA 12.3.2, which requires NVIDIA Driver release 545 or later. However, if you are running on a data center GPU (for example, T4 or any other data center GPU), you can use NVIDIA driver release 470.57 (or later R470), 525.85 (or later R525), 535.86 (or later R535), or 545.23 (or later R545).

The CUDA driver's compatibility package only supports particular drivers. Thus, users should upgrade from all R418, R440, R450, R460, R510, and R520 drivers, which are not forward-compatible with CUDA 12.3. For a complete list of supported drivers, see the CUDA Application Compatibility topic. For more information, see CUDA Compatibility and Upgrades.

GPU Requirements

Release 24.02 supports CUDA compute capability 6.0 and later. This corresponds to GPUs in the NVIDIA Pascal, NVIDIA Volta™, NVIDIA Turing™, NVIDIA Ampere architecture, NVIDIA Hopper™, and NVIDIA Ada Lovelace architecture families. For a list of GPUs to which this compute capability corresponds, see CUDA GPUs. For additional support details, see the Deep Learning Frameworks Support Matrix.

Key Features and Enhancements

This TensorRT container release includes the following key features and enhancements.
  • TensorRT container image version 24.02 is based on TensorRT 8.6.3.1.

    For a list of the features and enhancements that were introduced in TensorRT 8.6, refer to the TensorRT 8.6 release notes.

  • All dependencies on cuDNN have been removed from the TensorRT 8.6.3 release to reduce the overall container size. Any TensorRT features which depend on cuDNN, which are primarily some plugins and samples, will not work with this release.
  • Latest version of Ubuntu 22.04 with October 2023 updates.

Announcements

  • Starting with the 23.11 release, TensorRT containers supporting iGPU architectures are published, and run on Jetson devices. Please refer to the Frameworks Support Matrix for information regarding which iGPU hardware/software is supported by which container.
  • NVIDIA Deep Learning Profiler (DLProf) v1.8, which was included in the 21.12 container, was the last release of DLProf.

    Starting with the 22.01 container, DLProf is longer included. It can still be manually installed by using a pip wheel on the nvidia-pyindex.

Obtaining Missing Data Files

Some samples require data files that are not included in the TensorRT container because of licensing restrictions, or because they are too large. Samples that do not include the required data files include a README.md file in the corresponding source directory that provides information about how to obtain the necessary data files.

Installing Required Python Modules

  • To complete some of the samples, you might want to first run the Python setup script.
  • If you need to install the missing Python modules and their dependencies, run the /opt/tensorrt/python/python_setup.sh script.

Installing Open Source Components

A script has been added to clone, build, and replace the provided plug-in, the Caffe parser, and the ONNX parser libraries with the open source ones that are based on the 24.01 tag on the official TensorRT open source repository.

To install the open source components in the container, run the following script:

/opt/tensorrt/install_opensource.sh -b main

For more information, see GitHub: TensorRT.

Limitations

NVIDIA TensorRT Container Versions

The following table shows what versions of Ubuntu, CUDA, and TensorRT are supported in each of the NVIDIA containers for TensorRT. For older container versions, refer to the Frameworks Support Matrix.

Container Version Ubuntu CUDA Toolkit TensorRT
24.02 22.04 NVIDIA CUDA 12.3.2 TensorRT 8.6.3
24.01 NVIDIA CUDA 12.3.2 TensorRT 8.6.1.6
23.12
23.11 NVIDIA CUDA 12.3.0
23.10 NVIDIA CUDA 12.2.1
23.09 NVIDIA CUDA 12.2.1
23.08
23.07 NVIDIA CUDA 12.1.1
23.06
23.05 TensorRT 8.6.1.2
23.04 20.04 NVIDIA CUDA 12.1.0 TensorRT 8.6.1
23.03 TensorRT 8.5.3
23.02 NVIDIA CUDA 12.0.1
23.01 TensorRT 8.5.2.2
22.12 NVIDIA CUDA 11.8.0 TensorRT 8.5.1
22.11
22.10 TensorRT 8.5 EA
22.09
22.08 NVIDIA CUDA 11.7.1 TensorRT 8.4.2.4
22.07 NVIDIA CUDA 11.7 Update 1 Preview TensorRT 8.4.1
22.06 TensorRT 8.2.5
22.05 NVIDIA CUDA 11.7.0
22.04 NVIDIA CUDA 11.6.2 TensorRT 8.2.4.2
22.03 NVIDIA CUDA 11.6.1 TensorRT 8.2.3
22.02 NVIDIA CUDA 11.6.0 TensorRT 8.2.3
22.01 NVIDIA CUDA 11.6.0 TensorRT 8.2.2
21.12 NVIDIA CUDA 11.5.0 TensorRT 8.2.1.8
21.11

TensorRT 8.0.3.4 for x64 Linux

TensorRT 8.0.2.2 for Arm SBSA Linux

21.10 NVIDIA CUDA 11.4.2 with cuBLAS 11.6.5.2
21.09 NVIDIA CUDA 11.4.2 TensorRT 8.0.3
21.08 NVIDIA CUDA 11.4.1 TensorRT 8.0.1.6
21.07 NVIDIA CUDA 11.4.0
21.06 NVIDIA CUDA 11.3.1 TensorRT 7.2.3.4
21.05 NVIDIA CUDA 11.3.0
21.04
21.03 NVIDIA CUDA 11.2.1 TensorRT 7.2.2.3
21.02 NVIDIA CUDA 11.2.0 7.2.2.3+cuda11.1.0.024
20.12 NVIDIA CUDA 11.1.1 TensorRT 7.2.2
20.11

18.04

NVIDIA CUDA 11.1.0 TensorRT 7.2.1
20.10
20.09 NVIDIA CUDA 11.0.3 TensorRT 7.1.3
20.08
20.07 NVIDIA CUDA 11.0.194
20.06 NVIDIA CUDA 11.0.167 TensorRT 7.1.2
20.03

20.02

20.01

NVIDIA CUDA 10.2.89 TensorRT 7.0.0

19.12

19.11

TensorRT 6.0.1
19.10 NVIDIA CUDA 10.1.243
19.09
19.08 TensorRT 5.1.5

Known Issues

  • The onnx_graphsurgeon Python module on ARM Server systems is not compatible with ONNX version 1.11.0, which is normally recommended for the included TensorRT release. You will instead need to use ONNX version 1.15.0 to resolve a possible segmentation fault.
  • With r545 or r550 drivers, some models may run into "Unspecified Launch Failure" during engine building. This can be worked around by downgrading the driver version to r535.
  • TensorRT’s version compatibility feature has not been extensively tested and is therefore not supported with TensorRT 8.6.3. This TensorRT release is a special release that removes cuDNN as a dependency. Version compatibility between TensorRT 8.6.1 and future versions as documented will still be supported.
  • Due to removing TensorRT’s dependency on cuDNN the following networks may show performance regressions:
    • BasicUnet
    • DynUnet
    • HighResNet
    • StableDiffusion VAE-encoder
    • StableDiffusion VAE-decoder

5. TensorRT Release 24.01

The NVIDIA container image for TensorRT, release 24.01, is available on NGC.

Contents of the TensorRT container

This container includes the following:
  • The TensorRT C++ samples and C++ API documentation.
    • Build the samples can be by running make in the /workspace/tensorrt/samples directory.
    • The resulting executables are in the /workspace/tensorrt/bin directory.
    • The C++ API documentation is in the /workspace/tensorrt/doc/cpp directory.
  • The TensorRT Python samples and Python API documentation.
    • The Python samples are in the /workspace/tensorrt/samples/python directory.

      Refer to the respective README documents for more samples.

    • Many Python samples can be run by using python <script.py> -d /workspace/tensorrt/data.
      For example:
      python onnx_resnet50.py -d /workspace/tensorrt/data
    • The Python API documentation is in the /workspace/tensorrt/doc/python directory.
  • TensorRT 8.6.1.6.

    The ONNX parser and plug-in libraries that are bundled with this container are built from TensorRT Open Source Software.

The container also includes the following:

Driver Requirements

Release 24.01 is based on CUDA 12.3.2, which requires NVIDIA Driver release 545 or later. However, if you are running on a data center GPU (for example, T4 or any other data center GPU), you can use NVIDIA driver release 470.57 (or later R470), 525.85 (or later R525), 535.86 (or later R535), or 545.23 (or later R545).

The CUDA driver's compatibility package only supports particular drivers. Thus, users should upgrade from all R418, R440, R450, R460, R510, and R520 drivers, which are not forward-compatible with CUDA 12.3. For a complete list of supported drivers, see the CUDA Application Compatibility topic. For more information, see CUDA Compatibility and Upgrades.

GPU Requirements

Release 24.01 supports CUDA compute capability 6.0 and later. This corresponds to GPUs in the NVIDIA Pascal, NVIDIA Volta™, NVIDIA Turing™, NVIDIA Ampere architecture, NVIDIA Hopper™, and NVIDIA Ada Lovelace architecture families. For a list of GPUs to which this compute capability corresponds, see CUDA GPUs. For additional support details, see the Deep Learning Frameworks Support Matrix.

Key Features and Enhancements

This TensorRT container release includes the following key features and enhancements.
  • TensorRT container image version 24.01 is based on TensorRT 8.6.1.6.

    For a list of the features and enhancements that were introduced in TensorRT 8.6.1, refer to the TensorRT 8.6 release notes.

  • Ubuntu 22.04 with October 2023 updates.

Announcements

  • Starting with the 23.11 release, TensorRT containers supporting iGPU architectures are published, and run on Jetson devices. Please refer to the Frameworks Support Matrix for information regarding which iGPU hardware/software is supported by which container.
  • NVIDIA Deep Learning Profiler (DLProf) v1.8, which was included in the 21.12 container, was the last release of DLProf.

    Starting with the 22.01 container, DLProf is longer included. It can still be manually installed by using a pip wheel on the nvidia-pyindex.

Obtaining Missing Data Files

Some samples require data files that are not included in the TensorRT container because of licensing restrictions, or because they are too large. Samples that do not include the required data files include a README.md file in the corresponding source directory that provides information about how to obtain the necessary data files.

Installing Required Python Modules

  • To complete some of the samples, you might want to first run the Python setup script.
  • If you need to install the missing Python modules and their dependencies, run the /opt/tensorrt/python/python_setup.sh script.

Installing Open Source Components

A script has been added to clone, build, and replace the provided plug-in, the Caffe parser, and the ONNX parser libraries with the open source ones that are based on the 24.01 tag on the official TensorRT open source repository.

To install the open source components in the container, run the following script:

/opt/tensorrt/install_opensource.sh -b main

For more information, see GitHub: TensorRT.

Limitations

NVIDIA TensorRT Container Versions

The following table shows what versions of Ubuntu, CUDA, and TensorRT are supported in each of the NVIDIA containers for TensorRT. For older container versions, refer to the Frameworks Support Matrix.

Container Version Ubuntu CUDA Toolkit TensorRT
24.01 22.04 NVIDIA CUDA 12.3.2 TensorRT 8.6.1.6
23.12
23.11 NVIDIA CUDA 12.3.0
23.10 NVIDIA CUDA 12.2.1
23.09 NVIDIA CUDA 12.2.1
23.08
23.07 NVIDIA CUDA 12.1.1
23.06
23.05 TensorRT 8.6.1.2
23.04 20.04 NVIDIA CUDA 12.1.0 TensorRT 8.6.1
23.03 TensorRT 8.5.3
23.02 NVIDIA CUDA 12.0.1
23.01 TensorRT 8.5.2.2
22.12 NVIDIA CUDA 11.8.0 TensorRT 8.5.1
22.11
22.10 TensorRT 8.5 EA
22.09
22.08 NVIDIA CUDA 11.7.1 TensorRT 8.4.2.4
22.07 NVIDIA CUDA 11.7 Update 1 Preview TensorRT 8.4.1
22.06 TensorRT 8.2.5
22.05 NVIDIA CUDA 11.7.0
22.04 NVIDIA CUDA 11.6.2 TensorRT 8.2.4.2
22.03 NVIDIA CUDA 11.6.1 TensorRT 8.2.3
22.02 NVIDIA CUDA 11.6.0 TensorRT 8.2.3
22.01 NVIDIA CUDA 11.6.0 TensorRT 8.2.2
21.12 NVIDIA CUDA 11.5.0 TensorRT 8.2.1.8
21.11

TensorRT 8.0.3.4 for x64 Linux

TensorRT 8.0.2.2 for Arm SBSA Linux

21.10 NVIDIA CUDA 11.4.2 with cuBLAS 11.6.5.2
21.09 NVIDIA CUDA 11.4.2 TensorRT 8.0.3
21.08 NVIDIA CUDA 11.4.1 TensorRT 8.0.1.6
21.07 NVIDIA CUDA 11.4.0
21.06 NVIDIA CUDA 11.3.1 TensorRT 7.2.3.4
21.05 NVIDIA CUDA 11.3.0
21.04
21.03 NVIDIA CUDA 11.2.1 TensorRT 7.2.2.3
21.02 NVIDIA CUDA 11.2.0 7.2.2.3+cuda11.1.0.024
20.12 NVIDIA CUDA 11.1.1 TensorRT 7.2.2
20.11

18.04

NVIDIA CUDA 11.1.0 TensorRT 7.2.1
20.10
20.09 NVIDIA CUDA 11.0.3 TensorRT 7.1.3
20.08
20.07 NVIDIA CUDA 11.0.194
20.06 NVIDIA CUDA 11.0.167 TensorRT 7.1.2
20.03

20.02

20.01

NVIDIA CUDA 10.2.89 TensorRT 7.0.0

19.12

19.11

TensorRT 6.0.1
19.10 NVIDIA CUDA 10.1.243
19.09
19.08 TensorRT 5.1.5

Known Issues

None.

7. TensorRT Release 23.12

The NVIDIA container image for TensorRT, release 23.12, is available on NGC.

Contents of the TensorRT container

This container includes the following:
  • The TensorRT C++ samples and C++ API documentation.
    • Build the samples can be by running make in the /workspace/tensorrt/samples directory.
    • The resulting executables are in the /workspace/tensorrt/bin directory.
    • The C++ API documentation is in the /workspace/tensorrt/doc/cpp directory.
  • The TensorRT Python samples and Python API documentation.
    • The Python samples are in the /workspace/tensorrt/samples/python directory.

      Refer to the respective README documents for more samples.

    • Many Python samples can be run by using python <script.py> -d /workspace/tensorrt/data.
      For example:
      python onnx_resnet50.py -d /workspace/tensorrt/data
    • The Python API documentation is in the /workspace/tensorrt/doc/python directory.
  • TensorRT 8.6.1.6.

    The ONNX parser and plug-in libraries that are bundled with this container are built from TensorRT Open Source Software.

The container also includes the following:

Driver Requirements

Release 23.12 is based on CUDA 12.3.2, which requires NVIDIA Driver release 545 or later. However, if you are running on a data center GPU (for example, T4 or any other data center GPU), you can use NVIDIA driver release 450.51 (or later R450), 470.57 (or later R470), 510.47 (or later R510), or 525.85 (or later R525) 535.86 (or later R535), or 545.23 (or later R545).

The CUDA driver's compatibility package only supports particular drivers. Thus, users should upgrade from all R418, R440, R460, and R520 drivers, which are not forward-compatible with CUDA 12.3. For a complete list of supported drivers, see the CUDA Application Compatibility topic. For more information, see CUDA Compatibility and Upgrades.

GPU Requirements

Release 23.12 supports CUDA compute capability 6.0 and later. This corresponds to GPUs in the NVIDIA Kepler, Maxwell, NVIDIA Pascal, NVIDIA Volta™, NVIDIA Turing™, NVIDIA Ampere architecture, NVIDIA Hopper™, and NVIDIA Ada Lovelace architecture families. For a list of GPUs to which this compute capability corresponds, see CUDA GPUs. For additional support details, see the Deep Learning Frameworks Support Matrix.

Key Features and Enhancements

This TensorRT container release includes the following key features and enhancements.
  • TensorRT container image version 23.12 is based on TensorRT 8.6.1.6.

    For a list of the features and enhancements that were introduced in TensorRT 8.6.1, refer to the TensorRT 8.6 release notes.

  • Ubuntu 22.04 with October 2023 updates.

Announcements

  • Starting with the 23.11 release, TensorRT containers supporting iGPU architectures are published, and run on Jetson devices. Please refer to the Frameworks Support Matrix for information regarding which iGPU hardware/software is supported by which container.
  • NVIDIA Deep Learning Profiler (DLProf) v1.8, which was included in the 21.12 container, was the last release of DLProf.

    Starting with the 22.01 container, DLProf is longer included. It can still be manually installed by using a pip wheel on the nvidia-pyindex.

Obtaining Missing Data Files

Some samples require data files that are not included in the TensorRT container because of licensing restrictions, or because they are too large. Samples that do not include the required data files include a README.md file in the corresponding source directory that provides information about how to obtain the necessary data files.

Installing Required Python Modules

  • To complete some of the samples, you might want to first run the Python setup script.
  • If you need to install the missing Python modules and their dependencies, run the /opt/tensorrt/python/python_setup.sh script.

Installing Open Source Components

A script has been added to clone, build, and replace the provided plug-in, the Caffe parser, and the ONNX parser libraries with the open source ones that are based on the 23.12 tag on the official TensorRT open source repository.

To install the open source components in the container, run the following script:

/opt/tensorrt/install_opensource.sh -b main

For more information, see GitHub: TensorRT.

Limitations

NVIDIA TensorRT Container Versions

The following table shows what versions of Ubuntu, CUDA, and TensorRT are supported in each of the NVIDIA containers for TensorRT. For older container versions, refer to the Frameworks Support Matrix.

Known Issues

None.

TensorRT Release 23.11

The NVIDIA container image for TensorRT, release 23.11, is available on NGC.

Contents of the TensorRT container

This container includes the following:
  • The TensorRT C++ samples and C++ API documentation.
    • Build the samples can be by running make in the /workspace/tensorrt/samples directory.
    • The resulting executables are in the /workspace/tensorrt/bin directory.
    • The C++ API documentation is in the /workspace/tensorrt/doc/cpp directory.
  • The TensorRT Python samples and Python API documentation.
    • The Python samples are in the /workspace/tensorrt/samples/python directory.

      Refer to the respective README documents for more samples.

    • Many Python samples can be run by using python <script.py> -d /workspace/tensorrt/data.
      For example:
      python onnx_resnet50.py -d /workspace/tensorrt/data
    • The Python API documentation is in the /workspace/tensorrt/doc/python directory.
  • TensorRT 8.6.1.6.

    The ONNX parser and plug-in libraries that are bundled with this container are built from TensorRT Open Source Software.

The container also includes the following:

Driver Requirements

Release 23.11 is based on CUDA 12.3.0, which requires NVIDIA Driver release 545 or later. However, if you are running on a data center GPU (for example, T4 or any other data center GPU), you can use NVIDIA driver release 450.51 (or later R450), 470.57 (or later R470), 510.47 (or later R510), or 525.85 (or later R525) 535.86 (or later R535), or 545.23 (or later R545).

The CUDA driver's compatibility package only supports particular drivers. Thus, users should upgrade from all R418, R440, R460, and R520 drivers, which are not forward-compatible with CUDA 12.3. For a complete list of supported drivers, see the CUDA Application Compatibility topic. For more information, see CUDA Compatibility and Upgrades.

GPU Requirements

Release 23.11 supports CUDA compute capability 6.0 and later. This corresponds to GPUs in the NVIDIA Kepler, Maxwell, NVIDIA Pascal, NVIDIA Volta™, NVIDIA Turing™, NVIDIA Ampere architecture, NVIDIA Hopper™, and NVIDIA Ada Lovelace architecture families. For a list of GPUs to which this compute capability corresponds, see CUDA GPUs. For additional support details, see the Deep Learning Frameworks Support Matrix.

Key Features and Enhancements

This TensorRT container release includes the following key features and enhancements.
  • TensorRT container image version 23.11 is based on TensorRT 8.6.1.6.

    For a list of the features and enhancements that were introduced in TensorRT 8.6.1, refer to the TensorRT 8.6 release notes.

  • Ubuntu 22.04 with October 2023 updates.

Announcements

  • Starting with the 23.11 release, TensorRT containers supporting iGPU architectures are published, and run on Jetson devices. Please refer to the Frameworks Support Matrix for information regarding which iGPU hardware/software is supported by which container.
  • NVIDIA Deep Learning Profiler (DLProf) v1.8, which was included in the 21.12 container, was the last release of DLProf.

    Starting with the 22.01 container, DLProf is longer included. It can still be manually installed by using a pip wheel on the nvidia-pyindex.

Obtaining Missing Data Files

Some samples require data files that are not included in the TensorRT container because of licensing restrictions, or because they are too large. Samples that do not include the required data files include a README.md file in the corresponding source directory that provides information about how to obtain the necessary data files.

Installing Required Python Modules

  • To complete some of the samples, you might want to first run the Python setup script.
  • If you need to install the missing Python modules and their dependencies, run the /opt/tensorrt/python/python_setup.sh script.

Installing Open Source Components

A script has been added to clone, build, and replace the provided plug-in, the Caffe parser, and the ONNX parser libraries with the open source ones that are based on the 23.11 tag on the official TensorRT open source repository.

To install the open source components in the container, run the following script:

/opt/tensorrt/install_opensource.sh -b main

For more information, see GitHub: TensorRT.

Limitations

NVIDIA TensorRT Container Versions

The following table shows what versions of Ubuntu, CUDA, and TensorRT are supported in each of the NVIDIA containers for TensorRT. For older container versions, refer to the Frameworks Support Matrix.

Known Issues

None.

8. TensorRT Release 23.10

The NVIDIA container image for TensorRT, release 23.10, is available on NGC.

Contents of the TensorRT container

This container includes the following:
  • The TensorRT C++ samples and C++ API documentation.
    • Build the samples can be by running make in the /workspace/tensorrt/samples directory.
    • The resulting executables are in the /workspace/tensorrt/bin directory.
    • The C++ API documentation is in the /workspace/tensorrt/doc/cpp directory.
  • The TensorRT Python samples and Python API documentation.
    • The Python samples are in the /workspace/tensorrt/samples/python directory.

      Refer to the respective README documents for more samples.

    • Many Python samples can be run by using python <script.py> -d /workspace/tensorrt/data.
      For example:
      python onnx_resnet50.py -d /workspace/tensorrt/data
    • The Python API documentation is in the /workspace/tensorrt/doc/python directory.
  • TensorRT 8.6.1.6.

    The ONNX parser and plug-in libraries that are bundled with this container are built from TensorRT Open Source Software.

The container also includes the following:

Driver Requirements

Release 23.10 is based on CUDA 12.2.2, which requires NVIDIA Driver release 535 or later. However, if you are running on a data center GPU (for example, T4 or any other data center GPU), you can use NVIDIA driver release 450.51 (or later R450), 470.57 (or later R470), 510.47 (or later R510), or 525.85 (or later R525), or 535.86 (or later R535).

The CUDA driver's compatibility package only supports particular drivers. Thus, users should upgrade from all R418, R440, R460, and R520 drivers, which are not forward-compatible with CUDA 12.2. For a complete list of supported drivers, see the CUDA Application Compatibility topic. For more information, see CUDA Compatibility and Upgrades.

GPU Requirements

Release 23.10 supports CUDA compute capability 6.0 and later. This corresponds to GPUs in the NVIDIA Kepler, Maxwell, NVIDIA Pascal, NVIDIA Volta™, NVIDIA Turing™, NVIDIA Ampere architecture, NVIDIA Hopper™, and NVIDIA Ada Lovelace architecture families. For a list of GPUs to which this compute capability corresponds, see CUDA GPUs. For additional support details, see the Deep Learning Frameworks Support Matrix.

Key Features and Enhancements

This TensorRT container release includes the following key features and enhancements.
  • TensorRT container image version 23.10 is based on TensorRT 8.6.1.6.

    For a list of the features and enhancements that were introduced in TensorRT 8.6.1, refer to the TensorRT 8.6 release notes.

  • Ubuntu 22.04 with September 2023 updates.

Announcements

  • NVIDIA Deep Learning Profiler (DLProf) v1.8, which was included in the 21.12 container, was the last release of DLProf.

    Starting with the 22.01 container, DLProf is longer included. It can still be manually installed by using a pip wheel on the nvidia-pyindex.

Obtaining Missing Data Files

Some samples require data files that are not included in the TensorRT container because of licensing restrictions, or because they are too large. Samples that do not include the required data files include a README.md file in the corresponding source directory that provides information about how to obtain the necessary data files.

Installing Required Python Modules

  • To complete some of the samples, you might want to first run the Python setup script.
  • If you need to install the missing Python modules and their dependencies, run the /opt/tensorrt/python/python_setup.sh script.

Installing Open Source Components

A script has been added to clone, build, and replace the provided plug-in, the Caffe parser, and the ONNX parser libraries with the open source ones that are based on the 22.05 tag on the official TensorRT open source repository.

To install the open source components in the container, run the following script:

/opt/tensorrt/install_opensource.sh -b main

For more information, see GitHub: TensorRT.

Limitations

NVIDIA TensorRT Container Versions

The following table shows what versions of Ubuntu, CUDA, and TensorRT are supported in each of the NVIDIA containers for TensorRT. For older container versions, refer to the Frameworks Support Matrix.

Known Issues

None.

9. TensorRT Release 23.09

The NVIDIA container image for TensorRT, release 23.09, is available on NGC.

Contents of the TensorRT container

This container includes the following:
  • The TensorRT C++ samples and C++ API documentation.
    • Build the samples can be by running make in the /workspace/tensorrt/samples directory.
    • The resulting executables are in the /workspace/tensorrt/bin directory.
    • The C++ API documentation is in the /workspace/tensorrt/doc/cpp directory.
  • The TensorRT Python samples and Python API documentation.
    • The Python samples are in the /workspace/tensorrt/samples/python directory.

      Refer to the respective README documents for more samples.

    • Many Python samples can be run by using python <script.py> -d /workspace/tensorrt/data.
      For example:
      python onnx_resnet50.py -d /workspace/tensorrt/data
    • The Python API documentation is in the /workspace/tensorrt/doc/python directory.
  • TensorRT 8.6.1.6.

    The ONNX parser and plug-in libraries that are bundled with this container are built from TensorRT Open Source Software.

The container also includes the following:

Driver Requirements

Release 23.09 is based on CUDA 12.2.1, which requires NVIDIA Driver release 535 or later. However, if you are running on a data center GPU (for example, T4 or any other data center GPU), you can use NVIDIA driver release 450.51 (or later R450), 470.57 (or later R470), 510.47 (or later R510), or 525.85 (or later R525), or 535.86 (or later R535).

The CUDA driver's compatibility package only supports particular drivers. Thus, users should upgrade from all R418, R440, R460, and R520 drivers, which are not forward-compatible with CUDA 12.2. For a complete list of supported drivers, see the CUDA Application Compatibility topic. For more information, see CUDA Compatibility and Upgrades.

GPU Requirements

Release 23.09 supports CUDA compute capability 6.0 and later. This corresponds to GPUs in the NVIDIA Kepler, Maxwell, NVIDIA Pascal, NVIDIA Volta™, NVIDIA Turing™, NVIDIA Ampere architecture, NVIDIA Hopper™, and NVIDIA Ada Lovelace architecture families. For a list of GPUs to which this compute capability corresponds, see CUDA GPUs. For additional support details, see the Deep Learning Frameworks Support Matrix.

Key Features and Enhancements

This TensorRT container release includes the following key features and enhancements.
  • TensorRT container image version 23.09 is based on TensorRT 8.6.1.6.

    For a list of the features and enhancements that were introduced in TensorRT 8.6.1, refer to the TensorRT 8.6 release notes.

  • Ubuntu 22.04 with August 2023 updates.

Announcements

  • NVIDIA Deep Learning Profiler (DLProf) v1.8, which was included in the 21.12 container, was the last release of DLProf.

    Starting with the 22.01 container, DLProf is longer included. It can still be manually installed by using a pip wheel on the nvidia-pyindex.

Obtaining Missing Data Files

Some samples require data files that are not included in the TensorRT container because of licensing restrictions, or because they are too large. Samples that do not include the required data files include a README.md file in the corresponding source directory that provides information about how to obtain the necessary data files.

Installing Required Python Modules

  • To complete some of the samples, you might want to first run the Python setup script.
  • If you need to install the missing Python modules and their dependencies, run the /opt/tensorrt/python/python_setup.sh script.

Installing Open Source Components

A script has been added to clone, build, and replace the provided plug-in, the Caffe parser, and the ONNX parser libraries with the open source ones that are based on the 22.05 tag on the official TensorRT open source repository.

To install the open source components in the container, run the following script:

/opt/tensorrt/install_opensource.sh -b main

For more information, see GitHub: TensorRT.

Limitations

NVIDIA TensorRT Container Versions

The following table shows what versions of Ubuntu, CUDA, and TensorRT are supported in each of the NVIDIA containers for TensorRT. For older container versions, refer to the Frameworks Support Matrix.

Known Issues

None.

10. TensorRT Release 23.08

The NVIDIA container image for TensorRT, release 23.08, is available on NGC.

Contents of the TensorRT container

This container includes the following:
  • The TensorRT C++ samples and C++ API documentation.
    • Build the samples can be by running make in the /workspace/tensorrt/samples directory.
    • The resulting executables are in the /workspace/tensorrt/bin directory.
    • The C++ API documentation is in the /workspace/tensorrt/doc/cpp directory.
  • The TensorRT Python samples and Python API documentation.
    • The Python samples are in the /workspace/tensorrt/samples/python directory.

      Refer to the respective README documents for more samples.

    • Many Python samples can be run by using python <script.py> -d /workspace/tensorrt/data.
      For example:
      python onnx_resnet50.py -d /workspace/tensorrt/data
    • The Python API documentation is in the /workspace/tensorrt/doc/python directory.
  • TensorRT 8.6.1.6.

    The ONNX parser and plug-in libraries that are bundled with this container are built from TensorRT Open Source Software.

The container also includes the following:

Driver Requirements

Release 23.08 is based on CUDA 12.2.1, which requires NVIDIA Driver release 535 or later. However, if you are running on a data center GPU (for example, T4 or any other data center GPU), you can use NVIDIA driver release 450.51 (or later R450), 470.57 (or later R470), 510.47 (or later R510), or 525.85 (or later R525), or 535.86 (or later R535).

The CUDA driver's compatibility package only supports particular drivers. Thus, users should upgrade from all R418, R440, R460, and R520 drivers, which are not forward-compatible with CUDA 12.2. For a complete list of supported drivers, see the CUDA Application Compatibility topic. For more information, see CUDA Compatibility and Upgrades.

GPU Requirements

Release 23.08 supports CUDA compute capability 6.0 and later. This corresponds to GPUs in the NVIDIA Kepler, Maxwell, NVIDIA Pascal, NVIDIA Volta™, NVIDIA Turing™, NVIDIA Ampere architecture, NVIDIA Hopper™, and NVIDIA Ada Lovelace architecture families. For a list of GPUs to which this compute capability corresponds, see CUDA GPUs. For additional support details, see the Deep Learning Frameworks Support Matrix.

Key Features and Enhancements

This TensorRT container release includes the following key features and enhancements.
  • TensorRT container image version 23.08 is based on TensorRT 8.6.1.6.

    For a list of the features and enhancements that were introduced in TensorRT 8.6.1, refer to the TensorRT 8.6 release notes.

  • Ubuntu 22.04 with July 2023 updates.

Announcements

  • NVIDIA Deep Learning Profiler (DLProf) v1.8, which was included in the 21.12 container, was the last release of DLProf.

    Starting with the 22.01 container, DLProf is longer included. It can still be manually installed by using a pip wheel on the nvidia-pyindex.

Obtaining Missing Data Files

Some samples require data files that are not included in the TensorRT container because of licensing restrictions, or because they are too large. Samples that do not include the required data files include a README.md file in the corresponding source directory that provides information about how to obtain the necessary data files.

Installing Required Python Modules

  • To complete some of the samples, you might want to first run the Python setup script.
  • If you need to install the missing Python modules and their dependencies, run the /opt/tensorrt/python/python_setup.sh script.

Installing Open Source Components

A script has been added to clone, build, and replace the provided plug-in, the Caffe parser, and the ONNX parser libraries with the open source ones that are based on the 22.05 tag on the official TensorRT open source repository.

To install the open source components in the container, run the following script:

/opt/tensorrt/install_opensource.sh -b main

For more information, see GitHub: TensorRT.

Limitations

NVIDIA TensorRT Container Versions

The following table shows what versions of Ubuntu, CUDA, and TensorRT are supported in each of the NVIDIA containers for TensorRT. For older container versions, refer to the Frameworks Support Matrix.

Known Issues

None.

11. TensorRT Release 23.07

The NVIDIA container image for TensorRT, release 23.07, is available on NGC.

Contents of the TensorRT container

This container includes the following:
  • The TensorRT C++ samples and C++ API documentation.
    • Build the samples can be by running make in the /workspace/tensorrt/samples directory.
    • The resulting executables are in the /workspace/tensorrt/bin directory.
    • The C++ API documentation is in the /workspace/tensorrt/doc/cpp directory.
  • The TensorRT Python samples and Python API documentation.
    • The Python samples are in the /workspace/tensorrt/samples/python directory.

      Refer to the respective README documents for more samples.

    • Many Python samples can be run by using python <script.py> -d /workspace/tensorrt/data.
      For example:
      python onnx_resnet50.py -d /workspace/tensorrt/data
    • The Python API documentation is in the /workspace/tensorrt/doc/python directory.
  • TensorRT 8.6.1.6.

    The ONNX parser and plug-in libraries that are bundled with this container are built from TensorRT Open Source Software.

The container also includes the following:

Driver Requirements

Release 23.07 is based on CUDA 12.1.1, which requires NVIDIA Driver release 530 or later. However, if you are running on a data center GPU (for example, T4 or any other data center GPU), you can use NVIDIA driver release 450.51 (or later R450), 470.57 (or later R470), 510.47 (or later R510), 515.65 (or later R515), 525.85 (or later R525), or 530.30 (or later R530).

The CUDA driver's compatibility package only supports particular drivers. Thus, users should upgrade from all R418, R440, R460, and R520 drivers, which are not forward-compatible with CUDA 12.1. For a complete list of supported drivers, see the CUDA Application Compatibility topic. For more information, see CUDA Compatibility and Upgrades.

GPU Requirements

Release 23.07 supports CUDA compute capability 3.5 and later. This corresponds to GPUs in the NVIDIA Kepler, Maxwell, NVIDIA Pascal, NVIDIA Volta™, NVIDIA Turing™, NVIDIA Ampere architecture, NVIDIA Hopper™, and NVIDIA Ada Lovelace architecture families. For a list of GPUs to which this compute capability corresponds, see CUDA GPUs. For additional support details, see the Deep Learning Frameworks Support Matrix.

Key Features and Enhancements

This TensorRT container release includes the following key features and enhancements.
  • TensorRT container image version 23.07 is based on TensorRT 8.6.1.6.

    For a list of the features and enhancements that were introduced in TensorRT 8.6.1, refer to the TensorRT 8.6 release notes.

  • Ubuntu 22.04 with June 2023 updates.

Announcements

  • NVIDIA Deep Learning Profiler (DLProf) v1.8, which was included in the 21.12 container, was the last release of DLProf.

    Starting with the 22.01 container, DLProf is longer included. It can still be manually installed by using a pip wheel on the nvidia-pyindex.

Obtaining Missing Data Files

Some samples require data files that are not included in the TensorRT container because of licensing restrictions, or because they are too large. Samples that do not include the required data files include a README.md file in the corresponding source directory that provides information about how to obtain the necessary data files.

Installing Required Python Modules

  • To complete some of the samples, you might want to first run the Python setup script.
  • If you need to install the missing Python modules and their dependencies, run the /opt/tensorrt/python/python_setup.sh script.

Installing Open Source Components

A script has been added to clone, build, and replace the provided plug-in, the Caffe parser, and the ONNX parser libraries with the open source ones that are based on the 22.05 tag on the official TensorRT open source repository.

To install the open source components in the container, run the following script:

/opt/tensorrt/install_opensource.sh -b main

For more information, see GitHub: TensorRT.

Limitations

NVIDIA TensorRT Container Versions

The following table shows what versions of Ubuntu, CUDA, and TensorRT are supported in each of the NVIDIA containers for TensorRT. For older container versions, refer to the Frameworks Support Matrix.

Known Issues

None.

12. TensorRT Release 23.06

The NVIDIA container image for TensorRT, release 23.06, is available on NGC.

Contents of the TensorRT container

This container includes the following:
  • The TensorRT C++ samples and C++ API documentation.
    • Build the samples can be by running make in the /workspace/tensorrt/samples directory.
    • The resulting executables are in the /workspace/tensorrt/bin directory.
    • The C++ API documentation is in the /workspace/tensorrt/doc/cpp directory.
  • The TensorRT Python samples and Python API documentation.
    • The Python samples are in the /workspace/tensorrt/samples/python directory.

      Refer to the respective README documents for more samples.

    • Many Python samples can be run by using python <script.py> -d /workspace/tensorrt/data.
      For example:
      python onnx_resnet50.py -d /workspace/tensorrt/data
    • The Python API documentation is in the /workspace/tensorrt/doc/python directory.
  • TensorRT 8.6.1.6.

    The ONNX parser and plug-in libraries that are bundled with this container are built from TensorRT Open Source Software.

The container also includes the following:

Driver Requirements

Release 23.06 is based on CUDA 12.1.1, which requires NVIDIA Driver release 530 or later. However, if you are running on a data center GPU (for example, T4 or any other data center GPU), you can use NVIDIA driver release 450.51 (or later R450), 470.57 (or later R470), 510.47 (or later R510), 515.65 (or later R515), 525.85 (or later R525), or 530.30 (or later R530).

The CUDA driver's compatibility package only supports particular drivers. Thus, users should upgrade from all R418, R440, R460, and R520 drivers, which are not forward-compatible with CUDA 12.1. For a complete list of supported drivers, see the CUDA Application Compatibility topic. For more information, see CUDA Compatibility and Upgrades.

GPU Requirements

Release 23.06 supports CUDA compute capability 3.5 and later. This corresponds to GPUs in the NVIDIA Kepler, Maxwell, NVIDIA Pascal, NVIDIA Volta™, NVIDIA Turing™, NVIDIA Ampere architecture, NVIDIA Hopper™, and NVIDIA Ada Lovelace architecture families. For a list of GPUs to which this compute capability corresponds, see CUDA GPUs. For additional support details, see the Deep Learning Frameworks Support Matrix.

Key Features and Enhancements

This TensorRT container release includes the following key features and enhancements.
  • TensorRT container image version 23.06 is based on TensorRT 8.6.1.6.

    For a list of the features and enhancements that were introduced in TensorRT 8.6.1, refer to the TensorRT 8.6 release notes.

  • Ubuntu 22.04 with May 2023 updates.

Announcements

  • NVIDIA Deep Learning Profiler (DLProf) v1.8, which was included in the 21.12 container, was the last release of DLProf.

    Starting with the 22.01 container, DLProf is longer included. It can still be manually installed by using a pip wheel on the nvidia-pyindex.

Obtaining Missing Data Files

Some samples require data files that are not included in the TensorRT container because of licensing restrictions, or because they are too large. Samples that do not include the required data files include a README.md file in the corresponding source directory that provides information about how to obtain the necessary data files.

Installing Required Python Modules

  • To complete some of the samples, you might want to first run the Python setup script.
  • If you need to install the missing Python modules and their dependencies, run the /opt/tensorrt/python/python_setup.sh script.

Installing Open Source Components

A script has been added to clone, build, and replace the provided plug-in, the Caffe parser, and the ONNX parser libraries with the open source ones that are based on the 22.05 tag on the official TensorRT open source repository.

To install the open source components in the container, run the following script:

/opt/tensorrt/install_opensource.sh -b main

For more information, see GitHub: TensorRT.

Limitations

NVIDIA TensorRT Container Versions

The following table shows what versions of Ubuntu, CUDA, and TensorRT are supported in each of the NVIDIA containers for TensorRT. For older container versions, refer to the Frameworks Support Matrix.

Known Issues

None.

13. TensorRT Release 23.05

The NVIDIA container image for TensorRT, release 23.05, is available on NGC.

Contents of the TensorRT container

This container includes the following:
  • The TensorRT C++ samples and C++ API documentation.
    • Build the samples can be by running make in the /workspace/tensorrt/samples directory.
    • The resulting executables are in the /workspace/tensorrt/bin directory.
    • The C++ API documentation is in the /workspace/tensorrt/doc/cpp directory.
  • The TensorRT Python samples and Python API documentation.
    • The Python samples are in the /workspace/tensorrt/samples/python directory.

      Refer to the respective README documents for more samples.

    • Many Python samples can be run by using python <script.py> -d /workspace/tensorrt/data.
      For example:
      python onnx_resnet50.py -d /workspace/tensorrt/data
    • The Python API documentation is in the /workspace/tensorrt/doc/python directory.
  • TensorRT 8.6.1.2.

    The ONNX parser and plug-in libraries that are bundled with this container are built from TensorRT Open Source Software.

The container also includes the following:

Driver Requirements

Release 23.05 is based on CUDA 12.1.1, which requires NVIDIA Driver release 530 or later. However, if you are running on a data center GPU (for example, T4 or any other data center GPU), you can use NVIDIA driver release 450.51 (or later R450), 470.57 (or later R470), 510.47 (or later R510), 515.65 (or later R515), 525.85 (or later R525), or 530.30 (or later R530).

The CUDA driver's compatibility package only supports particular drivers. Thus, users should upgrade from all R418, R440, R460, and R520 drivers, which are not forward-compatible with CUDA 12.1. For a complete list of supported drivers, see the CUDA Application Compatibility topic. For more information, see CUDA Compatibility and Upgrades.

GPU Requirements

Release 23.05 supports CUDA compute capability 3.5 and later. This corresponds to GPUs in the NVIDIA Kepler, Maxwell, NVIDIA Pascal, NVIDIA Volta™, NVIDIA Turing™, NVIDIA Ampere architecture, NVIDIA Hopper™, and NVIDIA Ada Lovelace architecture families. For a list of GPUs to which this compute capability corresponds, see CUDA GPUs. For additional support details, see the Deep Learning Frameworks Support Matrix.

Key Features and Enhancements

This TensorRT container release includes the following key features and enhancements.
  • TensorRT container image version 23.05 is based on TensorRT 8.6.1.2.

    For a list of the features and enhancements that were introduced in TensorRT 8.6.1, refer to the TensorRT 8.6 release notes.

  • Ubuntu 22.04 with April 2023 updates.

Announcements

  • Starting with the 22.05 release, the TensorRT container is available for the Arm SBSA platform.

    For example, when you pull the nvcr.io/nvidia/tensorrt:22.05-py3 Docker image on an Arm SBSA machine, the Arm-specific image is automatically fetched.

  • NVIDIA Deep Learning Profiler (DLProf) v1.8, which was included in the 21.12 container, was the last release of DLProf.

    Starting with the 22.01 container, DLProf is longer included. It can still be manually installed by using a pip wheel on the nvidia-pyindex.

Obtaining Missing Data Files

Some samples require data files that are not included in the TensorRT container because of licensing restrictions, or because they are too large. Samples that do not include the required data files include a README.md file in the corresponding source directory that provides information about how to obtain the necessary data files.

Installing Required Python Modules

  • To complete some of the samples, you might want to first run the Python setup script.
  • If you need to install the missing Python modules and their dependencies, run the /opt/tensorrt/python/python_setup.sh script.

Installing Open Source Components

A script has been added to clone, build, and replace the provided plug-in, the Caffe parser, and the ONNX parser libraries with the open source ones that are based on the 22.05 tag on the official TensorRT open source repository.

To install the open source components in the container, run the following script:

/opt/tensorrt/install_opensource.sh -b main

For more information, see GitHub: TensorRT.

Limitations

NVIDIA TensorRT Container Versions

The following table shows what versions of Ubuntu, CUDA, and TensorRT are supported in each of the NVIDIA containers for TensorRT. For older container versions, refer to the Frameworks Support Matrix.

Known Issues

None.

14. TensorRT Release 23.04

The NVIDIA container image for TensorRT, release 23.04, is available on NGC.

Contents of the TensorRT container

This container includes the following:
  • The TensorRT C++ samples and C++ API documentation.
    • Build the samples can be by running make in the /workspace/tensorrt/samples directory.
    • The resulting executables are in the /workspace/tensorrt/bin directory.
    • The C++ API documentation is in the /workspace/tensorrt/doc/cpp directory.
  • The TensorRT Python samples and Python API documentation.
    • The Python samples are in the /workspace/tensorrt/samples/python directory.

      Refer to the respective README documents for more samples.

    • Many Python samples can be run by using python <script.py> -d /workspace/tensorrt/data.
      For example:
      python onnx_resnet50.py -d /workspace/tensorrt/data
    • The Python API documentation is in the /workspace/tensorrt/doc/python directory.
  • TensorRT 8.6.1.

    The ONNX parser and plug-in libraries that are bundled with this container are built from TensorRT Open Source Software.

The container also includes the following:

Driver Requirements

Release 23.04 is based on CUDA 12.1.0, which requires NVIDIA Driver release 530 or later. However, if you are running on a data center GPU (for example, T4 or any other data center GPU), you can use NVIDIA driver release 450.51 (or later R450), 470.57 (or later R470), 510.47 (or later R510), 515.65 (or later R515), 525.85 (or later R525), or 530.30 (or later R530).

The CUDA driver's compatibility package only supports particular drivers. Thus, users should upgrade from all R418, R440, R460, and R520 drivers, which are not forward-compatible with CUDA 12.1. For a complete list of supported drivers, see the CUDA Application Compatibility topic. For more information, see CUDA Compatibility and Upgrades.

GPU Requirements

Release 23.04 supports CUDA compute capability 3.5 and later. This corresponds to GPUs in the NVIDIA Kepler, Maxwell, NVIDIA Pascal, NVIDIA Volta™, NVIDIA Turing™, NVIDIA Ampere architecture, NVIDIA Hopper™, and NVIDIA Ada Lovelace architecture families. For a list of GPUs to which this compute capability corresponds, see CUDA GPUs. For additional support details, see the Deep Learning Frameworks Support Matrix.

Key Features and Enhancements

This TensorRT container release includes the following key features and enhancements.
  • TensorRT container image version 23.04 is based on TensorRT 8.6.1.

    For a list of the features and enhancements that were introduced in TensorRT 8.6.1, refer to the TensorRT 8.6 release notes.

  • Ubuntu 20.04 with March 2023 updates.

Announcements

  • Starting with the 22.05 release, the TensorRT container is available for the Arm SBSA platform.

    For example, when you pull the nvcr.io/nvidia/tensorrt:22.05-py3 Docker image on an Arm SBSA machine, the Arm-specific image is automatically fetched.

  • NVIDIA Deep Learning Profiler (DLProf) v1.8, which was included in the 21.12 container, was the last release of DLProf.

    Starting with the 22.01 container, DLProf is longer included. It can still be manually installed by using a pip wheel on the nvidia-pyindex.

Obtaining Missing Data Files

Some samples require data files that are not included in the TensorRT container because of licensing restrictions, or because they are too large. Samples that do not include the required data files include a README.md file in the corresponding source directory that provides information about how to obtain the necessary data files.

Installing Required Python Modules

  • To complete some of the samples, you might want to first run the Python setup script.
  • If you need to install the missing Python modules and their dependencies, run the /opt/tensorrt/python/python_setup.sh script.

Installing Open Source Components

A script has been added to clone, build, and replace the provided plug-in, the Caffe parser, and the ONNX parser libraries with the open source ones that are based on the 22.05 tag on the official TensorRT open source repository.

To install the open source components in the container, run the following script:

/opt/tensorrt/install_opensource.sh -b main

For more information, see GitHub: TensorRT.

Limitations

NVIDIA TensorRT Container Versions

The following table shows what versions of Ubuntu, CUDA, and TensorRT are supported in each of the NVIDIA containers for TensorRT. For older container versions, refer to the Frameworks Support Matrix.

Known Issues

None.

15. TensorRT Release 23.03

The NVIDIA container image for TensorRT, release 23.03, is available on NGC.

Contents of the TensorRT container

This container includes the following:
  • The TensorRT C++ samples and C++ API documentation.
    • Build the samples can be by running make in the /workspace/tensorrt/samples directory.
    • The resulting executables are in the /workspace/tensorrt/bin directory.
    • The C++ API documentation is in the /workspace/tensorrt/doc/cpp directory.
  • The TensorRT Python samples and Python API documentation.
    • The Python samples are in the /workspace/tensorrt/samples/python directory.

      Refer to the respective README documents for more samples.

    • Many Python samples can be run by using python <script.py> -d /workspace/tensorrt/data.
      For example:
      python onnx_resnet50.py -d /workspace/tensorrt/data
    • The Python API documentation is in the /workspace/tensorrt/doc/python directory.
  • TensorRT 8.5.3.

    The ONNX parser and plug-in libraries that are bundled with this container are built from TensorRT Open Source Software.

The container also includes the following:

Driver Requirements

Release 23.03 is based on CUDA 12.1.0, which requires NVIDIA Driver release 530 or later. However, if you are running on a data center GPU (for example, T4 or any other data center GPU), you can use NVIDIA driver release 450.51 (or later R450), 470.57 (or later R470), 510.47 (or later R510), 515.65 (or later R515), 525.85 (or later R525), or 530.30 (or later R530).

The CUDA driver's compatibility package only supports particular drivers. Thus, users should upgrade from all R418, R440, R460, and R520 drivers, which are not forward-compatible with CUDA 12.1. For a complete list of supported drivers, see the CUDA Application Compatibility topic. For more information, see CUDA Compatibility and Upgrades.

GPU Requirements

Release 23.03 supports CUDA compute capability 3.5 and later. This corresponds to GPUs in the NVIDIA Kepler, Maxwell, NVIDIA Pascal, NVIDIA Volta™, NVIDIA Turing™, NVIDIA Ampere architecture, and NVIDIA Hopper™ architecture families. For a list of GPUs to which this compute capability corresponds, see CUDA GPUs. For additional support details, see the Deep Learning Frameworks Support Matrix.

Key Features and Enhancements

This TensorRT container release includes the following key features and enhancements.
  • TensorRT container image version 23.03 is based on TensorRT 8.5.3.

    For a list of the features and enhancements that were introduced in TensorRT 8.5.3, refer to the TensorRT 8.5 release notes.

  • Ubuntu 20.04 with February 2023 updates.

Announcements

  • Starting with the 22.05 release, the TensorRT container is available for the Arm SBSA platform.

    For example, when you pull the nvcr.io/nvidia/tensorrt:22.05-py3 Docker image on an Arm SBSA machine, the Arm-specific image is automatically fetched.

  • NVIDIA Deep Learning Profiler (DLProf) v1.8, which was included in the 21.12 container, was the last release of DLProf.

    Starting with the 22.01 container, DLProf is longer included. It can still be manually installed by using a pip wheel on the nvidia-pyindex.

Obtaining Missing Data Files

Some samples require data files that are not included in the TensorRT container because of licensing restrictions, or because they are too large. Samples that do not include the required data files include a README.md file in the corresponding source directory that provides information about how to obtain the necessary data files.

Installing Required Python Modules

  • To complete some of the samples, you might want to first run the Python setup script.
  • If you need to install the missing Python modules and their dependencies, run the /opt/tensorrt/python/python_setup.sh script.

Installing Open Source Components

A script has been added to clone, build, and replace the provided plug-in, the Caffe parser, and the ONNX parser libraries with the open source ones that are based on the 22.05 tag on the official TensorRT open source repository.

To install the open source components in the container, run the following script:

/opt/tensorrt/install_opensource.sh -b main

For more information, see GitHub: TensorRT.

Limitations

NVIDIA TensorRT Container Versions

The following table shows what versions of Ubuntu, CUDA, and TensorRT are supported in each of the NVIDIA containers for TensorRT. For older container versions, refer to the Frameworks Support Matrix.

Known Issues

None.

16. TensorRT Release 23.02

The NVIDIA container image for TensorRT, release 23.02, is available on NGC.

Contents of the TensorRT container

This container includes the following:
  • The TensorRT C++ samples and C++ API documentation.
    • Build the samples can be by running make in the /workspace/tensorrt/samples directory.
    • The resulting executables are in the /workspace/tensorrt/bin directory.
    • The C++ API documentation is in the /workspace/tensorrt/doc/cpp directory.
  • The TensorRT Python samples and Python API documentation.
    • The Python samples are in the /workspace/tensorrt/samples/python directory.

      Refer to the respective README documents for more samples.

    • Many Python samples can be run by using python <script.py> -d /workspace/tensorrt/data.
      For example:
      python onnx_resnet50.py -d /workspace/tensorrt/data
    • The Python API documentation is in the /workspace/tensorrt/doc/python directory.
  • TensorRT 8.5.3.

    The ONNX parser and plug-in libraries that are bundled with this container are built from TensorRT Open Source Software.

The container also includes the following:

Driver Requirements

Release 23.02 is based on CUDA 12.0.1, which requires NVIDIA Driver release 525 or later. However, if you are running on a data center GPU (for example, T4 or any other data center GPU), you can use NVIDIA driver release 450.51 (or later R450), 470.57 (or later R470), 510.47 (or later R510), 515.65 (or later R515), or 525.85 (or later R525).

The CUDA driver's compatibility package only supports particular drivers. Thus, users should upgrade from all R418, R440, R460, and R520 drivers, which are not forward-compatible with CUDA 12.0. For a complete list of supported drivers, see the CUDA Application Compatibility topic. For more information, see CUDA Compatibility and Upgrades.

GPU Requirements

Release 23.02 supports CUDA compute capability 3.5 and later. This corresponds to GPUs in the NVIDIA Kepler, Maxwell, NVIDIA Pascal, NVIDIA Volta™, NVIDIA Turing™, NVIDIA Ampere architecture, and NVIDIA Hopper™ architecture families. For a list of GPUs to which this compute capability corresponds, see CUDA GPUs. For additional support details, see the Deep Learning Frameworks Support Matrix.

Key Features and Enhancements

This TensorRT container release includes the following key features and enhancements.
  • TensorRT container image version 23.02 is based on TensorRT 8.5.3.

    For a list of the features and enhancements that were introduced in TensorRT 8.5.3, refer to the TensorRT 8.5 release notes.

  • Ubuntu 20.04 with January 2023 updates.

Announcements

  • Starting with the 22.05 release, the TensorRT container is available for the Arm SBSA platform.

    For example, when you pull the nvcr.io/nvidia/tensorrt:22.05-py3 Docker image on an Arm SBSA machine, the Arm-specific image is automatically fetched.

  • NVIDIA Deep Learning Profiler (DLProf) v1.8, which was included in the 21.12 container, was the last release of DLProf.

    Starting with the 22.01 container, DLProf is longer included. It can still be manually installed by using a pip wheel on the nvidia-pyindex.

Obtaining Missing Data Files

Some samples require data files that are not included in the TensorRT container because of licensing restrictions, or because they are too large. Samples that do not include the required data files include a README.md file in the corresponding source directory that provides information about how to obtain the necessary data files.

Installing Required Python Modules

  • To complete some of the samples, you might want to first run the Python setup script.
  • If you need to install the missing Python modules and their dependencies, run the /opt/tensorrt/python/python_setup.sh script.

Installing Open Source Components

A script has been added to clone, build, and replace the provided plug-in, the Caffe parser, and the ONNX parser libraries with the open source ones that are based on the 22.05 tag on the official TensorRT open source repository.

To install the open source components in the container, run the following script:

/opt/tensorrt/install_opensource.sh -b main
Note: Since the 22.09 release is based on an early access version of TensorRT 8.5, which is not accompanied by the publication of a corresponding TensorRT Open Source Software (OSS) release to GitHub, please specify building from the main branch in install_opensource.sh until the TensorRT OSS 8.5.1 release is posted.

For more information, see GitHub: TensorRT.

Limitations

NVIDIA TensorRT Container Versions

The following table shows what versions of Ubuntu, CUDA, and TensorRT are supported in each of the NVIDIA containers for TensorRT. For older container versions, refer to the Frameworks Support Matrix.

Known Issues

None.

17. TensorRT Release 23.01

The NVIDIA container image for TensorRT, release 23.01, is available on NGC.

Contents of the TensorRT container

This container includes the following:
  • The TensorRT C++ samples and C++ API documentation.
    • Build the samples can be by running make in the /workspace/tensorrt/samples directory.
    • The resulting executables are in the /workspace/tensorrt/bin directory.
    • The C++ API documentation is in the /workspace/tensorrt/doc/cpp directory.
  • The TensorRT Python samples and Python API documentation.
    • The Python samples are in the /workspace/tensorrt/samples/python directory.

      Refer to the respective README documents for more samples.

    • Many Python samples can be run by using python <script.py> -d /workspace/tensorrt/data.
      For example:
      python onnx_resnet50.py -d /workspace/tensorrt/data
    • The Python API documentation is in the /workspace/tensorrt/doc/python directory.
  • TensorRT 8.5.2.2.

    The ONNX parser and plug-in libraries that are bundled with this container are built from TensorRT Open Source Software.

The container also includes the following:

Driver Requirements

Release 23.01 is based on CUDA 12.0.1, which requires NVIDIA Driver release 525 or later. However, if you are running on a data center GPU (for example, T4 or any other data center GPU), you can use NVIDIA driver release 450.51 (or later R450), 470.57 (or later R470), 510.47 (or later R510), 515.65 (or later R515), or 525.85 (or later R525).

The CUDA driver's compatibility package only supports particular drivers. Thus, users should upgrade from all R418, R440, and R460 drivers, which are not forward-compatible with CUDA 12.0. For a complete list of supported drivers, see the CUDA Application Compatibility topic. For more information, see CUDA Compatibility and Upgrades.

GPU Requirements

Release 23.01 supports CUDA compute capability 3.5 and later. This corresponds to GPUs in the NVIDIA Kepler, Maxwell, NVIDIA Pascal, NVIDIA Volta™, NVIDIA Turing™, NVIDIA Ampere architecture, and NVIDIA Hopper™ architecture families. For a list of GPUs to which this compute capability corresponds, see CUDA GPUs. For additional support details, see the Deep Learning Frameworks Support Matrix.

Key Features and Enhancements

This TensorRT container release includes the following key features and enhancements.
  • TensorRT container image version 23.01 is based on TensorRT 8.5.2.2.

    For a list of the features and enhancements that were introduced in TensorRT 8.5.2, refer to the TensorRT 8.5 release notes.

  • Ubuntu 20.04 with December 2022 updates.

Announcements

  • Starting with the 22.05 release, the TensorRT container is available for the Arm SBSA platform.

    For example, when you pull the nvcr.io/nvidia/tensorrt:22.05-py3 Docker image on an Arm SBSA machine, the Arm-specific image is automatically fetched.

  • NVIDIA Deep Learning Profiler (DLProf) v1.8, which was included in the 21.12 container, was the last release of DLProf.

    Starting with the 22.01 container, DLProf is longer included. It can still be manually installed by using a pip wheel on the nvidia-pyindex.

Obtaining Missing Data Files

Some samples require data files that are not included in the TensorRT container because of licensing restrictions, or because they are too large. Samples that do not include the required data files include a README.md file in the corresponding source directory that provides information about how to obtain the necessary data files.

Installing Required Python Modules

  • To complete some of the samples, you might want to first run the Python setup script.
  • If you need to install the missing Python modules and their dependencies, run the /opt/tensorrt/python/python_setup.sh script.

Installing Open Source Components

A script has been added to clone, build, and replace the provided plug-in, the Caffe parser, and the ONNX parser libraries with the open source ones that are based on the 22.05 tag on the official TensorRT open source repository.

To install the open source components in the container, run the following script:

/opt/tensorrt/install_opensource.sh -b main
Note: Since the 22.09 release is based on an early access version of TensorRT 8.5, which is not accompanied by the publication of a corresponding TensorRT Open Source Software (OSS) release to GitHub, please specify building from the main branch in install_opensource.sh until the TensorRT OSS 8.5.1 release is posted.

For more information, see GitHub: TensorRT.

Limitations

Known Issues

None.

18. TensorRT Release 22.12

The NVIDIA container image for TensorRT, release 22.12, is available on NGC.

Contents of the TensorRT container

This container includes the following:
  • The TensorRT C++ samples and C++ API documentation.
    • Build the samples can be by running make in the /workspace/tensorrt/samples directory.
    • The resulting executables are in the /workspace/tensorrt/bin directory.
    • The C++ API documentation is in the /workspace/tensorrt/doc/cpp directory.
  • The TensorRT Python samples and Python API documentation.
    • The Python samples are in the /workspace/tensorrt/samples/python directory.

      Refer to the respective README documents for more samples.

    • Many Python samples can be run by using python <script.py> -d /workspace/tensorrt/data.
      For example:
      python onnx_resnet50.py -d /workspace/tensorrt/data
    • The Python API documentation is in the /workspace/tensorrt/doc/python directory.
  • TensorRT 8.5.1.

    The ONNX parser and plug-in libraries that are bundled with this container are built from TensorRT Open Source Software.

The container also includes the following:

Driver Requirements

Release 22.12 is based on CUDA 11.8.0, which requires NVIDIA Driver release 520 or later. However, if you are running on a data center GPU (for example, T4 or any other data center GPU), you can use NVIDIA driver release 450.51 (or later R450), 470.57 (or later R470), 510.47 (or later R510), or 515.65 (or later R515).

The CUDA driver's compatibility package only supports particular drivers. Thus, users should upgrade from all R418, R440, and R460 drivers, which are not forward-compatible with CUDA 11.8. For a complete list of supported drivers, see the CUDA Application Compatibility topic. For more information, see CUDA Compatibility and Upgrades.

GPU Requirements

Release 22.12 supports CUDA compute capability 3.5 and later. This corresponds to GPUs in the NVIDIA Kepler, Maxwell, NVIDIA Pascal, NVIDIA Volta™, NVIDIA Turing™, NVIDIA Ampere architecture, and NVIDIA Hopper™ architecture families. For a list of GPUs to which this compute capability corresponds, see CUDA GPUs. For additional support details, see the Deep Learning Frameworks Support Matrix.

Key Features and Enhancements

This TensorRT container release includes the following key features and enhancements.
  • TensorRT container image version 22.12 is based on TensorRT 8.5.1.

    For a list of the features and enhancements that were introduced in TensorRT 8.5.1, refer to the TensorRT 8.5 release notes.

  • Ubuntu 20.04 with November 2022 updates.

Announcements

  • Starting with the 22.05 release, the TensorRT container is available for the Arm SBSA platform.

    For example, when you pull the nvcr.io/nvidia/tensorrt:22.05-py3 Docker image on an Arm SBSA machine, the Arm-specific image is automatically fetched.

  • NVIDIA Deep Learning Profiler (DLProf) v1.8, which was included in the 21.12 container, was the last release of DLProf.

    Starting with the 22.01 container, DLProf is longer included. It can still be manually installed by using a pip wheel on the nvidia-pyindex.

Obtaining Missing Data Files

Some samples require data files that are not included in the TensorRT container because of licensing restrictions, or because they are too large. Samples that do not include the required data files include a README.md file in the corresponding source directory that provides information about how to obtain the necessary data files.

Installing Required Python Modules

  • To complete some of the samples, you might want to first run the Python setup script.
  • If you need to install the missing Python modules and their dependencies, run the /opt/tensorrt/python/python_setup.sh script.

Installing Open Source Components

A script has been added to clone, build, and replace the provided plug-in, the Caffe parser, and the ONNX parser libraries with the open source ones that are based on the 22.05 tag on the official TensorRT open source repository.

To install the open source components in the container, run the following script:

/opt/tensorrt/install_opensource.sh -b main
Note: Since the 22.09 release is based on an early access version of TensorRT 8.5, which is not accompanied by the publication of a corresponding TensorRT Open Source Software (OSS) release to GitHub, please specify building from the main branch in install_opensource.sh until the TensorRT OSS 8.5.1 release is posted.

For more information, see GitHub: TensorRT.

Limitations

Known Issues

None.

19. TensorRT Release 22.11

The NVIDIA container image for TensorRT, release 22.10, is available on NGC.

Contents of the TensorRT container

This container includes the following:
  • The TensorRT C++ samples and C++ API documentation.
    • Build the samples can be by running make in the /workspace/tensorrt/samples directory.
    • The resulting executables are in the /workspace/tensorrt/bin directory.
    • The C++ API documentation is in the /workspace/tensorrt/doc/cpp directory.
  • The TensorRT Python samples and Python API documentation.
    • The Python samples are in the /workspace/tensorrt/samples/python directory.

      Refer to the respective README documents for more samples.

    • Many Python samples can be run by using python <script.py> -d /workspace/tensorrt/data.
      For example:
      python onnx_resnet50.py -d /workspace/tensorrt/data
    • The Python API documentation is in the /workspace/tensorrt/doc/python directory.
  • TensorRT 8.5.1.

    The ONNX parser and plug-in libraries that are bundled with this container are built from TensorRT Open Source Software.

The container also includes the following:

Driver Requirements

Release 22.11 is based on CUDA 11.8.0, which requires NVIDIA Driver release 520 or later. However, if you are running on a data center GPU (for example, T4 or any other data center GPU), you can use NVIDIA driver release 450.51 (or later R450), 470.57 (or later R470), 510.47 (or later R510), or 515.65 (or later R515).

The CUDA driver's compatibility package only supports particular drivers. Thus, users should upgrade from all R418, R440, and R460 drivers, which are not forward-compatible with CUDA 11.8. For a complete list of supported drivers, see the CUDA Application Compatibility topic. For more information, see CUDA Compatibility and Upgrades.

GPU Requirements

Release 22.11 supports CUDA compute capability 3.5 and later. This corresponds to GPUs in the NVIDIA Kepler, Maxwell, NVIDIA Pascal, NVIDIA Volta™, NVIDIA Turing™, NVIDIA Ampere architecture, and NVIDIA Hopper™ architecture families. For a list of GPUs to which this compute capability corresponds, see CUDA GPUs. For additional support details, see the Deep Learning Frameworks Support Matrix.

Key Features and Enhancements

This TensorRT container release includes the following key features and enhancements.
  • TensorRT container image version 22.11 is based on TensorRT 8.5.1.

    For a list of the features and enhancements that were introduced in TensorRT 8.5.1, refer to the TensorRT 8.5 release notes.

  • Ubuntu 20.04 with October 2022 updates.

Announcements

  • Starting with the 22.05 release, the TensorRT container is available for the Arm SBSA platform.

    For example, when you pull the nvcr.io/nvidia/tensorrt:22.05-py3 Docker image on an Arm SBSA machine, the Arm-specific image is automatically fetched.

  • NVIDIA Deep Learning Profiler (DLProf) v1.8, which was included in the 21.12 container, was the last release of DLProf.

    Starting with the 22.01 container, DLProf is longer included. It can still be manually installed by using a pip wheel on the nvidia-pyindex.

Obtaining Missing Data Files

Some samples require data files that are not included in the TensorRT container because of licensing restrictions, or because they are too large. Samples that do not include the required data files include a README.md file in the corresponding source directory that provides information about how to obtain the necessary data files.

Installing Required Python Modules

  • To complete some of the samples, you might want to first run the Python setup script.
  • If you need to install the missing Python modules and their dependencies, run the /opt/tensorrt/python/python_setup.sh script.

Installing Open Source Components

A script has been added to clone, build, and replace the provided plug-in, the Caffe parser, and the ONNX parser libraries with the open source ones that are based on the 22.05 tag on the official TensorRT open source repository.

To install the open source components in the container, run the following script:

/opt/tensorrt/install_opensource.sh -b main
Note: Since the 22.09 release is based on an early access version of TensorRT 8.5, which is not accompanied by the publication of a corresponding TensorRT Open Source Software (OSS) release to GitHub, please specify building from the main branch in install_opensource.sh until the TensorRT OSS 8.5.1 release is posted.

For more information, see GitHub: TensorRT.

Limitations

Known Issues

None.

20. TensorRT Release 22.10

The NVIDIA container image for TensorRT, release 22.10, is available on NGC.

Contents of the TensorRT container

This container includes the following:
  • The TensorRT C++ samples and C++ API documentation.
    • Build the samples can be by running make in the /workspace/tensorrt/samples directory.
    • The resulting executables are in the /workspace/tensorrt/bin directory.
    • The C++ API documentation is in the /workspace/tensorrt/doc/cpp directory.
  • The TensorRT Python samples and Python API documentation.
    • The Python samples are in the /workspace/tensorrt/samples/python directory.

      Refer to the respective README documents for more samples.

    • Many Python samples can be run by using python <script.py> -d /workspace/tensorrt/data.
      For example:
      python onnx_resnet50.py -d /workspace/tensorrt/data
    • The Python API documentation is in the /workspace/tensorrt/doc/python directory.
  • TensorRT 8.5 EA.

    The ONNX parser and plug-in libraries that are bundled with this container are built from TensorRT Open Source Software.

The container also includes the following:

Driver Requirements

Release 22.10 is based on CUDA 11.8.0, which requires NVIDIA Driver release 520 or later. However, if you are running on a data center GPU (for example, T4 or any other data center GPU), you can use NVIDIA driver release 450.51 (or later R450), 470.57 (or later R470), 510.47 (or later R510), or 515.65 (or later R515).

The CUDA driver's compatibility package only supports particular drivers. Thus, users should upgrade from all R418, R440, and R460 drivers, which are not forward-compatible with CUDA 11.8. For a complete list of supported drivers, see the CUDA Application Compatibility topic. For more information, see CUDA Compatibility and Upgrades.

GPU Requirements

Release 22.10 supports CUDA compute capability 3.5 and later. This corresponds to GPUs in the NVIDIA Kepler, Maxwell, NVIDIA Pascal, NVIDIA Volta™, NVIDIA Turing™, NVIDIA Ampere architecture, and NVIDIA Hopper™ architecture families. For a list of GPUs to which this compute capability corresponds, see CUDA GPUs. For additional support details, see the Deep Learning Frameworks Support Matrix.

Key Features and Enhancements

This TensorRT container release includes the following key features and enhancements.
  • TensorRT container image version 22.10 is based on TensorRT 8.5 EA.

    For a list of the features and enhancements that were introduced in TensorRT 8.5.0.12, refer to the TensorRT 8.5 release notes.

  • Ubuntu 20.04 with September 2022 updates.

Announcements

  • Starting with the 22.05 release, the TensorRT container is available for the Arm SBSA platform.

    For example, when you pull the nvcr.io/nvidia/tensorrt:22.05-py3 Docker image on an Arm SBSA machine, the Arm-specific image is automatically fetched.

  • NVIDIA Deep Learning Profiler (DLProf) v1.8, which was included in the 21.12 container, was the last release of DLProf.

    Starting with the 22.01 container, DLProf is longer included. It can still be manually installed by using a pip wheel on the nvidia-pyindex.

Obtaining Missing Data Files

Some samples require data files that are not included in the TensorRT container because of licensing restrictions, or because they are too large. Samples that do not include the required data files include a README.md file in the corresponding source directory that provides information about how to obtain the necessary data files.

Installing Required Python Modules

  • To complete some of the samples, you might want to first run the Python setup script.
  • If you need to install the missing Python modules and their dependencies, run the /opt/tensorrt/python/python_setup.sh script.

Installing Open Source Components

A script has been added to clone, build, and replace the provided plug-in, the Caffe parser, and the ONNX parser libraries with the open source ones that are based on the 22.05 tag on the official TensorRT open source repository.

To install the open source components in the container, run the following script:

/opt/tensorrt/install_opensource.sh -b main
Note: Since the 22.09 release is based on an early access version of TensorRT 8.5, which is not accompanied by the publication of a corresponding TensorRT Open Source Software (OSS) release to GitHub, please specify building from the main branch in install_opensource.sh until the TensorRT OSS 8.5.1 release is posted.

For more information, see GitHub: TensorRT.

Limitations

Known Issues

None.

21. TensorRT Release 22.09

The NVIDIA container image for TensorRT, release 22.09, is available on NGC.

Contents of the TensorRT container

This container includes the following:
  • The TensorRT C++ samples and C++ API documentation.
    • Build the samples can be by running make in the /workspace/tensorrt/samples directory.
    • The resulting executables are in the /workspace/tensorrt/bin directory.
    • The C++ API documentation is in the /workspace/tensorrt/doc/cpp directory.
  • The TensorRT Python samples and Python API documentation.
    • The Python samples are in the /workspace/tensorrt/samples/python directory.

      Refer to the respective README documents for more samples.

    • Many Python samples can be run by using python <script.py> -d /workspace/tensorrt/data.
      For example:
      python onnx_resnet50.py -d /workspace/tensorrt/data
    • The Python API documentation is in the /workspace/tensorrt/doc/python directory.
  • TensorRT 8.5 EA.

    The ONNX parser and plug-in libraries that are bundled with this container are built from TensorRT Open Source Software.

The container also includes the following:

Driver Requirements

Release 22.09 is based on CUDA 11.8.0, which requires NVIDIA Driver release 520 or later. However, if you are running on a data center GPU (for example, T4 or any other data center GPU), you can use NVIDIA driver release 450.51 (or later R450), 470.57 (or later R470), 510.47 (or later R510), or 515.65 (or later R515).

The CUDA driver's compatibility package only supports particular drivers. Thus, users should upgrade from all R418, R440, and R460 drivers, which are not forward-compatible with CUDA 11.8. For a complete list of supported drivers, see the CUDA Application Compatibility topic. For more information, see CUDA Compatibility and Upgrades.

GPU Requirements

Release 22.09 supports CUDA compute capability 3.5 and later. This corresponds to GPUs in the NVIDIA Kepler, Maxwell, NVIDIA Pascal, NVIDIA Volta™, NVIDIA Turing™, NVIDIA Ampere architecture, and NVIDIA Hopper™ architecture families. For a list of GPUs to which this compute capability corresponds, see CUDA GPUs. For additional support details, see the Deep Learning Frameworks Support Matrix.

Key Features and Enhancements

This TensorRT container release includes the following key features and enhancements.
  • TensorRT container image version 22.09 is based on TensorRT 8.5 EA.

    For a list of the features and enhancements that were introduced in TensorRT 8.5.0.12, refer to the TensorRT 8.5 release notes.

  • Ubuntu 20.04 with August 2022 updates.

Announcements

  • Starting with the 22.05 release, the TensorRT container is available for the Arm SBSA platform.

    For example, when you pull the nvcr.io/nvidia/tensorrt:22.05-py3 Docker image on an Arm SBSA machine, the Arm-specific image is automatically fetched.

  • NVIDIA Deep Learning Profiler (DLProf) v1.8, which was included in the 21.12 container, was the last release of DLProf.

    Starting with the 22.01 container, DLProf is longer included. It can still be manually installed by using a pip wheel on the nvidia-pyindex.

Obtaining Missing Data Files

Some samples require data files that are not included in the TensorRT container because of licensing restrictions, or because they are too large. Samples that do not include the required data files include a README.md file in the corresponding source directory that provides information about how to obtain the necessary data files.

Installing Required Python Modules

  • To complete some of the samples, you might want to first run the Python setup script.
  • If you need to install the missing Python modules and their dependencies, run the /opt/tensorrt/python/python_setup.sh script.

Installing Open Source Components

A script has been added to clone, build, and replace the provided plug-in, the Caffe parser, and the ONNX parser libraries with the open source ones that are based on the 22.05 tag on the official TensorRT open source repository.

To install the open source components in the container, run the following script:

/opt/tensorrt/install_opensource.sh -b main
Note: Since the 22.09 release is based on an early access version of TensorRT 8.5, which is not accompanied by the publication of a corresponding TensorRT Open Source Software (OSS) release to GitHub, please specify building from the main branch in install_opensource.sh until the TensorRT OSS 8.5.1 release is posted.

For more information, see GitHub: TensorRT.

Limitations

Known Issues

None.

22. TensorRT Release 22.08

The NVIDIA container image for TensorRT, release 22.08, is available on NGC.

Contents of the TensorRT container

This container includes the following:
  • The TensorRT C++ samples and C++ API documentation.
    • Build the samples can be by running make in the /workspace/tensorrt/samples directory.
    • The resulting executables are in the /workspace/tensorrt/bin directory.
    • The C++ API documentation is in the /workspace/tensorrt/doc/cpp directory.
  • The TensorRT Python samples and Python API documentation.
    • The Python samples are in the /workspace/tensorrt/samples/python directory.

      Refer to the respective README documents for more samples.

    • Many Python samples can be run by using python <script.py> -d /workspace/tensorrt/data.
      For example:
      python onnx_resnet50.py -d /workspace/tensorrt/data
    • The Python API documentation is in the /workspace/tensorrt/doc/python directory.
  • TensorRT 8.4.2.4.
    The ONNX parser and plug-in libraries that are bundled with this container are built from TensorRT Open Source Software:
The container also includes the following:

Driver Requirements

Release 22.08 is based on CUDA 11.7.1, which requires NVIDIA Driver release 515 or later. However, if you are running on a data center GPU (for example, T4 or any other data center GPU), you can use NVIDIA driver release 450.51 (or later R450), 470.57 (or later R470), or 510.47 (or later R510).

The CUDA driver's compatibility package only supports particular drivers. Thus, users should upgrade from all R418, R440, and R460 drivers, which are not forward-compatible with CUDA 11.7. For a complete list of supported drivers, see the CUDA Application Compatibility topic. For more information, see CUDA Compatibility and Upgrades.

GPU Requirements

Release 22.08 supports CUDA compute capability 3.5 and later. This corresponds to GPUs in the Kepler, Maxwell, Pascal, Volta, Turing, and NVIDIA Ampere Architecture GPU families. For a list of GPUs to which this compute capability corresponds, see CUDA GPUs. For additional support details, see the Deep Learning Frameworks Support Matrix.

Key Features and Enhancements

This TensorRT container release includes the following key features and enhancements.
  • TensorRT container image version 22.08 is based on TensorRT 8.4.2.4.

    For a list of the features and enhancements that were introduced in TensorRT 8.4.2.4, refer to the TensorRT 8.4.2 release notes.

  • Ubuntu 20.04 with July 2022 updates.

Announcements

  • Starting with the 22.05 release, the TensorRT container is available for the Arm SBSA platform.

    For example, when you pull the nvcr.io/nvidia/tensorrt:22.05-py3 Docker image on an Arm SBSA machine, the Arm-specific image is automatically fetched.

  • NVIDIA Deep Learning Profiler (DLProf) v1.8, which was included in the 21.12 container, was the last release of DLProf.

    Starting with the 22.01 container, DLProf is longer included. It can still be manually installed by using a pip wheel on the nvidia-pyindex.

Obtaining Missing Data Files

Some samples require data files that are not included in the TensorRT container because of licensing restrictions, or because they are too large. Samples that do not include the required data files include a README.md file in the corresponding source directory that provides information about how to obtain the necessary data files.

Installing Required Python Modules

  • To complete some of the samples, you might want to first run the Python setup script.
  • If you need to install the missing Python modules and their dependencies, run the /opt/tensorrt/python/python_setup.sh script.

Installing Open Source Components

A script has been added to clone, build, and replace the provided plug-in, the Caffe parser, and the ONNX parser libraries with the open source ones that are based on the 22.05 tag on the official TensorRT open source repository.

To install the open source components in the container, run the following script:

/opt/tensorrt/install_opensource.sh

For more information, see GitHub: TensorRT 22.08.

Limitations

Known Issues

None.

23. TensorRT Release 22.07

The NVIDIA container image for TensorRT, release 22.07, is available on NGC.

Contents of the TensorRT container

This container includes the following:
  • The TensorRT C++ samples and C++ API documentation.
    • Build the samples can be by running make in the /workspace/tensorrt/samples directory.
    • The resulting executables are in the /workspace/tensorrt/bin directory.
    • The C++ API documentation is in the /workspace/tensorrt/doc/cpp directory.
  • The TensorRT Python samples and Python API documentation.
    • The Python samples are in the /workspace/tensorrt/samples/python directory.

      Refer to the respective README documents for more samples.

    • Many Python samples can be run by using python <script.py> -d /workspace/tensorrt/data.
      For example:
      python onnx_resnet50.py -d /workspace/tensorrt/data
    • The Python API documentation is in the /workspace/tensorrt/doc/python directory.
  • TensorRT 8.4.1.
    The ONNX parser and plug-in libraries that are bundled with this container are built from TensorRT Open Source Software:
The container also includes the following:

Driver Requirements

Release 22.07 is based on CUDA 11.7 Update 1 Preview, which requires NVIDIA Driver release 515 or later. However, if you are running on a data center GPU (for example, T4 or any other data center GPU), you can use NVIDIA driver release 450.51 (or later R450), 470.57 (or later R470), or 510.47 (or later R510).

The CUDA driver's compatibility package only supports particular drivers. Thus, users should upgrade from all R418, R440, and R460 drivers, which are not forward-compatible with CUDA 11.7. For a complete list of supported drivers, see the CUDA Application Compatibility topic. For more information, see CUDA Compatibility and Upgrades.

GPU Requirements

Release 22.07 supports CUDA compute capability 3.5 and later. This corresponds to GPUs in the Kepler, Maxwell, Pascal, Volta, Turing, and NVIDIA Ampere Architecture GPU families. For a list of GPUs to which this compute capability corresponds, see CUDA GPUs. For additional support details, see the Deep Learning Frameworks Support Matrix.

Key Features and Enhancements

This TensorRT container release includes the following key features and enhancements.
  • TensorRT container image version 22.07 is based on TensorRT 8.4.1.

    For a list of the features and enhancements that were introduced in TensorRT 8.4.1, refer to the TensorRT 8.4.1 release notes.

  • Ubuntu 20.04 with June 2022 updates.

Announcements

  • Starting with the 22.05 release, the TensorRT container is available for the Arm SBSA platform.

    For example, when you pull the nvcr.io/nvidia/tensorrt:22.05-py3 Docker image on an Arm SBSA machine, the Arm-specific image is automatically fetched.

  • NVIDIA Deep Learning Profiler (DLProf) v1.8, which was included in the 21.12 container, was the last release of DLProf.

    Starting with the 22.01 container, DLProf is longer included. It can still be manually installed by using a pip wheel on the nvidia-pyindex.

Obtaining Missing Data Files

Some samples require data files that are not included in the TensorRT container because of licensing restrictions, or because they are too large. Samples that do not include the required data files include a README.md file in the corresponding source directory that provides information about how to obtain the necessary data files.

Installing Required Python Modules

  • To complete some of the samples, you might want to first run the Python setup script.
  • If you need to install the missing Python modules and their dependencies, run the /opt/tensorrt/python/python_setup.sh script.

Installing Open Source Components

A script has been added to clone, build, and replace the provided plug-in, the Caffe parser, and the ONNX parser libraries with the open source ones that are based on the 22.05 tag on the official TensorRT open source repository.

To install the open source components in the container, run the following script:

/opt/tensorrt/install_opensource.sh

For more information, see GitHub: TensorRT 22.07.

Limitations