Installing TensorRT

This guide provides step-by-step instructions for installing TensorRT using various methods. Choose the installation method that best fits your development environment and deployment needs.

Before You Begin: Ensure you have reviewed the Prerequisites to confirm your system meets all requirements.

Installation Method Comparison

Quick Comparison Table:

Method	Best For	Requires Root	C++ Headers	Multi-Version	Installation Time
pip (Python)	Python development	No	No	Yes (venv)	⚡ Fastest (~2 min)
Debian/RPM	System-wide install	Yes	Yes	No	🔵 Fast (~5 min)
Tar/Zip	Multiple versions	No	Yes	Yes	🟡 Moderate (~10 min)
Container (NGC)	Isolated environments	No (Docker)	Yes	Yes	⚡ Fastest (~5 min)

Choosing Your Installation Method

Choose pip if you:

• Are developing primarily in Python

• Want the fastest installation

• Are working in a Python virtual environment

• Don’t need C++ development headers

Choose Debian/RPM if you:

• Want system-wide installation with automatic updates

• Need C++ development support

• Have sudo/root access

• Prefer standard Linux package management

Choose Tar/Zip if you:

• Need multiple TensorRT versions simultaneously

• Want control over installation location

• Are installing without root privileges

• Need C++ headers but want flexibility

Choose Container if you:

• Want a pre-configured environment

• Are deploying in Kubernetes or Docker

• Need consistent environments across systems

• Want to avoid dependency management

Understanding TensorRT Packages

TensorRT offers three runtime configurations:

Full Runtime (Recommended for Development)

• Complete builder and runtime functionality

• Build and optimize models

• Run inference

• Size: Largest footprint (~2 GB)

• Package names: tensorrt (pip), tensorrt (deb/rpm), TensorRT-* (tar/zip)

Lean Runtime (Recommended for Production Deployment)

• Runtime-only functionality

• Run pre-built engines

• Cannot build new engines

• Size: Medium footprint (~500 MB)

• Package names: tensorrt_lean (pip), tensorrt-lean (deb/rpm)

• Requirements: Engines must be built with version-compatible builder flag

Dispatch Runtime (Recommended for Minimal Footprint Deployment)

• Minimal runtime functionality

• Run pre-built engines with smallest footprint

• Includes lean runtime

• Size: Smallest footprint (~200 MB)

• Package names: tensorrt_dispatch (pip), tensorrt-dispatch (deb/rpm)

• Requirements: Engines must be built with version-compatible builder flag

Component Versioning

TensorRT is a product made up of separately versioned components. The product version conveys important information about the significance of new features, while the library version conveys information about the compatibility or incompatibility of the API.

The following table shows product versions and corresponding component library versions for TensorRT 10.15.1, illustrating semantic versioning patterns for each component.

Table 2 TensorRT Product and Component Version Matrix

Product/Component		Previous Released Version	Current Version	Version Description
TensorRT product		10.14.1	10.15.1	+1.0.0 when significant new capabilities are added. +0.1.0 when capabilities have been improved.
nvinfer documents, headers, and documentations		10.14.1	10.15.1	+1.0.0 when the API or ABI changes in a non-compatible way. +0.1.0 when the API or ABI changes are backward compatible.
nvinfer-lean lean runtime library		10.14.1	10.15.1	+1.0.0 when the API or ABI changes in a non-compatible way. +0.1.0 when the API or ABI changes are backward compatible.
nvinfer-dispatch dispatch runtime library		10.14.1	10.15.1	+1.0.0 when the API or ABI changes in a non-compatible way. +0.1.0 when the API or ABI changes are backward compatible.
libnvinfer Python packages	python3-libnvinfer python3-libnvinfer-dev Debian and RPM packages	10.14.1	10.15.1	+1.0.0 when the API or ABI changes in a non-compatible way. +0.1.0 when the API or ABI changes are backward compatible.
libnvinfer Python package	tensorrt-*.whl file for standard TensorRT runtime	10.14.1	10.15.1	+1.0.0 when the API or ABI changes in a non-compatible way. +0.1.0 when the API or ABI changes are backward compatible.
libnvinfer Python package	tensorrt_lean-*.whl file for lean TensorRT runtime	10.14.1	10.15.1	+1.0.0 when the API or ABI changes in a non-compatible way. +0.1.0 when the API or ABI changes are backward compatible.
libnvinfer Python package	tensorrt_dispatch-*.whl file for dispatch TensorRT runtime	10.14.1	10.15.1	+1.0.0 when the API or ABI changes in a non-compatible way. +0.1.0 when the API or ABI changes are backward compatible.

Downloading TensorRT

Before installing with Debian (local repo), RPM (local repo), Tar, or Zip methods, you must download TensorRT packages. Prerequisites:

• NVIDIA Developer Program membership (free)

• Account login

Download Steps:

1. Go to https://developer.nvidia.com/tensorrt.

2. Click GET STARTED, then Download Now.

3. Select TensorRT version 10.15.1 (or your target version).

4. Accept the license agreement.

5. Download the package for your platform:

   • Linux x86-64: Debian local repo (.deb), RPM local repo (.rpm), or Tar (.tar.gz)

   • Linux ARM SBSA: Debian local repo or Tar

   • Windows x64: Zip (.zip)

Tip

For pip installation: Skip this section. The pip method downloads packages automatically from PyPI.

Installation Methods

This section provides detailed instructions for each installation method. Jump to the method you selected:

• Python Package Index (pip) - Fastest for Python users

• Debian Packages - Ubuntu/Debian systems

• RPM Packages - RHEL/CentOS/Fedora systems

• Tar File - Flexible cross-platform installation

• Zip File - Windows installation

• Container Images - Docker/Kubernetes deployments

• Alternative Methods - NIM, JetPack, DriveOS, Nsight

Method 1: Python Package Index (pip)

Recommended for: Python developers, quick prototyping, virtual environments

Advantages:

✓ Fastest installation method ✓ No manual downloads required ✓ Automatic dependency management ✓ Works in virtual environments ✓ No root access needed

Limitations:

✗ No C++ development headers ✗ Python-only (no standalone C++ apps) ✗ May install redundant libraries if C++ TensorRT already installed

Platform Support

Supported Configurations:

• Python versions: 3.8, 3.9, 3.10, 3.11, 3.12, 3.13

• Operating Systems:

  • Linux x86-64: Ubuntu 22.04+, RHEL 8+

  • Linux ARM SBSA: Ubuntu 22.04+

  • Windows x64: Windows 10+

Attention

Python 3.10+ is required for TensorRT samples. Python 3.8/3.9 support bindings only.

Installation Steps

Step 1: Update pip and install wheel

python3 -m pip install --upgrade pip
python3 -m pip install wheel

Step 2: Install TensorRT

Full Runtime (Development)

Install the complete TensorRT package with builder and runtime:

python3 -m pip install --upgrade tensorrt

This installs:

• tensorrt-libs: TensorRT core libraries

• tensorrt-bindings: Python bindings matching your Python version

Lean Runtime (Deployment)

Install only the lean runtime for running pre-built engines:

python3 -m pip install --upgrade tensorrt-lean

Dispatch Runtime (Minimal Deployment)

Install the minimal dispatch runtime:

python3 -m pip install --upgrade tensorrt-dispatch

CUDA Version Selection:

By default, TensorRT Python packages install the CUDA 13.x variants (the latest CUDA version supported by TensorRT). If you need a specific CUDA major version, append -cu12 or -cu13 to the package name:

# For CUDA 12.x
python3 -m pip install --upgrade tensorrt-cu12

# For CUDA 13.x (default, explicit)
python3 -m pip install --upgrade tensorrt-cu13

# Lean and Dispatch runtimes with CUDA 12.x
python3 -m pip install --upgrade tensorrt-lean-cu12 tensorrt-dispatch-cu12

Tip

For upgrading an existing installation, clear the pip cache first:

python3 -m pip cache remove "tensorrt*"
python3 -m pip install --upgrade tensorrt tensorrt-lean tensorrt-dispatch

Verification

Verify Full Runtime (if installed):

To confirm your installation is working:

• Import the tensorrt module

• Check the installed version

• Create a Builder object to verify CUDA installation

import tensorrt as trt
print(trt.__version__)
assert trt.Builder(trt.Logger())

Expected output:

10.x.x

Verify Lean Runtime (if installed):

import tensorrt_lean as trt
print(trt.__version__)
assert trt.Runtime(trt.Logger())

Verify Dispatch Runtime (if installed):

import tensorrt_dispatch as trt
print(trt.__version__)
assert trt.Runtime(trt.Logger())

Next Steps:

If the verification commands above worked successfully, you can now run TensorRT Python samples to further confirm your installation. For more information about TensorRT samples, refer to the TensorRT Sample Support Guide.

Troubleshooting

Issue: ModuleNotFoundError: No module named 'tensorrt'

• Solution: Ensure you are in the correct Python environment. Activate your virtual environment if using one.

Issue: Need C++ headers or samples

• Solution: pip installation does not include C++ headers. Use Debian Packages, RPM Packages, Tar File, or Zip File installation instead.

Issue: User-specific installation (no root)

• Solution: Use --user flag:

  python3 -m pip install --user tensorrt
  

Issue: Encounter a TypeError while executing pip install

• Solution: Update the setuptools and packaging Python modules:

  .. code-block:: bash
  

  python3 -m pip install –upgrade setuptools packaging

Issue: CUDA initialization failure during verification

If the verification command fails with an error similar to:

[TensorRT] ERROR: CUDA initialization failure with error 100. Please check your CUDA installation: ...

• Solution: This indicates the NVIDIA driver is not installed or not functioning properly. Check the following:

  1. Verify the NVIDIA driver is installed:

     nvidia-smi
     

  2. If running inside a container, ensure you are using a base image with GPU support, such as nvidia/cuda:13.1.0-base-ubuntu24.04.

  3. Reinstall or update the NVIDIA driver if necessary. Refer to the NVIDIA Driver Downloads page.

Method 2: Debian Package Installation

Recommended for: System-wide installation, C++ and Python development, Ubuntu/Debian users

Advantages:

✓ Automatic dependency installation ✓ Integrates with system package manager ✓ Includes C++ headers and samples ✓ Easy updates with apt

Limitations:

✗ Requires sudo or root privileges ✗ Fixed installation location (/usr) ✗ Only one minor version of TensorRT can be installed at a time ✗ Linux only (Ubuntu/Debian)

Platform Support

Supported Operating Systems:

• Ubuntu 22.04 (x86-64)

• Ubuntu 24.04 (x86-64, ARM SBSA)

• Debian 12 (x86-64, ARM SBSA)

Prerequisites:

• CUDA Toolkit installed using Debian packages

• sudo or root access

Tip

Choose your installation method:

• Local Repo (below): For new users or complete developer installation

• Network Repo (see Network Repo Method): For advanced users, containers, or automation

Installation Steps (Local Repo Method)

Step 1: Download the TensorRT Debian repository package

From the TensorRT download page, download the Debian repository package for your CUDA version and OS.

Example filename: nv-tensorrt-local-repo-ubuntu2404-10.15.1-cuda-13.1_1.0-1_amd64.deb

Replace 10.x.x with your TensorRT version and cuda-x.x with your CUDA version. Replace amd64 with arm64 for ARM SBSA.

Step 2: Install the repository package

sudo dpkg -i nv-tensorrt-local-repo-ubuntu2404-10.x.x-cuda-x.x_1.0-1_amd64.deb

Step 3: Copy the keyring

sudo cp /var/nv-tensorrt-local-repo-ubuntu2404-10.x.x-cuda-x.x/*-keyring.gpg /usr/share/keyrings/

Step 4: Update the package index

sudo apt-get update

Step 5: Install TensorRT packages

Choose the package type that matches your needs:

Table 3 Package Installation Options

Package Type	Installation Command
Full C++ and Python Runtime	sudo apt-get install tensorrt Installs all TensorRT components including runtime, development headers, and Python bindings.
Lean Runtime Only	sudo apt-get install libnvinfer-lean10 sudo apt-get install libnvinfer-vc-plugin10
Lean Runtime Python Package	sudo apt-get install python3-libnvinfer-lean
Dispatch Runtime Only	sudo apt-get install libnvinfer-dispatch10 sudo apt-get install libnvinfer-vc-plugin10
Dispatch Runtime Python Package	sudo apt-get install python3-libnvinfer-dispatch
Windows Builder Resource Library (for cross-platform support)	sudo apt-get install libnvinfer-win-builder-resource10
All TensorRT Python Packages	python3 -m pip install numpy sudo apt-get install python3-libnvinfer-dev This installs: python3-libnvinfer, python3-libnvinfer-lean, python3-libnvinfer-dispatch Note If you need Python for a non-default Python version, install the .whl files from the tar package instead.
ONNX Graph Surgeon (for samples or projects)	python3 -m pip install numpy onnx onnx-graphsurgeon

Note

When installing Python packages using the local repo method, you must manually install TensorRT’s Python dependencies with pip.

Installation Steps (Network Repo Method)

This method is for advanced users already familiar with TensorRT who want quick setup or automation (such as when using containers). New users should use the local repo method above.

Note

If you are using a CUDA container, the NVIDIA CUDA network repository is already set up. Skip Step 1.

Step 1: Set up CUDA network repository

Follow the CUDA Toolkit download page instructions:

1. Select Linux operating system

2. Select desired architecture

3. Select Ubuntu or Debian distribution

4. Select desired Ubuntu or Debian version

5. Select deb (network) installer type

6. Enter the provided commands into your terminal

Tip

You can omit the final apt-get install command if you do not require the entire CUDA Toolkit. When installing TensorRT, apt downloads required CUDA dependencies automatically.

Step 2: Install TensorRT packages

Choose the package type for your needs:

Table 4 Network Repo Package Options

Package Type	Installation Command
Lean Runtime Only	sudo apt-get install libnvinfer-lean10
Lean Runtime Python Package	sudo apt-get install python3-libnvinfer-lean
Dispatch Runtime Only	sudo apt-get install libnvinfer-dispatch10
Dispatch Runtime Python Package	sudo apt-get install python3-libnvinfer-dispatch
Windows Builder Resource Library	sudo apt-get install libnvinfer-win-builder-resource10
C++ Applications Runtime Only	sudo apt-get install tensorrt-libs
C++ Applications with Development	sudo apt-get install tensorrt-dev
C++ with Lean Runtime Development	sudo apt-get install libnvinfer-lean-dev
C++ with Dispatch Runtime Development	sudo apt-get install libnvinfer-dispatch-dev
Standard Runtime Python Package	python3 -m pip install numpy sudo apt-get install python3-libnvinfer
Additional Python Modules (onnx-graphsurgeon)	Use pip to install additional modules. Refer to onnx-graphsurgeon on PyPI.

Step 3 (Optional): Install specific CUDA version

By default, Ubuntu installs TensorRT for the latest CUDA version when using the CUDA network repository. To install for a specific CUDA version and prevent automatic updates:

version="10.x.x.x-1+cudax.x"
sudo apt-get install \
libnvinfer-bin=${version} \
libnvinfer-dev=${version} \
libnvinfer-dispatch-dev=${version} \
libnvinfer-dispatch10=${version} \
libnvinfer-headers-dev=${version} \
libnvinfer-headers-plugin-dev=${version} \
libnvinfer-headers-python-plugin-dev=${version} \
libnvinfer-lean-dev=${version} \
libnvinfer-lean10=${version} \
libnvinfer-plugin-dev=${version} \
libnvinfer-plugin10=${version} \
libnvinfer-vc-plugin-dev=${version} \
libnvinfer-vc-plugin10=${version} \
libnvinfer-win-builder-resource10=${version} \
libnvinfer10=${version} \
libnvonnxparsers-dev=${version} \
libnvonnxparsers10=${version} \
python3-libnvinfer-dev=${version} \
python3-libnvinfer-dispatch=${version} \
python3-libnvinfer-lean=${version} \
python3-libnvinfer=${version} \
tensorrt-dev=${version} \
tensorrt-libs=${version} \
tensorrt=${version}

sudo apt-mark hold \
libnvinfer-bin \
libnvinfer-dev \
libnvinfer-dispatch-dev \
libnvinfer-dispatch10 \
libnvinfer-headers-dev \
libnvinfer-headers-plugin-dev \
libnvinfer-headers-python-plugin-dev \
libnvinfer-lean-dev \
libnvinfer-lean10 \
libnvinfer-plugin-dev \
libnvinfer-plugin10 \
libnvinfer-vc-plugin-dev \
libnvinfer-vc-plugin10 \
libnvinfer-win-builder-resource10 \
libnvinfer10 \
libnvonnxparsers-dev \
libnvonnxparsers10 \
python3-libnvinfer-dev \
python3-libnvinfer-dispatch \
python3-libnvinfer-lean \
python3-libnvinfer \
tensorrt-dev \
tensorrt-libs \
tensorrt

To upgrade to latest version, unhold the packages:

sudo apt-mark unhold \
libnvinfer-bin \
libnvinfer-dev \
libnvinfer-dispatch-dev \
libnvinfer-dispatch10 \
libnvinfer-headers-dev \
libnvinfer-headers-plugin-dev \
libnvinfer-headers-python-plugin-dev \
libnvinfer-lean-dev \
libnvinfer-lean10 \
libnvinfer-plugin-dev \
libnvinfer-plugin10 \
libnvinfer-vc-plugin-dev \
libnvinfer-vc-plugin10 \
libnvinfer-win-builder-resource10 \
libnvinfer10 \
libnvonnxparsers-dev \
libnvonnxparsers10 \
python3-libnvinfer-dev \
python3-libnvinfer-dispatch \
python3-libnvinfer-lean \
python3-libnvinfer \
tensorrt-dev \
tensorrt-libs \
tensorrt

Verification

These verification steps apply to both Local Repo and Network Repo installation methods.

Package Verification:

Verify that TensorRT packages are installed:

Table 5 Package Verification Commands

Package Type	Command	Expected Output
Full TensorRT Release	dpkg-query -W tensorrt	tensorrt 10.15.1.x-1+cuda13.1
Lean/Dispatch Runtime	dpkg-query -W "*nvinfer*"	Lists all installed libnvinfer* files

C++ Verification:

Compile and run a sample, such as sampleOnnxMNIST. Samples and sample data are only available from GitHub. The instructions to prepare the sample data can be found within the samples README.md. To build all the samples, use the following commands:

$ cd <cloned_tensorrt_dir>
$ mkdir build && cd build
$ cmake .. \
   -DTRT_LIB_DIR=$TRT_LIBPATH \
   -DTRT_OUT_DIR=`pwd`/out \
   -DBUILD_SAMPLES=ON \
   -DBUILD_PARSERS=OFF \
   -DBUILD_PLUGINS=OFF
$ cmake --build . --parallel 4
$ ./build/out/sample_onnx_mnist

For detailed information about the samples, refer to TensorRT Sample Support Guide.

Python Verification:

import tensorrt as trt
print(trt.__version__)
assert trt.Builder(trt.Logger())

Troubleshooting

These troubleshooting steps apply to both Local Repo and Network Repo installation methods.

Issue: E: Unable to locate package tensorrt

• Solution: Ensure you completed the repository setup steps correctly. Check that the repository is enabled:

  apt-cache policy tensorrt
  

Issue: Dependency conflicts with existing CUDA installation

• Solution: Ensure CUDA Toolkit was installed using Debian packages (not tar file or runfile).

Issue: Samples fail to compile

• Solution: Install build essentials:

  sudo apt-get install build-essential cmake
  

Method 3: RPM Package Installation

Recommended for: System-wide installation, C++ and Python development, RHEL/CentOS/Fedora users

Advantages:

✓ Automatic dependency installation ✓ Integrates with system package manager ✓ Includes C++ headers and samples ✓ Easy updates with dnf

Limitations:

✗ Requires sudo or root privileges ✗ Fixed installation location (/usr) ✗ Only one minor version of TensorRT can be installed at a time ✗ Linux only (RHEL/CentOS/Fedora)

Platform Support

Supported Operating Systems:

• RHEL 8, 9 (x86-64)

• Rocky Linux 8, 9 (x86-64)

Prerequisites:

• CUDA Toolkit installed using RPM packages

• sudo or root access

Tip

Choose your installation method:

• Local Repo (below): For new users or complete developer installation

• Network Repo (see Network Repo Method): For advanced users, containers, or automation

Installation Steps (Local Repo Method)

Note

• Before issuing commands, replace rhelx, 10.x.x, and cuda-x.x with your specific OS, TensorRT, and CUDA versions.

• When installing Python packages using this method, you must manually install dependencies with pip.

Step 1: Download the TensorRT RPM repository package

From the TensorRT download page, download the RPM repository package for your CUDA version and OS.

Example filename: nv-tensorrt-local-repo-rhel8-10.15.1-cuda-13.1-1.0-1.x86_64.rpm

Step 2: Install the repository package

os="rhelx"
tag="10.x.x-cuda-x.x"
sudo rpm -Uvh nv-tensorrt-local-repo-${os}-${tag}-1.0-1.x86_64.rpm
sudo dnf clean expire-cache

Step 3: Install TensorRT packages

Choose the package type that matches your needs:

Table 6 Package Installation Options

Package Type	Installation Command
Full C++ and Python Runtime	sudo dnf install tensorrt Installs all TensorRT components including runtime, development headers, and Python bindings.
Lean Runtime Only	sudo dnf install libnvinfer-lean10 sudo dnf install libnvinfer-vc-plugin10
Lean Runtime Python Package	sudo dnf install python3-libnvinfer-lean
Dispatch Runtime Only	sudo dnf install libnvinfer-dispatch10 sudo dnf install libnvinfer-vc-plugin10
Dispatch Runtime Python Package	sudo dnf install python3-libnvinfer-dispatch
Windows Builder Resource Library (for cross-platform support)	sudo dnf install libnvinfer-win-builder-resource10
All TensorRT Python Packages	python3 -m pip install numpy sudo dnf install python3-libnvinfer-devel This installs: python3-libnvinfer, python3-libnvinfer-lean, python3-libnvinfer-dispatch
ONNX Graph Surgeon (for samples or projects)	python3 -m pip install numpy onnx onnx-graphsurgeon

TensorRT Python bindings are only installed for Python 3.12 due to package dependencies. If your default python3 is not version 3.12:

• Use update-alternatives to switch to Python 3.12 by default

• Invoke Python using python3.12

• For non-default Python versions, install .whl files directly from the tar package

Note

When installing Python packages using the local repo method, you must manually install TensorRT’s Python dependencies with pip.

Installation Steps (Network Repo Method)

This method is for advanced users already familiar with TensorRT who want quick setup or automation (such as when using containers). New users should use the local repo method above.

Note

If you are using a CUDA container, the NVIDIA CUDA network repository is already set up. Skip Step 1.

Step 1: Set up CUDA network repository

Follow the CUDA Toolkit download page instructions:

1. Select Linux operating system

2. Select desired architecture

3. Select RHEL or Rocky distribution

4. Select desired RHEL or Rocky version

5. Select rpm (network) installer type

6. Enter the provided commands into your terminal

Tip

You can omit the final dnf install command if you do not require the entire CUDA Toolkit. When installing TensorRT, dnf downloads required CUDA dependencies automatically.

Step 2: Install TensorRT packages

Choose the package type for your needs:

Table 7 Network Repo Package Options

Package Type	Installation Command
Lean Runtime Only	sudo dnf install libnvinfer-lean10
Lean Runtime Python Package	sudo dnf install python3-libnvinfer-lean
Dispatch Runtime Only	sudo dnf install libnvinfer-dispatch10
Dispatch Runtime Python Package	sudo dnf install python3-libnvinfer-dispatch
Windows Builder Resource Library	sudo dnf install libnvinfer-win-builder-resource10
C++ Applications Runtime Only	sudo dnf install tensorrt-libs
C++ Applications with Development	sudo dnf install tensorrt-devel
C++ with Lean Runtime Development	sudo dnf install libnvinfer-lean-devel
C++ with Dispatch Runtime Development	sudo dnf install libnvinfer-dispatch-devel
Standard Runtime Python Package	python3 -m pip install numpy sudo dnf install python3-libnvinfer
Additional Python Modules (onnx-graphsurgeon)	Use pip to install additional modules. Refer to onnx-graphsurgeon on PyPI.

Step 3 (Optional): Install specific CUDA version

By default, RHEL installs TensorRT for the latest CUDA version when using the CUDA network repository. To install TensorRT for a specific CUDA version and prevent automatic updates:

version="10.x.x.x-1.cudax.x"
sudo dnf install \
libnvinfer-bin-${version} \
libnvinfer-devel-${version} \
libnvinfer-dispatch-devel-${version} \
libnvinfer-dispatch10-${version} \
libnvinfer-headers-devel-${version} \
libnvinfer-headers-plugin-devel-${version} \
libnvinfer-headers-python-plugin-devel-${version} \
libnvinfer-lean-devel-${version} \
libnvinfer-lean10-${version} \
libnvinfer-plugin-devel-${version} \
libnvinfer-plugin10-${version} \
libnvinfer-vc-plugin-devel-${version} \
libnvinfer-vc-plugin10-${version} \
libnvinfer-win-builder-resource10-${version} \
libnvinfer10-${version} \
libnvonnxparsers-devel-${version} \
libnvonnxparsers10-${version} \
python3-libnvinfer-${version} \
python3-libnvinfer-devel-${version} \
python3-libnvinfer-dispatch-${version} \
python3-libnvinfer-lean-${version} \
tensorrt-${version} \
tensorrt-devel-${version} \
tensorrt-libs-${version}

sudo dnf install dnf-plugin-versionlock
sudo dnf versionlock \
libnvinfer-bin \
libnvinfer-devel \
libnvinfer-dispatch-devel \
libnvinfer-dispatch10 \
libnvinfer-headers-devel \
libnvinfer-headers-plugin-devel \
libnvinfer-headers-python-plugin-devel \
libnvinfer-lean-devel \
libnvinfer-lean10 \
libnvinfer-plugin-devel \
libnvinfer-plugin10 \
libnvinfer-vc-plugin-devel \
libnvinfer-vc-plugin10 \
libnvinfer-win-builder-resource10 \
libnvinfer10 \
libnvonnxparsers-devel \
libnvonnxparsers10 \
python3-libnvinfer \
python3-libnvinfer-devel \
python3-libnvinfer-dispatch \
python3-libnvinfer-lean \
tensorrt \
tensorrt-devel \
tensorrt-libs

To upgrade to latest version, unlock the packages:

sudo dnf versionlock delete \
libnvinfer-bin \
libnvinfer-devel \
libnvinfer-dispatch-devel \
libnvinfer-dispatch10 \
libnvinfer-headers-devel \
libnvinfer-headers-plugin-devel \
libnvinfer-headers-python-plugin-devel \
libnvinfer-lean-devel \
libnvinfer-lean10 \
libnvinfer-plugin-devel \
libnvinfer-plugin10 \
libnvinfer-vc-plugin-devel \
libnvinfer-vc-plugin10 \
libnvinfer-win-builder-resource10 \
libnvinfer10 \
libnvonnxparsers-devel \
libnvonnxparsers10 \
python3-libnvinfer \
python3-libnvinfer-devel \
python3-libnvinfer-dispatch \
python3-libnvinfer-lean \
tensorrt \
tensorrt-devel \
tensorrt-libs

Verification

These verification steps apply to both Local Repo and Network Repo installation methods.

Package Verification:

Verify that TensorRT packages are installed:

Table 8 Package Verification Commands

Package Type	Command	Expected Output
Full TensorRT Release	rpm -q tensorrt	tensorrt-10.15.1.x-1.cuda-13.1.x86_64
Lean/Dispatch Runtime	rpm -qa | grep nvinfer	Lists all installed libnvinfer* files

C++ Verification:

Compile and run a sample, such as sampleOnnxMNIST. Samples and sample data are only available from GitHub. The instructions to prepare the sample data can be found within the samples README.md. To build all the samples, use the following commands:

$ cd <cloned_tensorrt_dir>
$ mkdir build && cd build
$ cmake .. \
   -DTRT_LIB_DIR=$TRT_LIBPATH \
   -DTRT_OUT_DIR=`pwd`/out \
   -DBUILD_SAMPLES=ON \
   -DBUILD_PARSERS=OFF \
   -DBUILD_PLUGINS=OFF
$ cmake --build . --parallel 4
$ ./build/out/sample_onnx_mnist

For detailed information about the samples, refer to TensorRT Sample Support Guide.

Python Verification:

import tensorrt as trt
print(trt.__version__)
assert trt.Builder(trt.Logger())

Troubleshooting

These troubleshooting steps apply to both Local Repo and Network Repo installation methods.

Issue: No package tensorrt available

• Solution: Ensure you completed the repository setup steps correctly. Check that the repository is enabled:

  dnf repolist
  

Issue: Dependency conflicts with existing CUDA installation

• Solution: Ensure CUDA Toolkit was installed using RPM packages (not tar file or runfile).

Issue: Samples fail to compile

• Solution: Install development tools:

  sudo dnf groupinstall "Development Tools"
  

Method 4: Tar File Installation

Recommended for: Multiple TensorRT versions, custom installation paths, C++ and Python development on Linux

Advantages:

✓ High flexibility in installation location ✓ No root privileges needed for installation ✓ Multiple versions can coexist ✓ Includes C++ headers and samples ✓ Complete control over environment

Limitations:

✗ Requires manual dependency management ✗ Manual LD_LIBRARY_PATH configuration ✗ No automatic updates ✗ More complex setup than other methods

Platform Support

Supported Operating Systems:

• Linux x86-64: Ubuntu 22.04+, RHEL 8+, Debian 12+

• Linux ARM SBSA: Ubuntu 22.04+, Debian 12+

Prerequisites:

• CUDA Toolkit installed using Tar file

• sudo or root access

Installation Steps

Step 1: Download the TensorRT tar file

From the TensorRT download page, download the tar file that matches the CPU architecture and CUDA version you are using.

Step 2: Choose installation directory

Choose where you want to install TensorRT. The tar file will install everything into a subdirectory called TensorRT-10.x.x.x, where 10.x.x.x is your TensorRT version.

Step 3: Extract the tar file

version="10.x.x.x"
arch=$(uname -m)
cuda="cuda-x.x"
tar -xzvf TensorRT-${version}.Linux.${arch}-gnu.${cuda}.tar.gz

Where 10.x.x.x is your TensorRT version and cuda-x.x is CUDA version.

Step 4: Set environment variables

Add the absolute path to the TensorRT lib directory to the environment variable LD_LIBRARY_PATH:

export LD_LIBRARY_PATH=<TensorRT-${version}/lib>:$LD_LIBRARY_PATH

For permanent configuration, add this line to ~/.bashrc or ~/.profile:

echo 'export LD_LIBRARY_PATH=<TensorRT-${version}/lib>:$LD_LIBRARY_PATH' >> ~/.bashrc
source ~/.bashrc

Step 5 (Optional): Install Python wheels

Install the Python TensorRT wheel file. Replace cp3x with the desired Python version (such as cp310 for Python 3.10):

cd TensorRT-${version}/python
python3 -m pip install tensorrt-*-cp3x-none-linux_x86_64.whl

Optionally, install the TensorRT lean and dispatch runtime wheel files:

python3 -m pip install tensorrt_lean-*-cp3x-none-linux_x86_64.whl
python3 -m pip install tensorrt_dispatch-*-cp3x-none-linux_x86_64.whl

Verification

Ensure that the installed files are located in the correct directories. For example, run the tree -d command to check whether all supported installed files are in place in the bin, lib, include, and other directories.

C++ Verification:

Compile and run a sample, such as sampleOnnxMNIST. Samples and sample data are only available from GitHub. The instructions to prepare the sample data can be found within the samples README.md. To build all the samples, use the following commands:

$ cd <cloned_tensorrt_dir>
$ mkdir build && cd build
$ cmake .. \
   -DTRT_LIB_DIR=$TRT_LIBPATH \
   -DTRT_OUT_DIR=`pwd`/out \
   -DBUILD_SAMPLES=ON \
   -DBUILD_PARSERS=OFF \
   -DBUILD_PLUGINS=OFF
$ cmake --build . --parallel 4
$ ./build/out/sample_onnx_mnist

For detailed information about the samples, refer to TensorRT Sample Support Guide.

Python Verification:

import tensorrt as trt
print(trt.__version__)
assert trt.Builder(trt.Logger())

Troubleshooting

Issue: error while loading shared libraries: libnvinfer.so.10

• Solution: Ensure LD_LIBRARY_PATH is set correctly. Check:

  echo $LD_LIBRARY_PATH
  

  It should include $TENSORRT_INSTALL_DIR/lib.

Issue: Samples fail to compile

• Solution: Install build essentials and CUDA development headers:

  sudo apt-get install build-essential cmake
  

Issue: Wrong Python wheel version

• Solution: Check your Python version:

  python3 --version
  

  Download the matching wheel (cp38 for Python 3.8, cp39 for Python 3.9, etc.).

Method 5: Zip File Installation (Windows)

Recommended for: C++ and Python development on Windows, custom installation paths

Advantages:

✓ High flexibility in installation location ✓ No administrator privileges needed for installation ✓ Multiple versions can coexist ✓ Includes C++ headers and samples

Limitations:

✗ Requires manual dependency management ✗ Manual PATH configuration ✗ No automatic updates ✗ More complex setup than pip

Platform Support

Supported Operating Systems:

• Windows 10 (x64)

• Windows 11 (x64)

• Windows Server 2022 (x64)

Prerequisites:

• CUDA Toolkit installed using Zip file

• sudo or root access

Installation Steps

Step 1: Download the TensorRT zip file

From the TensorRT download page, download the TensorRT zip file for Windows.

Step 2: Choose installation directory

Choose where you want to install TensorRT. This zip file will install everything into a subdirectory called TensorRT-10.x.x.x. This new subdirectory will be called <installpath> in the steps below.

Step 3: Extract the zip file

Unzip the TensorRT-10.x.x.x.Windows.win10.cuda-x.x.zip file to the location you chose.

Step 4: Set environment variables

Add the TensorRT library files to your system PATH by adding <installpath>\bin to your system PATH environment variable. Follow these steps:

1. Press the Windows key and search for environment variables. You should then be able to click Edit the System Environment Variables.

2. Click Environment Variables… at the bottom of the window.

3. Under System variables, select Path and click Edit….

4. Click either New or Browse to add a new item that contains <installpath>\bin.

5. Continue to click OK until all the newly opened windows are closed.

Step 5 (Optional): Install Python wheels

Install one of the TensorRT Python wheel files from <installpath>\python. Replace cp3x with the desired Python version (such as cp310 for Python 3.10):

python.exe -m pip install tensorrt-*-cp3x-none-win_amd64.whl

Optionally, install the TensorRT lean and dispatch runtime wheel files:

python.exe -m pip install tensorrt_lean-*-cp3x-none-win_amd64.whl
python.exe -m pip install tensorrt_dispatch-*-cp3x-none-win_amd64.whl

Verification

C++ Verification:

Samples and sample data are only available from GitHub. Refer to the TensorRT Sample Support Guide for how to obtain the samples from GitHub and build the samples using CMake. The instructions to prepare the sample data can be found within the samples README.md.

If you want to use TensorRT in your project, ensure that the following is present in your Visual Studio Solution project properties:

1. <installpath>\bin has been added to your PATH variable and is present under VC++ Directories > Executable Directories.

2. <installpath>\include is present under C/C++ > General > Additional Directories.

3. nvinfer_10.lib and any other LIB files your project requires are present under Linker > Input > Additional Dependencies.

Note

You should install Visual Studio 2022 or later to build the included samples. The community edition is sufficient to build the TensorRT samples.

Python Verification:

import tensorrt as trt
print(trt.__version__)
assert trt.Builder(trt.Logger())

Troubleshooting

Issue: DLL load failed: The specified module could not be found

• Solution: Ensure PATH includes <installpath>\lib and that CUDA libraries are in your system PATH.

Issue: Visual Studio cannot find TensorRT headers

• Solution: Add <installpath>\include to your project’s Additional Include Directories in Visual Studio project properties.

Issue: Linker errors when building samples

• Solution: Add <installpath>\lib to your project’s Additional Library Directories in Visual Studio project properties.

Method 6: Container Installation

Recommended for: Isolated development environments, reproducible builds, CI/CD pipelines, cloud deployments

Advantages:

✓ All dependencies pre-packaged ✓ Consistent environment across systems ✓ Easy to deploy and share ✓ No local installation conflicts ✓ Includes development tools and samples

Limitations:

✗ Requires Docker or compatible container runtime ✗ Requires NVIDIA Container Toolkit for GPU access ✗ Larger download size (multi-GB) ✗ Additional container overhead

Platform Support

Supported Platforms:

• Linux x86-64

• Linux ARM64 (NVIDIA Jetson)

Prerequisites:

• Docker (version 19.03+) or Podman installed

• NVIDIA Container Toolkit installed and configured

• NVIDIA GPU with appropriate drivers

Installation Steps

Step 1: Install NVIDIA Container Toolkit (if not already installed)

Follow the instructions on the NVIDIA Container Toolkit Installation Guide.

Quick installation for Ubuntu/Debian:

distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add -
curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | \
   sudo tee /etc/apt/sources.list.d/nvidia-docker.list
sudo apt-get update
sudo apt-get install -y nvidia-container-toolkit
sudo systemctl restart docker

Step 2: Pull the TensorRT NGC container

Find the latest TensorRT container on NVIDIA NGC Catalog.

docker pull nvcr.io/nvidia/tensorrt:<container-tag>

Step 3: Run the container

docker run --gpus all -it --rm \
   -v $PWD:/workspace \
   nvcr.io/nvidia/tensorrt:<container-tag>

This command:

• --gpus all: Enables GPU access

• -it: Interactive terminal

• --rm: Removes container on exit

• -v $PWD:/workspace: Mounts current directory to /workspace in container

Optional: Run with specific GPU(s):

docker run --gpus '"device=0,1"' -it --rm nvcr.io/nvidia/tensorrt:<container-tag>

Verification

Inside the container, verify TensorRT installation:

Check TensorRT version:

trtexec --version

Python verification:

import tensorrt as trt
print(f"TensorRT version: {trt.__version__}")

Run a sample:

cd /workspace/tensorrt/oss
mkdir build && cd build
cmake .. -DBUILD_PARSERS=OFF -DBUILD_PLUGINS=OFF -DBUILD_SAMPLES=ON
make -j8

./sample_onnx_mnist

Troubleshooting

For detailed information about the container, refer to the NVIDIA TensorRT Container Release Notes.

Issue: docker: Error response from daemon: could not select device driver "" with capabilities: [[gpu]]

• Solution: NVIDIA Container Toolkit is not installed or not configured correctly. Restart Docker daemon after installation:

  sudo systemctl restart docker
  

Issue: Container cannot access GPU

• Solution: Verify NVIDIA drivers are installed and working:

  nvidia-smi
  

  Ensure NVIDIA Container Toolkit runtime is set as default:

  sudo nvidia-ctk runtime configure --runtime=docker
  sudo systemctl restart docker
  

Issue: Permission denied when mounting volumes

• Solution: Add your user to the docker group:

  sudo usermod -aG docker $USER
  newgrp docker
  

Alternative Installation Methods

Aside from installing TensorRT from the standard product packages described above, TensorRT is also integrated into other NVIDIA platforms and tools. These alternative methods provide convenient ways to access TensorRT without manual installation.

NVIDIA NIM

For developing AI-powered enterprise applications and deploying AI models in production. NVIDIA NIM offers optimized inference microservices for deploying AI models at scale.

For more information, refer to the NVIDIA NIM technical blog post.

NVIDIA JetPack

For NVIDIA Jetson platforms, JetPack bundles all Jetson platform software, including TensorRT. Use it to flash your Jetson Developer Kit with the latest OS image, install NVIDIA SDKs, and jumpstart your development environment.

Key Features:

• Complete software stack for Jetson platforms

• TensorRT pre-installed and optimized

• Integrated development environment

• Automatic dependency management

Resources:

• JetPack documentation - Installation and usage instructions

• JetPack downloads - Download page

NVIDIA DriveOS

For automotive applications, TensorRT is included in NVIDIA DriveOS Linux Standard. The safety proxy runtime is not installed by default in the NVIDIA DriveOS Linux SDK.

Installation:

For step-by-step instructions on installing TensorRT with NVIDIA SDK Manager, refer to the NVIDIA DRIVE Platform Installation section in the DriveOS Installation Guide.

Nsight Deep Learning Designer

For developers who primarily convert ONNX models into TensorRT engines, Nsight Deep Learning Designer provides a GUI-based tool that can be used without a separate installation of TensorRT.

Key Features:

• No installation required: Automatically downloads necessary TensorRT components (including CUDA, cuDNN, and cuBLAS) on-demand

• GUI interface: Visual model editing and optimization workflow

• All-in-one solution: Complete environment for model conversion

• Primary use case: ONNX model conversion and optimization

When to Use:

• You prefer a GUI-based workflow over command-line tools

• You primarily work with ONNX models

• You want to avoid manual installation and dependency management

• You need a quick way to convert and optimize models

Download: Nsight Deep Learning Designer

Cross-Compilation for AArch64

If you intend to cross-compile TensorRT applications for AArch64 targets (e.g., NVIDIA Jetson platforms) from an x86 host, follow these steps to prepare your machine for cross-compilation and build TensorRT samples.

Step 1: Set up the network repository and TensorRT for the host

Start with the Network Repo Installation Method section to set up the network repository and install TensorRT on your x86 host.

Step 2: Install cross-compilation tools

sudo apt-get install g++-aarch64-linux-gnu

Step 3: Download AArch64 TensorRT package

From the TensorRT download page, download the tar file for AArch64.

Example: TensorRT-10.15.1.x.Linux.aarch64-gnu.cuda-13.1.tar.gz

Step 4: Extract and configure

tar -xzvf TensorRT-10.x.x.x.Linux.aarch64-gnu.cuda-x.x.tar.gz -C $HOME/cross-compile

Step 5: Get the TensorRT Samples via GitHub

$ git clone git@github.com:NVIDIA/TensorRT.git $HOME/TensorRT

Step 6: Build samples with cross-compilation

$ cd $HOME/TensorRT
$ mkdir build && cd build
$ cmake .. \
   -DTRT_LIB_DIR=$HOME/cross-compile/TensorRT-10.x.x/lib \
   -DTRT_OUT_DIR=`pwd`/out \
   -DBUILD_SAMPLES=ON \
   -DBUILD_PARSERS=OFF \
   -DBUILD_PLUGINS=OFF
$ cmake –build . –parallel 4

For detailed cross-compilation instructions and additional information, refer to the TensorRT samples README and the Cross Compiling Samples documentation.

Next Steps

After successfully installing TensorRT, you can:

• Explore the Quick Start Guide to build and deploy your first optimized inference engine.

• Dive into the Sample Support Guide for practical examples and advanced features.

• Refer to the C++ API Documentation and the Python API Documentation for detailed information on C++ and Python APIs.

• Review the Best Practices to optimize your inference performance.