Method 3: RPM Package Installation#

Recommended for: System-wide installation, C++ and Python development, RHEL and Rocky Linux users

Advantages:

✓ Automatic dependency installation
✓ Integrates with system package manager
✓ Includes C++ headers
✓ Updates with dnf

Limitations:

✗ Requires sudo or root privileges
✗ Fixed installation location (/usr)
✗ Only one minor version of TensorRT can be installed at a time
✗ Linux only (RHEL and Rocky Linux)

Platform Support#

Supported Operating Systems:

  • RHEL 8, 9, 10 (x86-64)

  • Rocky Linux 8, 9, 10 (x86-64)

Prerequisites:

  • CUDA Toolkit installed using RPM packages

  • sudo or root access

Installation Steps#

Tip

Choose Local Repo for a complete developer install (new users). Choose Network Repo for containers, automation, or when the CUDA network repository is already configured.

Step 1: Download the TensorRT RPM repository package

From the TensorRT download page, download the RPM repository package for your CUDA version and OS.

Example filename: nv-tensorrt-local-repo-rhel8-11.5.0-cuda-13.5-1.0-1.x86_64.rpm

Note

  • Before issuing the following commands, replace rhelx, 11.x.x, and cuda-x.x with your specific OS, TensorRT, and CUDA versions.

  • When installing Python packages using this method, you must manually install dependencies with pip.

Step 2: Install the repository package

os="rhelx"
tag="11.x.x-cuda-x.x"
sudo rpm -Uvh nv-tensorrt-local-repo-${os}-${tag}-1.0-1.x86_64.rpm
sudo dnf clean expire-cache

Step 3: Install TensorRT packages

Choose the package type that matches your needs:

Table 5 Package Installation Options#

Package Type

Installation Command

Full C++ and Python Runtime

sudo dnf install tensorrt

Installs all TensorRT components including runtime, development headers, and Python bindings.

Lean Runtime Only

sudo dnf install libnvinfer-lean11
sudo dnf install libnvinfer-vc-plugin11

Lean Runtime Python Package

sudo dnf install python3-libnvinfer-lean

Dispatch Runtime Only

sudo dnf install libnvinfer-dispatch11
sudo dnf install libnvinfer-vc-plugin11

Dispatch Runtime Python Package

sudo dnf install python3-libnvinfer-dispatch

Windows Builder Resource Library (for cross-platform support)

sudo dnf install libnvinfer-win-builder-resource11

All TensorRT Python Packages

sudo dnf install python3-numpy
sudo dnf install python3-libnvinfer-devel

This installs: python3-libnvinfer, python3-libnvinfer-lean, python3-libnvinfer-dispatch

ONNX Graph Surgeon (for samples or projects)

onnx-graphsurgeon is published only on PyPI, so it must come from pip. Create the virtual environment with --system-site-packages so that it can still import the TensorRT bindings that dnf installed into the system Python:

python3 -m venv --system-site-packages trt-env
source trt-env/bin/activate
python3 -m pip install numpy onnx onnx-graphsurgeon

TensorRT Python bindings from RPM packages target the platform-default Python on RHEL and Rocky Linux (3.12 on RHEL 8/9/10 per the TensorRT Support Matrix). If your default python3 is not version 3.12:

  • Use update-alternatives to switch to Python 3.12 by default

  • Invoke Python using python3.12

  • For non-default Python versions, install .whl files directly from the tar package

Note

When installing Python packages using the local repo method, you must install the TensorRT Python dependencies yourself.

Note

Prefer the distribution package for a dependency whenever one exists, as shown above. Recent distributions mark the system Python as externally managed (PEP 668), and on those releases python3 -m pip install into the system Python fails with error: externally-managed-environment. Use a virtual environment for packages that PyPI alone provides.

This method is for advanced users already familiar with TensorRT who want quick setup or automation (such as when using containers). New users should use the local repo method above.

Note

Skip Step 1 if the NVIDIA CUDA network repository is already set up on your system. For example, when using an NVIDIA CUDA container (where it is set up by default), or if you previously configured it manually.

Step 1: Set up CUDA network repository

Follow the CUDA Toolkit download page instructions:

  1. Select Linux operating system

  2. Select desired architecture

  3. Select RHEL or Rocky distribution

  4. Select desired RHEL or Rocky version

  5. Select rpm (network) installer type

  6. Enter the provided commands into your terminal

Tip

You can omit the final dnf install command if you do not require the entire CUDA Toolkit. When installing TensorRT, dnf downloads required CUDA dependencies automatically.

Step 2: Install TensorRT packages

Choose the package type for your needs:

Table 6 Network Repo Package Options#

Package Type

Installation Command

Full C++ and Python Runtime

sudo dnf install tensorrt

Lean Runtime Only

sudo dnf install libnvinfer-lean11

Lean Runtime Python Package

sudo dnf install python3-libnvinfer-lean

Dispatch Runtime Only

sudo dnf install libnvinfer-dispatch11

Dispatch Runtime Python Package

sudo dnf install python3-libnvinfer-dispatch

Windows Builder Resource Library

sudo dnf install libnvinfer-win-builder-resource11

C++ Applications Runtime Only

sudo dnf install tensorrt-libs

C++ Applications with Development

sudo dnf install tensorrt-devel

C++ with Lean Runtime Development

sudo dnf install libnvinfer-lean-devel

C++ with Dispatch Runtime Development

sudo dnf install libnvinfer-dispatch-devel

Standard Runtime Python Package

sudo dnf install python3-numpy
sudo dnf install python3-libnvinfer

Additional Python Modules (onnx-graphsurgeon)

Use pip inside a --system-site-packages virtual environment to install additional modules, as shown in the Local Repo tab. Refer to onnx-graphsurgeon on PyPI.

Step 3 (Optional): Install specific CUDA version

By default, RHEL installs TensorRT for the latest CUDA version when using the CUDA network repository. To install TensorRT for a specific CUDA version and prevent automatic updates:

version="11.x.x.x-1.cudax.x"
sudo dnf install tensorrt-${version}

sudo dnf install dnf-plugin-versionlock
sudo dnf versionlock \
libnvinfer-bin \
libnvinfer-cmake-devel \
libnvinfer-devel \
libnvinfer-dispatch-devel \
libnvinfer-dispatch11 \
libnvinfer-headers-devel \
libnvinfer-headers-plugin-devel \
libnvinfer-headers-python-plugin-devel \
libnvinfer-lean-devel \
libnvinfer-lean11 \
libnvinfer-plugin-devel \
libnvinfer-plugin11 \
libnvinfer-safe-headers-devel \
libnvinfer-vc-plugin-devel \
libnvinfer-vc-plugin11 \
libnvinfer-win-builder-resource11 \
libnvinfer11 \
libnvonnxparsers-devel \
libnvonnxparsers11 \
python3-libnvinfer \
python3-libnvinfer-devel \
python3-libnvinfer-dispatch \
python3-libnvinfer-lean \
tensorrt \
tensorrt-devel \
tensorrt-libs

To upgrade to latest version, unlock the packages:

sudo dnf versionlock delete \
libnvinfer-bin \
libnvinfer-cmake-devel \
libnvinfer-devel \
libnvinfer-dispatch-devel \
libnvinfer-dispatch11 \
libnvinfer-headers-devel \
libnvinfer-headers-plugin-devel \
libnvinfer-headers-python-plugin-devel \
libnvinfer-lean-devel \
libnvinfer-lean11 \
libnvinfer-plugin-devel \
libnvinfer-plugin11 \
libnvinfer-safe-headers-devel \
libnvinfer-vc-plugin-devel \
libnvinfer-vc-plugin11 \
libnvinfer-win-builder-resource11 \
libnvinfer11 \
libnvonnxparsers-devel \
libnvonnxparsers11 \
python3-libnvinfer \
python3-libnvinfer-devel \
python3-libnvinfer-dispatch \
python3-libnvinfer-lean \
tensorrt \
tensorrt-devel \
tensorrt-libs

Verification#

For quick checks that apply to every install method, refer to Verifying Your Installation in the installation overview. The steps below are specific to RPM packages.

These verification steps apply to both Local Repo and Network Repo installation methods.

Package Verification:

Verify that TensorRT packages are installed:

Table 7 Package Verification Commands#

Package Type

Command

Expected Output

Full TensorRT Release

rpm -q tensorrt
tensorrt-11.5.0.x-1.cuda13.5.x86_64

Lean/Dispatch Runtime

rpm -qa | grep nvinfer

Lists the name and version of every installed libnvinfer* package, such as libnvinfer-lean11 and libnvinfer-dispatch11. To list the files a package installed instead, use rpm -ql <package>.

C++ Verification:

Verify the trtexec executable is working:

trtexec --help

If the shell reports trtexec: command not found, the RPM package installed the executable but its directory is not on your PATH. List the files the package installed and filter for it:

rpm -ql tensorrt | grep trtexec

Add the directory holding the executable to PATH, then run trtexec --help again:

export PATH=<directory from the previous command>:$PATH

Compile and run a sample, such as sampleOnnxMNIST. Samples and sample data are only available from GitHub. The instructions to prepare the sample data can be found within the samples README.md.

Clone the repository and initialize its submodules, which the sample build requires:

$ git clone -b main https://github.com/NVIDIA/TensorRT.git
$ git -C TensorRT submodule update --init --recursive

Note

The samples are built as C++20, so the host needs a C++20-capable compiler. RHEL and Rocky Linux 8 default to GCC 8.5, which is not, and the build fails while compiling the samples rather than at the configure step. On those releases, enable a gcc-toolset that provides GCC 11 or later before you configure:

$ sudo dnf install gcc-toolset-13
$ scl enable gcc-toolset-13 bash
$ export CXX=$(command -v g++)

The export is not redundant. Where the configure step passes -DCMAKE_PREFIX_PATH=/usr, CMake searches that prefix for a compiler before it searches PATH, so it selects /usr/bin/c++, the GCC 8.5 that the CUDA toolkit installs, even inside the toolset shell. Setting CXX takes priority over that search.

The supported Ubuntu, Debian, and RHEL and Rocky Linux 9 and later releases ship a C++20-capable compiler as the default.

Then build all the samples with the following commands:

$ cd <cloned_tensorrt_dir>
$ export TRT_LIBPATH=/usr/lib64
$ mkdir build && cd build
$ cmake .. \
   -DCMAKE_PREFIX_PATH=/usr \
   -DCMAKE_RUNTIME_OUTPUT_DIRECTORY=`pwd`/out \
   -DCMAKE_LIBRARY_OUTPUT_DIRECTORY=`pwd`/out \
   -DCMAKE_ARCHIVE_OUTPUT_DIRECTORY=`pwd`/out \
   -DBUILD_SAMPLES=ON \
   -DBUILD_PARSERS=OFF \
   -DBUILD_PLUGINS=OFF
$ cmake --build . --parallel 4
$ ./out/sample_onnx_mnist

CMAKE_PREFIX_PATH points at the root of the TensorRT installation, which is /usr for RPM packages. The build locates TensorRT through the CMake configuration files that libnvinfer-cmake-devel installs, so install that package before configuring. The three output directory settings collect the built binaries into build/out; without them, CMake writes each sample to its own subdirectory under build/samples.

For information about the samples, refer to TensorRT Sample Support Guide.

Python Verification:

import tensorrt as trt
print(trt.__version__)
assert trt.Builder(trt.Logger())

Troubleshooting#

For install-wide diagnostics (CUDA, drivers, samples, and runtime failures), refer to Troubleshooting. These steps apply to both Local Repo and Network Repo RPM installs.

Method-specific issues for RPM packages:

Issue: No package tensorrt available

  • Solution: Ensure you completed the repository setup steps correctly. Check that the repository is enabled:

    dnf repolist
    

Issue: Dependency conflicts with existing CUDA installation

  • Solution: Ensure CUDA Toolkit was installed using RPM packages (not tar file or runfile).

Issue: Samples fail to compile

  • Solution: Install development tools:

    sudo dnf groupinstall "Development Tools"