Method 3: RPM Package Installation#
Recommended for: System-wide installation, C++ and Python development, RHEL and Rocky Linux users
Advantages:
dnfLimitations:
sudo or root privileges/usr)Platform Support#
Supported Operating Systems:
RHEL 8, 9, 10 (x86-64)
Rocky Linux 8, 9, 10 (x86-64)
Prerequisites:
CUDA Toolkit installed using RPM packages
sudoor root access
Installation Steps#
Tip
Choose Local Repo for a complete developer install (new users). Choose Network Repo for containers, automation, or when the CUDA network repository is already configured.
Step 1: Download the TensorRT RPM repository package
From the TensorRT download page, download the RPM repository package for your CUDA version and OS.
Example filename: nv-tensorrt-local-repo-rhel8-11.5.0-cuda-13.5-1.0-1.x86_64.rpm
Note
Before issuing the following commands, replace
rhelx,11.x.x, andcuda-x.xwith your specific OS, TensorRT, and CUDA versions.When installing Python packages using this method, you must manually install dependencies with
pip.
Step 2: Install the repository package
os="rhelx"
tag="11.x.x-cuda-x.x"
sudo rpm -Uvh nv-tensorrt-local-repo-${os}-${tag}-1.0-1.x86_64.rpm
sudo dnf clean expire-cache
Step 3: Install TensorRT packages
Choose the package type that matches your needs:
Package Type |
Installation Command |
|---|---|
Full C++ and Python Runtime |
sudo dnf install tensorrt
Installs all TensorRT components including runtime, development headers, and Python bindings. |
Lean Runtime Only |
sudo dnf install libnvinfer-lean11
sudo dnf install libnvinfer-vc-plugin11
|
Lean Runtime Python Package |
sudo dnf install python3-libnvinfer-lean
|
Dispatch Runtime Only |
sudo dnf install libnvinfer-dispatch11
sudo dnf install libnvinfer-vc-plugin11
|
Dispatch Runtime Python Package |
sudo dnf install python3-libnvinfer-dispatch
|
Windows Builder Resource Library (for cross-platform support) |
sudo dnf install libnvinfer-win-builder-resource11
|
All TensorRT Python Packages |
sudo dnf install python3-numpy
sudo dnf install python3-libnvinfer-devel
This installs: |
ONNX Graph Surgeon (for samples or projects) |
python3 -m venv --system-site-packages trt-env
source trt-env/bin/activate
python3 -m pip install numpy onnx onnx-graphsurgeon
|
TensorRT Python bindings from RPM packages target the platform-default Python on RHEL and Rocky Linux (3.12 on RHEL 8/9/10 per the TensorRT Support Matrix). If your default python3 is not version 3.12:
Use
update-alternativesto switch to Python 3.12 by defaultInvoke Python using
python3.12For non-default Python versions, install
.whlfiles directly from the tar package
Note
When installing Python packages using the local repo method, you must install the TensorRT Python dependencies yourself.
Note
Prefer the distribution package for a dependency whenever one exists, as shown above. Recent distributions mark the system Python as externally managed (PEP 668), and on those releases python3 -m pip install into the system Python fails with error: externally-managed-environment. Use a virtual environment for packages that PyPI alone provides.
This method is for advanced users already familiar with TensorRT who want quick setup or automation (such as when using containers). New users should use the local repo method above.
Note
Skip Step 1 if the NVIDIA CUDA network repository is already set up on your system. For example, when using an NVIDIA CUDA container (where it is set up by default), or if you previously configured it manually.
Step 1: Set up CUDA network repository
Follow the CUDA Toolkit download page instructions:
Select Linux operating system
Select desired architecture
Select RHEL or Rocky distribution
Select desired RHEL or Rocky version
Select rpm (network) installer type
Enter the provided commands into your terminal
Tip
You can omit the final dnf install command if you do not require the entire CUDA Toolkit. When installing TensorRT, dnf downloads required CUDA dependencies automatically.
Step 2: Install TensorRT packages
Choose the package type for your needs:
Package Type |
Installation Command |
|---|---|
Full C++ and Python Runtime |
sudo dnf install tensorrt
|
Lean Runtime Only |
sudo dnf install libnvinfer-lean11
|
Lean Runtime Python Package |
sudo dnf install python3-libnvinfer-lean
|
Dispatch Runtime Only |
sudo dnf install libnvinfer-dispatch11
|
Dispatch Runtime Python Package |
sudo dnf install python3-libnvinfer-dispatch
|
Windows Builder Resource Library |
sudo dnf install libnvinfer-win-builder-resource11
|
C++ Applications Runtime Only |
sudo dnf install tensorrt-libs
|
C++ Applications with Development |
sudo dnf install tensorrt-devel
|
C++ with Lean Runtime Development |
sudo dnf install libnvinfer-lean-devel
|
C++ with Dispatch Runtime Development |
sudo dnf install libnvinfer-dispatch-devel
|
Standard Runtime Python Package |
sudo dnf install python3-numpy
sudo dnf install python3-libnvinfer
|
Additional Python Modules (onnx-graphsurgeon) |
Use |
Step 3 (Optional): Install specific CUDA version
By default, RHEL installs TensorRT for the latest CUDA version when using the CUDA network repository. To install TensorRT for a specific CUDA version and prevent automatic updates:
version="11.x.x.x-1.cudax.x"
sudo dnf install tensorrt-${version}
sudo dnf install dnf-plugin-versionlock
sudo dnf versionlock \
libnvinfer-bin \
libnvinfer-cmake-devel \
libnvinfer-devel \
libnvinfer-dispatch-devel \
libnvinfer-dispatch11 \
libnvinfer-headers-devel \
libnvinfer-headers-plugin-devel \
libnvinfer-headers-python-plugin-devel \
libnvinfer-lean-devel \
libnvinfer-lean11 \
libnvinfer-plugin-devel \
libnvinfer-plugin11 \
libnvinfer-safe-headers-devel \
libnvinfer-vc-plugin-devel \
libnvinfer-vc-plugin11 \
libnvinfer-win-builder-resource11 \
libnvinfer11 \
libnvonnxparsers-devel \
libnvonnxparsers11 \
python3-libnvinfer \
python3-libnvinfer-devel \
python3-libnvinfer-dispatch \
python3-libnvinfer-lean \
tensorrt \
tensorrt-devel \
tensorrt-libs
To upgrade to latest version, unlock the packages:
sudo dnf versionlock delete \
libnvinfer-bin \
libnvinfer-cmake-devel \
libnvinfer-devel \
libnvinfer-dispatch-devel \
libnvinfer-dispatch11 \
libnvinfer-headers-devel \
libnvinfer-headers-plugin-devel \
libnvinfer-headers-python-plugin-devel \
libnvinfer-lean-devel \
libnvinfer-lean11 \
libnvinfer-plugin-devel \
libnvinfer-plugin11 \
libnvinfer-safe-headers-devel \
libnvinfer-vc-plugin-devel \
libnvinfer-vc-plugin11 \
libnvinfer-win-builder-resource11 \
libnvinfer11 \
libnvonnxparsers-devel \
libnvonnxparsers11 \
python3-libnvinfer \
python3-libnvinfer-devel \
python3-libnvinfer-dispatch \
python3-libnvinfer-lean \
tensorrt \
tensorrt-devel \
tensorrt-libs
Verification#
For quick checks that apply to every install method, refer to Verifying Your Installation in the installation overview. The steps below are specific to RPM packages.
These verification steps apply to both Local Repo and Network Repo installation methods.
Package Verification:
Verify that TensorRT packages are installed:
Package Type |
Command |
Expected Output |
|---|---|---|
Full TensorRT Release |
rpm -q tensorrt
|
tensorrt-11.5.0.x-1.cuda13.5.x86_64
|
Lean/Dispatch Runtime |
rpm -qa | grep nvinfer
|
Lists the name and version of every installed |
C++ Verification:
Verify the trtexec executable is working:
trtexec --help
If the shell reports trtexec: command not found, the RPM package installed
the executable but its directory is not on your PATH. List the files the
package installed and filter for it:
rpm -ql tensorrt | grep trtexec
Add the directory holding the executable to PATH, then run
trtexec --help again:
export PATH=<directory from the previous command>:$PATH
Compile and run a sample, such as sampleOnnxMNIST. Samples and sample data are only available from GitHub. The instructions to prepare the sample data can be found within the samples README.md.
Clone the repository and initialize its submodules, which the sample build requires:
$ git clone -b main https://github.com/NVIDIA/TensorRT.git
$ git -C TensorRT submodule update --init --recursive
Note
The samples are built as C++20, so the host needs a C++20-capable compiler.
RHEL and Rocky Linux 8 default to GCC 8.5, which is not, and the build fails
while compiling the samples rather than at the configure step. On those
releases, enable a gcc-toolset that provides GCC 11 or later before you
configure:
$ sudo dnf install gcc-toolset-13
$ scl enable gcc-toolset-13 bash
$ export CXX=$(command -v g++)
The export is not redundant. Where the configure step passes
-DCMAKE_PREFIX_PATH=/usr, CMake searches that prefix for a compiler
before it searches PATH, so it selects /usr/bin/c++, the GCC 8.5
that the CUDA toolkit installs, even inside the toolset shell. Setting
CXX takes priority over that search.
The supported Ubuntu, Debian, and RHEL and Rocky Linux 9 and later releases ship a C++20-capable compiler as the default.
Then build all the samples with the following commands:
$ cd <cloned_tensorrt_dir>
$ export TRT_LIBPATH=/usr/lib64
$ mkdir build && cd build
$ cmake .. \
-DCMAKE_PREFIX_PATH=/usr \
-DCMAKE_RUNTIME_OUTPUT_DIRECTORY=`pwd`/out \
-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=`pwd`/out \
-DCMAKE_ARCHIVE_OUTPUT_DIRECTORY=`pwd`/out \
-DBUILD_SAMPLES=ON \
-DBUILD_PARSERS=OFF \
-DBUILD_PLUGINS=OFF
$ cmake --build . --parallel 4
$ ./out/sample_onnx_mnist
CMAKE_PREFIX_PATH points at the root of the TensorRT installation, which is
/usr for RPM packages. The build locates TensorRT through the CMake
configuration files that libnvinfer-cmake-devel installs, so install that
package before configuring. The three output directory settings collect the
built binaries into build/out; without them, CMake writes each sample to its
own subdirectory under build/samples.
For information about the samples, refer to TensorRT Sample Support Guide.
Python Verification:
import tensorrt as trt
print(trt.__version__)
assert trt.Builder(trt.Logger())
Troubleshooting#
For install-wide diagnostics (CUDA, drivers, samples, and runtime failures), refer to Troubleshooting. These steps apply to both Local Repo and Network Repo RPM installs.
Method-specific issues for RPM packages:
Issue: No package tensorrt available
Solution: Ensure you completed the repository setup steps correctly. Check that the repository is enabled:
dnf repolist
Issue: Dependency conflicts with existing CUDA installation
Solution: Ensure CUDA Toolkit was installed using RPM packages (not tar file or runfile).
Issue: Samples fail to compile
Solution: Install development tools:
sudo dnf groupinstall "Development Tools"