Method 2: Debian Package Installation#

Recommended for: System-wide installation, C++ and Python development, Ubuntu/Debian users

Advantages:

✓ Automatic dependency installation
✓ Integrates with system package manager
✓ Includes C++ headers
✓ Updates with apt

Limitations:

✗ Requires sudo or root privileges
✗ Fixed installation location (/usr)
✗ Only one minor version of TensorRT can be installed at a time
✗ Linux only (Ubuntu/Debian)

Platform Support#

Supported Operating Systems:

  • Ubuntu 22.04 (x86-64)

  • Ubuntu 24.04 (x86-64, ARM SBSA, JetPack)

  • Ubuntu 26.04 (x86-64, ARM SBSA)

  • Debian 12 (x86-64, ARM SBSA)

ARM SBSA refers to the Arm Server Base System Architecture platform.

Prerequisites:

  • CUDA Toolkit installed using Debian packages

  • sudo or root access

Installation Steps#

Tip

Choose Local Repo for a complete developer install (new users). Choose Network Repo for containers, automation, or when the CUDA network repository is already configured.

Step 1: Download the TensorRT Debian repository package

From the TensorRT download page, download the Debian repository package for your CUDA version and OS.

Example filename: nv-tensorrt-local-repo-ubuntu2404-11.5.0-cuda-13.5_1.0-1_amd64.deb

For the following steps, replace ubuntu2x04 with your Ubuntu version, 11.x.x with your TensorRT version, and cuda-x.x with your CUDA version. Replace amd64 with arm64 for ARM SBSA or JetPack.

Step 2: Install the repository package

sudo dpkg -i nv-tensorrt-local-repo-ubuntu2x04-11.x.x-cuda-x.x_1.0-1_amd64.deb

Step 3: Copy the keyring

sudo cp /var/nv-tensorrt-local-repo-ubuntu2x04-11.x.x-cuda-x.x/*-keyring.gpg /usr/share/keyrings/

Step 4: Update the package index

sudo apt-get update

Step 5: Install TensorRT packages

Choose the package type that matches your needs:

Table 2 Package Installation Options#

Package Type

Installation Command

Full C++ and Python Runtime

sudo apt-get install tensorrt

Installs all TensorRT components including runtime, development headers, and Python bindings.

Lean Runtime Only

sudo apt-get install libnvinfer-lean11
sudo apt-get install libnvinfer-vc-plugin11

Lean Runtime Python Package

sudo apt-get install python3-libnvinfer-lean

Dispatch Runtime Only

sudo apt-get install libnvinfer-dispatch11
sudo apt-get install libnvinfer-vc-plugin11

Dispatch Runtime Python Package

sudo apt-get install python3-libnvinfer-dispatch

DLA Compiler Package

sudo apt-get install libnvinfer-dla-compiler11

Windows Builder Resource Library (for cross-platform support)

sudo apt-get install libnvinfer-win-builder-resource11

All TensorRT Python Packages

sudo apt-get install python3-numpy
sudo apt-get install python3-libnvinfer-dev

This installs: python3-libnvinfer, python3-libnvinfer-lean, python3-libnvinfer-dispatch

Note

If you need Python for a non-default Python version, install the .whl files from the tar package instead.

ONNX Graph Surgeon (for samples or projects)

onnx-graphsurgeon is published only on PyPI, so it must come from pip. Create the virtual environment with --system-site-packages so that it can still import the TensorRT bindings that apt-get installed into the system Python:

# Debian and Ubuntu ship venv separately from the base Python.
sudo apt-get install -y python3-venv

python3 -m venv --system-site-packages trt-env
source trt-env/bin/activate
python3 -m pip install numpy onnx onnx-graphsurgeon

Note

When installing Python packages using the local repo method, you must install the TensorRT Python dependencies yourself.

Note

On Debian 12, Ubuntu 23.04, and later releases, the system Python is marked as externally managed (PEP 668), so python3 -m pip install into it fails with error: externally-managed-environment. Install dependencies from the distribution packages where one exists, as shown above, and use a virtual environment for packages that PyPI alone provides.

This method is for advanced users already familiar with TensorRT who want quick setup or automation (such as when using containers). New users should use the local repo method above.

Note

Skip Step 1 if the NVIDIA CUDA network repository is already set up on your system. For example, when using an NVIDIA CUDA container (where it is set up by default), or if you previously configured it manually.

Step 1: Set up CUDA network repository

Follow the CUDA Toolkit download page instructions:

  1. Select Linux operating system

  2. Select desired architecture

  3. Select Ubuntu or Debian distribution

  4. Select desired Ubuntu or Debian version

  5. Select deb (network) installer type

  6. Enter the provided commands into your terminal

Tip

You can omit the final apt-get install command if you do not require the entire CUDA Toolkit. When installing TensorRT, apt downloads required CUDA dependencies automatically.

Step 2: Install TensorRT packages

Choose the package type for your needs:

Table 3 Network Repo Package Options#

Package Type

Installation Command

Full C++ and Python Runtime

sudo apt-get install tensorrt

Lean Runtime Only

sudo apt-get install libnvinfer-lean11

Lean Runtime Python Package

sudo apt-get install python3-libnvinfer-lean

Dispatch Runtime Only

sudo apt-get install libnvinfer-dispatch11

Dispatch Runtime Python Package

sudo apt-get install python3-libnvinfer-dispatch

Windows Builder Resource Library

sudo apt-get install libnvinfer-win-builder-resource11

C++ Applications Runtime Only

sudo apt-get install tensorrt-libs

C++ Applications with Development

sudo apt-get install tensorrt-dev

C++ with Lean Runtime Development

sudo apt-get install libnvinfer-lean-dev

C++ with Dispatch Runtime Development

sudo apt-get install libnvinfer-dispatch-dev

DLA Development

sudo apt-get install libnvinfer-dla-compiler11

Standard Runtime Python Package

sudo apt-get install python3-numpy
sudo apt-get install python3-libnvinfer

Additional Python Modules (onnx-graphsurgeon)

Use pip inside a --system-site-packages virtual environment to install additional modules, as shown in the Local Repo tab. Refer to onnx-graphsurgeon on PyPI.

Step 3 (Optional): Install specific CUDA version

By default, Ubuntu installs TensorRT for the latest CUDA version and the latest TensorRT version available when using the CUDA network repository. To install for a specific CUDA version or specific TensorRT version and prevent automatic updates, use the following commands:

version="11.x.x.x-1+cudax.x"
sudo apt-get install \
libnvinfer-bin=${version} \
libnvinfer-cmake-dev=${version} \
libnvinfer-dev=${version} \
libnvinfer-dispatch-dev=${version} \
libnvinfer-dispatch11=${version} \
libnvinfer-headers-dev=${version} \
libnvinfer-headers-plugin-dev=${version} \
libnvinfer-headers-python-plugin-dev=${version} \
libnvinfer-lean-dev=${version} \
libnvinfer-lean11=${version} \
libnvinfer-plugin-dev=${version} \
libnvinfer-plugin11=${version} \
libnvinfer-safe-headers-dev=${version} \
libnvinfer-vc-plugin-dev=${version} \
libnvinfer-vc-plugin11=${version} \
libnvinfer-win-builder-resource11=${version} \
libnvinfer11=${version} \
libnvonnxparsers-dev=${version} \
libnvonnxparsers11=${version} \
python3-libnvinfer-dev=${version} \
python3-libnvinfer-dispatch=${version} \
python3-libnvinfer-lean=${version} \
python3-libnvinfer=${version} \
tensorrt-dev=${version} \
tensorrt-libs=${version} \
tensorrt=${version}

If you prefer to use aptitude you can instead use the following shorter commands to install an older version of TensorRT or with an older version of CUDA.

sudo apt-get install aptitude
sudo aptitude install -o "Aptitude::ProblemResolver::Hints::=reject tensorrt :UNINST" tensorrt=${version}

Hold the TensorRT packages at the currently installed version so they don’t upgrade to the latest version automatically.

sudo apt-mark hold \
libnvinfer-bin \
libnvinfer-cmake-dev \
libnvinfer-dev \
libnvinfer-dispatch-dev \
libnvinfer-dispatch11 \
libnvinfer-headers-dev \
libnvinfer-headers-plugin-dev \
libnvinfer-headers-python-plugin-dev \
libnvinfer-lean-dev \
libnvinfer-lean11 \
libnvinfer-plugin-dev \
libnvinfer-plugin11 \
libnvinfer-safe-headers-dev \
libnvinfer-vc-plugin-dev \
libnvinfer-vc-plugin11 \
libnvinfer-win-builder-resource11 \
libnvinfer11 \
libnvonnxparsers-dev \
libnvonnxparsers11 \
python3-libnvinfer-dev \
python3-libnvinfer-dispatch \
python3-libnvinfer-lean \
python3-libnvinfer \
tensorrt-dev \
tensorrt-libs \
tensorrt

To upgrade to latest version, unhold the packages:

sudo apt-mark unhold \
libnvinfer-bin \
libnvinfer-cmake-dev \
libnvinfer-dev \
libnvinfer-dispatch-dev \
libnvinfer-dispatch11 \
libnvinfer-headers-dev \
libnvinfer-headers-plugin-dev \
libnvinfer-headers-python-plugin-dev \
libnvinfer-lean-dev \
libnvinfer-lean11 \
libnvinfer-plugin-dev \
libnvinfer-plugin11 \
libnvinfer-safe-headers-dev \
libnvinfer-vc-plugin-dev \
libnvinfer-vc-plugin11 \
libnvinfer-win-builder-resource11 \
libnvinfer11 \
libnvonnxparsers-dev \
libnvonnxparsers11 \
python3-libnvinfer-dev \
python3-libnvinfer-dispatch \
python3-libnvinfer-lean \
python3-libnvinfer \
tensorrt-dev \
tensorrt-libs \
tensorrt

Verification#

For quick checks that apply to every install method, refer to Verifying Your Installation in the installation overview. The steps below are specific to Debian packages.

These verification steps apply to both Local Repo and Network Repo installation methods.

Package Verification:

Verify that TensorRT packages are installed:

Table 4 Package Verification Commands#

Package Type

Command

Expected Output

Full TensorRT Release

dpkg-query -W tensorrt
tensorrt  11.5.0.x-1+cuda13.5

Lean/Dispatch Runtime

dpkg-query -W "*nvinfer*"

Lists the name and version of every installed libnvinfer* package, such as libnvinfer-lean11 and libnvinfer-dispatch11. To list the files a package installed instead, use dpkg -L <package>.

C++ Verification:

Verify the trtexec executable is working:

trtexec --help

If the shell reports trtexec: command not found, the Debian package installed the executable but its directory is not on your PATH. List the files the package installed and filter for it:

dpkg -L tensorrt | grep trtexec

Add the directory holding the executable to PATH, then run trtexec --help again:

export PATH=<directory from the previous command>:$PATH

Compile and run a sample, such as sampleOnnxMNIST. Samples and sample data are only available from GitHub. The instructions to prepare the sample data can be found within the samples README.md.

Clone the repository and initialize its submodules, which the sample build requires:

$ git clone -b main https://github.com/NVIDIA/TensorRT.git
$ git -C TensorRT submodule update --init --recursive

Note

The samples are built as C++20, so the host needs a C++20-capable compiler. RHEL and Rocky Linux 8 default to GCC 8.5, which is not, and the build fails while compiling the samples rather than at the configure step. On those releases, enable a gcc-toolset that provides GCC 11 or later before you configure:

$ sudo dnf install gcc-toolset-13
$ scl enable gcc-toolset-13 bash
$ export CXX=$(command -v g++)

The export is not redundant. Where the configure step passes -DCMAKE_PREFIX_PATH=/usr, CMake searches that prefix for a compiler before it searches PATH, so it selects /usr/bin/c++, the GCC 8.5 that the CUDA toolkit installs, even inside the toolset shell. Setting CXX takes priority over that search.

The supported Ubuntu, Debian, and RHEL and Rocky Linux 9 and later releases ship a C++20-capable compiler as the default.

Then build all the samples with the following commands:

$ export TRT_LIBPATH=/usr/lib/x86_64-linux-gnu
$ cd <cloned_tensorrt_dir>
$ mkdir build && cd build
$ cmake .. \
   -DCMAKE_PREFIX_PATH=/usr \
   -DCMAKE_RUNTIME_OUTPUT_DIRECTORY=`pwd`/out \
   -DCMAKE_LIBRARY_OUTPUT_DIRECTORY=`pwd`/out \
   -DCMAKE_ARCHIVE_OUTPUT_DIRECTORY=`pwd`/out \
   -DBUILD_SAMPLES=ON \
   -DBUILD_PARSERS=OFF \
   -DBUILD_PLUGINS=OFF
$ cmake --build . --parallel 4
$ ./out/sample_onnx_mnist

CMAKE_PREFIX_PATH points at the root of the TensorRT installation, which is /usr for Debian packages. The build locates TensorRT through the CMake configuration files that libnvinfer-cmake-dev installs, so install that package before configuring. The three output directory settings collect the built binaries into build/out; without them, CMake writes each sample to its own subdirectory under build/samples.

For detailed information about the samples, refer to TensorRT Sample Support Guide.

Python Verification:

import tensorrt as trt
print(trt.__version__)
assert trt.Builder(trt.Logger())

Troubleshooting#

For install-wide diagnostics (CUDA, drivers, samples, and runtime failures), refer to Troubleshooting. These steps apply to both Local Repo and Network Repo Debian installs.

Method-specific issues for Debian packages:

Issue: E: Unable to locate package tensorrt

  • Solution: Ensure you completed the repository setup steps correctly. Check that the repository is enabled:

    apt-cache policy tensorrt
    

Issue: Dependency conflicts with existing CUDA installation

  • Solution: Ensure CUDA Toolkit was installed using Debian packages (not tar file or runfile).

Issue: Samples fail to compile

  • Solution: Install build essentials:

    sudo apt-get install build-essential cmake