Installing TensorRT-RTX#

There are several installation methods for TensorRT-RTX. This section covers the most common options using:

  • An SDK zip file (Windows), or

  • A tarball file (Linux), or

  • A standalone pip wheel file (all platforms).

Prerequisites#

  1. Ensure you are a member of the NVIDIA Developer Program. If you need help, follow the prompts to gain access and download the package you want to install.

    1. Go to this download link.

    2. Click GET STARTED, then click Download Now.

    3. Select the version of TensorRT that you are interested in.

    4. Select the checkbox to agree to the license terms.

    5. Click the package you want to install. Your download begins.

  2. Install NVIDIA CUDA Toolkit 12.9 or later on the target system.

Windows SDK Installation#

  1. Decompress the Windows zip package. The following can be directly decompressed to extract the:

    • Core library and ONNX parser DLLs and import libraries

    • Development headers

    • Source code samples

    • Documentation

    • Python bindings

    • The tensorrt_rtx executable for building engines and running inference from the command line

    • Licensing information and open-source acknowledgments

    Note

    The DLLs are signed and verified with the signtool utility. Remember to add the directories containing the DLLs and executable files to your PATH environment variable.

  2. Optionally, install the Python bindings.

    $version = "1.0.0.14" # Replace with newest version
$arch = "amd64" # Replace with your architecture
$pyversion = "311" # For Python 3.11, replace with your
                   # Python version
$wheel = "TensorRT-RTX-$version\tensorrt_rtx-$version-cp$pyversion-none-win_$arch.whl"
python3 -m pip install $wheel

Linux Tarball Installation#

TensorRT-RTX can be installed from a tarball package on Linux similar to the zip package installation on Windows. It is supported on the following distributions:

  • Ubuntu 22.04

  • Ubuntu 24.04

  • Rocky Linux 8

Prerequisites

Ensure that NVIDIA CUDA Toolkit 12.9 or later is installed on the target system.

Steps

  1. Download the repo file that matches your operating system version and CPU architecture.

  2. Unzip the tarball, optionally adding the path of the executable to your PATH variable and the path of the library to your LD_LIBRARY_PATH variable.

    version = "1.0.0.14" # Replace with newest version
arch = "x86_64"      # Replace with your architecture
cuda = "12.9"        # Replace with your CUDA version
tarfile = "TensorRT-RTX-${version}.Linux.${arch}-gnu-${cuda}.tar.gz"
tar -xzf $tarfile
export PATH=$PATH:$PWD/TensorRT-RTX-${version}/bin
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$PWD/TensorRT-RTX-${version}/lib

  3. Optionally, install the Python bindings.

    pyversion = "311" # Assuming Python 3.11, else replace with your
                  # Python version
wheel = "tensorrt_rtx-${version}-cp${pyversion}-none-linux_${arch}.whl"
python3 -m pip install TensorRT-RTX-${version}/python/${wheel}

Python Package Index Installation#

When installing TensorRT-RTX from the Python package Index, it is not necessary to install it first from a zip file or tar file package. All necessary libraries are included in the Python package and will be downloaded automatically.

The tensorrt_rtx Python wheel files currently support Python versions 3.8 to 3.13 and will not work with other versions. We support Linux and Windows operating systems, specifically Ubuntu 20.04 or newer, Rocky Linux 8, and Windows 10 or newer.

We recommend installing inside a Python virtual environment. Alternatively, when not running as root, the --user flag may be appended to any of the pip commands provided to install the wheels in a local path.

Prerequisites

  • If necessary, upgrade the pip module to the latest version and ensure the wheel Python module is present.

    python3 -m pip install --upgrade pip
python3 -m pip install wheel

Steps

  1. Install the TensorRT-RTX Python wheel. The following pip command will pull in all the required CUDA libraries in Python wheel format from PyPI because they are dependencies of the TensorRT-RTX Python wheel. Also, it will upgrade tensorrt_rtx to the latest version if you have a previous version installed.

    python3 -m pip install --upgrade tensorrt_rtx

  2. Verify that the installation is working.

    python3
>>> import tensorrt_rtx
>>> print(tensorrt_rtx.__version__)
>>> assert(tensorrt_rtx.Builder(tensorrt_rtx.Logger())

Basic TensorRT-RTX Workflow#

Models can be specified via the TensorRT-RTX C++ or Python API, or read from the ONNX neural network exchange format. Popular model training frameworks like PyTorch or TensorFlow typically offer an ONNX export option. Alternatively, the native TensorRT-RTX API defines operators for manually assembling the network structure.

ONNX is a framework-agnostic option that works with models in TensorFlow, PyTorch, and more. TensorRT-RTX supports an automatic conversion from ONNX files using the TensorRT-RTX API or the tensorrt_rtx executable, which we will use in this section. ONNX conversion is all-or-nothing, meaning all operations in your model must be supported by TensorRT-RTX.

For the most performance and customizability possible, you can manually construct TensorRT-RTX engines using the TensorRT-RTX network definition API. This involves building an identical network to your target model in TensorRT-RTX operation by operation, using only TensorRT-RTX operations. After a TensorRT-RTX network is created, you will export just the weights of your model from the framework and load them into your TensorRT-RTX network. For this approach, more information about constructing the model using TensorRT-RTX’s network definition API can be found here: