Is this page helpful?

NVIDIA TensorRT Documentation#

NVIDIA® TensorRT™ is an SDK for optimizing deep learning inference on NVIDIA GPUs. It takes trained models from frameworks such as PyTorch and ONNX, and compiles them into engines for GPU deployment with support for mixed precision (FP32/FP16/BF16/FP8/INT8/FP4/INT4), dynamic shapes, and specialized optimizations for transformers and large language models (LLMs). Measure latency and throughput on your own model and hardware; refer to Best Practices.

Quick Start#

  • New to NVIDIA TensorRT? → Install first, then verify, then build your first engine:

    1. Installation Guide Overview and Installing TensorRT (Debian/RPM, tar/zip, or container for the ~10-minute CLI tutorial)

    2. Verify: trtexec --help (non-pip) or import tensorrt; print(tensorrt.__version__) (pip)

    3. Build Your First Engine (requires trtexec, which is not included in pip wheels)

  • Python API only (pip)? → Method 1: Python Package Index (pip) (pip install tensorrt). Pip installs bindings and libraries only, with no trtexec. For the first-engine CLI tutorial, use Debian/RPM, tar/zip, or container instead.

  • C++ or CLI workflows? → Choose Debian/RPM, tar/zip, or container on Installing TensorRT; run trtexec from the package bin directory

  • Ready for the full workflow menu? → After your first engine, use the Quick Start Guide for PyTorch and ONNX export paths, multiple runtimes, dynamic shapes, and quantization

  • Upgrading from 11.2 or earlier? → Refer to What’s New in 11.3.0 below

  • Upgrading from TensorRT 10.x? → Use the NVIDIA TensorRT Migration Guide to plan your API and builder changes

  • Need help with a specific task? → Jump to the Inference Library Overview for API walkthroughs, dynamic shapes, quantization, and more, or the Troubleshooting section

  • Optimize inference performance → Best Practices

What’s New in NVIDIA TensorRT 11.3.0#

Release Highlights

  • CUDA Toolkit 13.4 dependency upgrade: TensorRT 11.3.0 packages are built against CUDA® Toolkit 13.4; Debian, RPM, tar, and zip package filenames use cuda-13.4. TensorRT 11.3.1 DriveOS packages use CUDA Toolkit 13.4. Refer to the TensorRT Support Matrix for supported CUDA releases per platform and to Prerequisites for installer prerequisites.

  • NVIDIA Vera Rubin support added in TensorRT 11.3.0: NVIDIA Vera Rubin GPUs are supported by TensorRT 11.3.0 Enterprise (general-release). Linux x86, Linux SBSA, and Windows x64 support NVIDIA Vera Rubin GPUs with compute capability version 10.7. Refer to the TensorRT Support Matrix.

  • Refit enhancement in TensorRT 11.3.0: High-precision weights used in FP4 double quantization are now refittable. If you use BuilderFlag::kREFIT, refer to Known Issues in the TensorRT 11.3.0 Release Notes. Refer to Refitting an Engine.

  • Limitations in TensorRT 11.3.0: DLA is not supported in TensorRT 11.3.0 or in TensorRT 11.3.1 for DriveOS (last DLA release is 10.7). NVIDIA JetPack is not supported; Jetson deployments must remain on a TensorRT 10.x release supported by their JetPack version. Refer to the TensorRT 11.3.0 Release Notes.

View 11.3.0 Release Notes

Previous Releases#

Release 11.2.1 Highlights

  • Platform dependency upgrades: Updates internal build dependencies for TensorRT 11.2.1. TensorRT 11.2.1 packages are built against CUDA 13.3 update 1; Debian, RPM, tar, and zip package filenames continue to use cuda-13.3.

  • GridSample 3D support: Extends GridSample from 2D-only to 3D (rank-5 input) with FP32, FP16, and BF16

  • ONNX DFT operator support: Adds a cuFFT-based plugin for forward and inverse C2C, R2C, and C2R transforms

  • PluginV2 to PluginV3 migration sample: Adds a Python sample with PluginV2-to-PluginV3 method mappings

  • Improved CMake support: Tar and Zip packages include CMake configuration files under the cmake directory

View 11.2.1 Release Notes

Release 11.1.0 Highlights

  • CUDA 13.3 dependency upgrade: Updated CUDA Toolkit baseline across Linux x86-64, Windows x64, and SBSA platforms

  • Ubuntu 26.04 support: Adds Ubuntu 26.04 LTS to the supported Linux x86-64 and SBSA platform lists alongside the existing Ubuntu 22.04/24.04 packages

  • Python 3.14 bindings: Extends the Python wheel matrix to Python 3.14 on supported platforms

  • NVFP4 dual-GEMM fusion for SM121: Fuses the gate and up projection GEMMs in NVFP4 MoE/MLP blocks on NVIDIA DGX Spark (compute capability 12.1)

  • Global Performance Tuner: Automates trtexec build-route search to explore internal builder knobs, benchmark candidate engines, and optionally validate accuracy before selecting a route. Measure the benefit on your model and GPU. Refer to Global Performance Tuning.

View 11.1.0 Release Notes

Release 11.0.0 Highlights

  • Strongly typed networks are now the default: Weak-typing APIs (setPrecision, setDynamicRange, the per-precision BuilderFlag family) and implicit quantization (IInt8Calibrator) have been removed. Use the NVIDIA TensorRT Migration Guide to plan your upgrade

  • IPluginV2 has been removed: The entire IPluginV2 family is gone; migrate custom plugins to IPluginV3 with addPluginV3(). Refer to the V2 → V3 walkthrough for a side-by-side API mapping

  • Multi-Device Inference is generally available: Preview flag retired, plus new AllToAll, Gather, and Scatter collective ops, automatic NCCL library fallback, and a new context-parallel attention sample. Refer to Multi-Device Inference

  • Ragged batching for attention: IAttention and IKVCacheUpdateLayer now support packed (kPACKED_NHD) layouts so variable-length sequences can be concatenated end-to-end without padding. Refer to Fused Attention

  • MoE inference performance: NVIDIA Blackwell architecture (SM10x/SM110) backend improvements for common MoE patterns; the previous “keep seqLen ≤ 16” guidance no longer applies. Benchmark your own workload. Refer to MoE (Mixture of Experts)

  • Rewritten Best Practices and Benchmarking guide: Reframed as a measure-then-optimize loop with side-by-side ONNX-TRT (trtexec) and Torch-TRT workflows in synchronized tabs covering quantization, dynamic shapes, CUDA graphs, profiling, and Nsight Systems timeline reading. Refer to Performance Benchmarking

  • Platform updates: RHEL 10 / Rocky Linux 10 RPM and tar packages, and a new TensorRT 10.x to 11.x migration path with dedicated DriveOS and Jetson/JetPack chapters

View 11.0.0 Release Notes

Archived Releases

Earlier TensorRT releases with key highlights:

Legacy Versions

Note

For complete version history and detailed changelogs, visit the Release Notes section or the TensorRT GitHub Releases.