Is this page helpful?

NVIDIA TensorRT Documentation#

NVIDIA TensorRT is an SDK for optimizing deep learning inference on NVIDIA GPUs. It takes trained models from frameworks such as PyTorch, TensorFlow, and ONNX, and compiles them into engines for GPU deployment with support for mixed precision (FP32/FP16/BF16/FP8/INT8/FP4/INT4), dynamic shapes, and specialized optimizations for transformers and large language models (LLMs). Measure latency and throughput on your own model and hardware; refer to Best Practices.

Quick Start#

  • 📦 New to NVIDIA TensorRT? → Install first, then verify, then build your first engine:

    1. Installation Guide Overview and Installing TensorRT (Debian/RPM, tar/zip, or container for the ~10-minute CLI tutorial)

    2. Verify: trtexec --help (non-pip) or import tensorrt; print(tensorrt.__version__) (pip)

    3. Build Your First Engine (requires trtexec — not included in pip wheels)

  • 🐍 Python API only (pip)? → Method 1: Python Package Index (pip) (pip install tensorrt). Pip installs bindings and libraries only — no trtexec. For the first-engine CLI tutorial, use Debian/RPM, tar/zip, or container instead.

  • 🛠️ C++ or CLI workflows? → Choose Debian/RPM, tar/zip, or container on Installing TensorRT; run trtexec from the package bin directory

  • 📖 Ready for the full workflow menu? → After your first engine, use the Quick Start Guide for PyTorch and ONNX export paths, multiple runtimes, dynamic shapes, and quantization

  • ⬆️ Upgrading from 11.1 or earlier? → Refer to What’s New in 11.2.1 below

  • 🔄 Upgrading from TensorRT 10.x? → Use the NVIDIA TensorRT Migration Guide to plan your API and builder changes

  • 🔧 Need help with a specific task? → Jump to the Inference Library Overview for API walkthroughs, dynamic shapes, quantization, and more, or the Troubleshooting section

  • ⚡ Optimize inference performance → Best Practices

🆕 What’s New in NVIDIA TensorRT 11.2.1#

Release Highlights

  • Platform dependency upgrades: Updates internal build dependencies for TensorRT 11.2.1. TensorRT 11.2.1 packages are built against CUDA 13.3 update 1; Debian, RPM, tar, and zip package filenames continue to use cuda-13.3. Refer to the TensorRT Support Matrix for per-platform CUDA version pinning and to Prerequisites for installer prerequisites.

  • GridSample 3D support: Extends GridSample from 2D-only to 3D (rank-5 input) with FP32, FP16, and BF16; linear, nearest, and cubic interpolation; align_corners 0/1; and zeros, border, and reflection padding modes

  • ONNX DFT operator support: Adds a cuFFT-based plugin for forward and inverse C2C, R2C, and C2R transforms

  • PluginV2 to PluginV3 migration sample: Adds a Python sample with PluginV2-to-PluginV3 method mappings. Refer to Plugin API Description and the Sample Explorer

  • Improved CMake support: Tar and Zip packages now include CMake configuration files under the cmake directory. Import TensorRT with find_package(TensorRT-<Product>), where <Product> is Enterprise, Automotive, RTX, or SafeInference

View 11.2.1 Release Notes

Previous Releases#

📋 Release 11.1.0 Highlights

  • CUDA 13.3 dependency upgrade: Updated CUDA Toolkit baseline across Linux x86-64, Windows x64, and SBSA platforms

  • Ubuntu 26.04 support: Adds Ubuntu 26.04 LTS to the supported Linux x86-64 and SBSA platform lists alongside the existing Ubuntu 22.04/24.04 packages

  • Python 3.14 bindings: Extends the Python wheel matrix to Python 3.14 on supported platforms

  • NVFP4 dual-GEMM fusion for SM121: Fuses the gate and up projection GEMMs in NVFP4 MoE/MLP blocks on NVIDIA DGX Spark (compute capability 12.1)

  • Global Performance Tuner: Automates trtexec build-route search to explore internal builder knobs, benchmark candidate engines, and optionally validate accuracy before selecting a route. Measure the benefit on your model and GPU. Refer to Global Performance Tuning.

View 11.1.0 Release Notes

📋 Release 11.0.0 Highlights

  • Strongly typed networks are now the default: Weak-typing APIs (setPrecision, setDynamicRange, the per-precision BuilderFlag family) and implicit quantization (IInt8Calibrator) have been removed. Use the NVIDIA TensorRT Migration Guide to plan your upgrade

  • IPluginV2 has been removed: The entire IPluginV2 family is gone; migrate custom plugins to IPluginV3 with addPluginV3(). Refer to the V2 → V3 walkthrough for a side-by-side API mapping

  • Multi-Device Inference is generally available: Preview flag retired, plus new AllToAll, Gather, and Scatter collective ops, automatic NCCL library fallback, and a new context-parallel attention sample. Refer to Multi-Device Inference

  • Ragged batching for attention: IAttention and IKVCacheUpdateLayer now support packed (kPACKED_NHD) layouts so variable-length sequences can be concatenated end-to-end without padding. Refer to Fused Attention

  • MoE inference performance: Blackwell (SM10x/SM110) backend improvements for common MoE patterns; the previous “keep seqLen ≤ 16” guidance no longer applies. Benchmark your own workload. Refer to MoE (Mixture of Experts)

  • Rewritten Best Practices and Benchmarking guide: Reframed as a measure-then-optimize loop with side-by-side ONNX-TRT (trtexec) and Torch-TRT workflows in synchronized tabs covering quantization, dynamic shapes, CUDA graphs, profiling, and Nsight Systems timeline reading. Refer to Performance Benchmarking

  • Platform updates: RHEL 10 / Rocky Linux 10 RPM and tar packages, and a new TensorRT 10.x to 11.x migration path with dedicated DriveOS and Jetson/JetPack chapters

View 11.0.0 Release Notes

📦 Archived Releases

Earlier TensorRT releases with key highlights:

📖 Legacy Versions

Note

For complete version history and detailed changelogs, visit the Release Notes section or the TensorRT GitHub Releases.