TensorRT 11.5.0 Release Notes#
These Release Notes apply to x86 Linux and Windows users, ARM-based CPU cores for Server Base System Architecture (SBSA) users on Linux, and Windows on ARM users. Use them to identify the supported software components and versions in this release, review compatibility and support information, and locate new features, fixed issues, and known limitations.
Announcements#
Windows 10: TensorRT 11.5 will be the last release supporting Windows 10.
Key Features and Enhancements#
CUDA 13.5: TensorRT 11.5.0 uses CUDA 13.5. Refer to the TensorRT Support Matrix for the supported CUDA versions per platform and to TensorRT Prerequisites for installation requirements.
NVIDIA GB300 support on Windows on ARM: TensorRT 11.5.0 adds GPU support for NVIDIA GB300 (compute capability 10.3) on Windows on ARM. This is a beta release supporting NVIDIA GB300 on Windows on ARM. The supported precisions are the same as for compute capability 10.3 on x86-64: TF32, FP32, FP16, FP8, FP4, BF16, and INT8. GB300 has no DLA hardware, so the Windows on ARM DLA restrictions, including the absence of GPU fallback, do not apply to it. Refer to the TensorRT Support Matrix.
CMake COMPONENTS support: The TensorRT CMake config now supports the
COMPONENTSargument of CMake’sfind_packagesignature, so the config can be imported even when components such as builder resource libraries or plugins have been stripped from the installation. The available components vary by product and are listed in the mainTensorRT-<Product>Config.cmakefile.Hardware compatibility on Linux ARM SBSA: TensorRT 11.5.0 extends hardware compatibility to Linux ARM SBSA systems. An engine built with
HardwareCompatibilityLevel::kAMPERE_PLUScan run on supported NVIDIA Ampere and later GPUs across Linux ARM SBSA systems, including NVIDIA GH200, Grace Blackwell systems, and NVIDIA DGX Spark, so one engine can cover those GPU architectures. Hardware compatibility remains unsupported on NVIDIA DriveOS and NVIDIA JetPack. Refer to Hardware Compatibility.
Breaking ABI Changes#
TensorRT 11.0 removed weak typing APIs along with several other deprecated APIs. Refer to the NVIDIA TensorRT Migration Guide for more information.
TensorRT 11.3.0 changed the safety Consistency Checker C++ API (
nvinfer2::safe::consistency) from STL-container parameters to pointer and count parameters.createConsistencyCheckernow takeschar const* const* pluginBuildLibsandint64_t nbPluginBuildLibsinstead ofstd::vector<std::string>, and the serialized-blob size argument isint64_tinstead ofsize_t.IPluginChecker::validatenow takes a pointer and countsTensorDescriptorarguments instead ofstd::vector<TensorDescriptor>. Applications that callcreateConsistencyCheckerand custom libraries that implementIPluginCheckermust rebuild against matching TensorRT 11.3.0 headers and libraries. Mixing binaries built against the previous signatures with the new checker library can crash.
Compatibility#
For comprehensive platform compatibility, hardware requirements, and feature availability information, refer to the TensorRT Support Matrix. The support matrix provides detailed information about supported operating systems, CUDA versions, GPU architectures, precision modes, compiler requirements, and ONNX operator support for TensorRT 11.5.0.
Limitations#
CPU-only engine building (
BuilderFlag::kCPU_ONLY) does not support plugins that require GPU resources during construction, cloning, or engine building. Networks containing these plugins must be built on a host with a supported GPU, withkCPU_ONLYdisabled.DLA is not supported for NVIDIA DriveOS deployments in TensorRT 11.x, including NVIDIA DRIVE AGX Thor. Refer to Working with DLA for the platforms and packages that support DLA.
NVIDIA JetPack is not supported in TensorRT 11.5.0. Jetson AGX Orin and Jetson AGX Thor deployments remain on TensorRT 11.4.1, which is the latest release that supports NVIDIA JetPack 7.2.2 or later. Refer to NVIDIA Jetson platform eligibility for the TensorRT and CUDA versions each board supports.
Jetson Orin on NVIDIA JetPack 6.x is not supported in TensorRT 11.x. Those deployments must remain on a TensorRT 10.x release supported by their JetPack version.
Version Compatible engine builds from ONNX models with explicit quantization/dequantization (Q/DQ) nodes may fail during engine build if Q/DQ is per-channel scaling for convolution filter.
When implementing a custom layer using
IPluginV3plugin class where the custom layer has data-dependent shape (DDS), the size tensors must be of onlyINT64type and notINT32type, as the latter would result in a compilation failure. Related samples have been updated accordingly.
Deprecated API Lifetime#
APIs deprecated in TensorRT 11.5 will be retained until November 2027.
APIs deprecated in TensorRT 11.4.x will be retained until October 2027.
APIs deprecated in TensorRT 11.3.0 will be retained until August 2027.
APIs deprecated in TensorRT 11.2.1 will be retained until July 2027.
APIs deprecated in TensorRT 11.1.0 will be retained until June 2027.
APIs deprecated in TensorRT 11.0.0 will be retained until March 2027.
APIs deprecated in TensorRT 10.16 will be retained until March 2027.
APIs deprecated in TensorRT 10.15 will be retained until January 2027.
See also
- Migrating from TensorRT 10.x to 11.x
Step-by-step migration guide for upgrading from TensorRT 10.x.
- Upgrading TensorRT
Package-level upgrade instructions (pip, Debian, RPM, tar, zip).
Deprecated and Removed Features#
Windows 10 support is deprecated. TensorRT 11.5 is the last release that supports Windows 10, and later releases will not. Windows 10 has been End-of-Life since October 2025, and GeForce driver support for Windows 10 is also being reduced. For more information, refer to GeForce Support Plan for Windows 10.
The TensorRT Python bindings for Python versions 3.8 and 3.9 are deprecated. These Python versions are considered End-of-Life by Python upstream and have not been supported by the TensorRT samples for multiple releases. These bindings will be removed in future releases after March 2027.
The TensorRT Python bindings for Python version 3.10 are deprecated. Support for the TensorRT Python samples when using Python version 3.10 is also deprecated, and NVIDIA recommends upgrading to Python version 3.12 or later. The Python version 3.10 bindings and associated TensorRT sample support will be removed in future releases after November 2027.
Fixed Issues#
Fixed a GPU inference time regression for a strongly typed BF16 BERT base cased network built as a refittable engine, which ran slower in TensorRT 11.4.0 than in the previous release. The regression had been observed on NVIDIA H100 with CUDA 13.4 at sequence length 512. This addresses the performance issue described in the TensorRT 11.4.1 release notes Known Issues.
Fixed an enqueue failure for engines that contain a data-dependent-shape operation when the CUDA stream is in graph-capture mode. The call previously failed with TensorRT error code 1 and CUDA error 900 (
operation not permitted when stream is capturing), after TensorRT logged a warning that the engine contains operations that are not permitted under CUDA graph capture mode. The failure could also leave the stream and the allocator in an inconsistent state, producing a follow-on deallocation error. This addresses the functional issue described in the TensorRT 11.4.1 release notes Known Issues.Fixed the Engine Inspector omitting an original ONNX layer name from the per-layer metadata of a built engine. A name set on a transpose operation could be dropped when graph optimization folded that operation into a view, so the originating ONNX layer could not be found in the inspector output. Only layer attribution was affected, and the engine built and ran correctly. The behavior had been observed with a quantized ResNet-50 Q/DQ model on NVIDIA RTX 5090 and on NVIDIA DGX Spark with CUDA 13.2 and later. This addresses the functional issue described in the TensorRT 11.4.1 release notes Known Issues.
Fixed a CUDA unspecified launch failure when running inference with
trtexecon NVIDIA Blackwell architecture GPUs for networks whose engine compiles into a single fused subgraph. Observed cases included INT8 per-channel quantized BERT QAT models and strongly typed FP16 NVIDIA Maxine™ speech models on Linux with CUDA 13.4, and some runs reached the test time limit instead of crashing. This addresses the functional issue described in the TensorRT 11.4.1 release notes Known Issues.Fixed a segmentation fault when creating an execution context for a Version Compatible engine. The engine deserialized successfully, and the crash occurred while the lean runtime initialized the execution context and resolved a matrix-multiplication operation. The failure had been observed with
trtexec --vcfor strongly typed FP16 ResNet-50 v1.5 and ViT networks on CUDA 13.2, and the same networks ran correctly when the engine was built without--vc. This addresses the functional issue described in the TensorRT 11.4.1 release notes Known Issues.Fixed an engine build failure for SDXL UNetXL ONNX networks built with
trtexec, which failed during optimizer shape analysis with an API Usage Error fromIConcatenationLayer. The error was raised at the/up_blocks.1/Concatnode and reported that axis 2 dimensions must be equal for concatenation on axis 1, with mismatched dimensions 2 and 1. It occurred whentrtexecauto-overrode the sample input shape to1x4x1x1, which produced mismatched spatial dimensions on the UNet down and up path before concatenation. The failure had been observed on Windows with NVIDIA RTX 4090 for standard and strongly typed FP16 UNetXL ONNX models, with and without TF32. This addresses the functional issue described in the TensorRT 11.4.1 release notes Known Issues.Fixed an engine build failure for FLUX.1-dev single-transformer attention ONNX networks built with
trtexec, which failed during optimizer shape analysis with an API Usage Error fromIAttentionInputLayer. The error was raised atfused_attention_0_AttentionInputand reported that Key and Value did not match at dimension 2. Before the check failed,trtexeccould auto-override unspecified dynamic inputs, includinghiddento1x1x3072,img_idsto1x3, andpooled_projectionsto1x768. The failure had been observed on Windows with NVIDIA RTX 4090, with and without TF32. This addresses the functional issue described in the TensorRT 11.4.1 release notes Known Issues.Fixed numerically incorrect output from a strongly typed post-training quantized INT8 plus FP16 YOLOX-Tiny network at 640 resolution. The observed median error had required a tolerance of about 0.43 absolute or about 9.6% relative, against an expected tolerance of 0.05 absolute or 2% relative. The failure had been observed on Windows with NVIDIA RTX 5080 on CUDA 12.9 and NVIDIA RTX 5090 on CUDA 13.0. This addresses the functional issue described in the TensorRT 11.4.1 release notes Known Issues.
Known Issues#
Functional
On CUDA versions prior to 13.2, the compute-sanitizer
initchecktool may flagUninitialized __global__ memory readerrors when running TensorRT applications on NVIDIA Hopper GPUs. These errors are false positives and can be safely ignored. To suppress them, upgrade to CUDA 13.2 or later.Running TensorRT applications under Valgrind memcheck on CUDA 13.3 may report host-side memory leaks attributed to NVRTC components used during engine build and runtime compilation. These reports are not known to affect normal inference behavior.
On NVIDIA Vera Rubin, running TensorRT under Valgrind memcheck may report host-side memory leaks attributed to NVVM components used to compile kernels during engine build. The leak is roughly 40 KB for each kernel compiled through this path and is not known to affect inference results. Vera Rubin is the only architecture that compiles kernels through this path, so the same runs on other GPUs do not report it.
Running TensorRT inference under compute-sanitizer
memcheckmay report device memory errors during execution or engine teardown. These reports are not known to affect normal inference behavior when sanitizers are not enabled.On NVIDIA B100, networks that use multi-head attention with packed query and variable-length key-value input (
IAttentionwithsetKeyValueLengths) may fail under compute-sanitizermemcheck. The sanitizer can report invalid__global__atomic accesses in the attention kernel, followed by CUDA error 719 during memory deallocation and an unrecoverable CUDA error. This is not a memory leak; sanitizer leak summaries showed 0 leaked bytes. Equivalent runs without the sanitizer have passed. A related failure on NVIDIA B100 may reject an unsupported stride order[3, 2, 1, 0]for a key operand under compute-sanitizer or Valgrind.inplace_addmini-sample of thequickly_deployable_pluginsPython sample may produce incorrect outputs on Windows.TensorRT may exit if inputs with invalid values are provided to the
RoiAlignplugin (ROIAlign_TRT), especially if there is inconsistency in the indices specified in thebatch_indicesinput and the actual batch size used.On Windows, Version Compatible engines built with TensorRT 10.1 through 10.4 that use the
RoiAlignplugin (ROIAlign_TRT) cannot be deserialized with the TensorRT 11.x runtime due to a bug in how those engines were produced. Deserialization fails inside the runtime dispatch VC plugin loader when loadingnvinfer_vc_plugin.dll(WindowsERROR_MOD_NOT_FOUND/ error 126), surfaces as TensorRT error code 6 (API Usage Error), and leaves the returnedICudaEnginepointer null. The bug was fixed in TensorRT 10.5. Engines built with TensorRT 10.5 or later are expected to deserialize correctly. As a workaround, rebuild affected engines with TensorRT 10.5 or later before loading them with TensorRT 11.x.Inspecting very large engines with the Engine Inspector may fail on systems with limited host memory. The inspection process can exceed available memory and be terminated by the operating system without a TensorRT error message. This has been observed with very large diffusion-model engines. As a workaround, run engine inspection on a system with sufficient host memory.
Building TensorRT OSS samples with Visual Studio / MSBuild on Windows may fail during the CMake configure step when
CUDA_HOMEandCUDA_PATHpoint to different CUDA Toolkit installations. The failure is environment-wide rather than sample-specific and can affect multiple sample targets andtrtexec. As a workaround, ensureCUDA_HOMEandCUDA_PATHrefer to the same CUDA Toolkit, or unset one of the variables so CMake discovers a single consistent toolkit.IRuntime::getEngineValiditymay reportEngineValidity::kVALIDfor engine data that was modified after serialization, rather thanEngineValidity::kINVALID. The failure has been observed on Windows with NVIDIA RTX 4090. Do not rely on this API alone to detect a tampered or truncated engine plan.Changing input shapes on a network that uses an
IPluginV3plugin may fail with TensorRT error code 2 (Internal Error) from the plugin shape-change path. The failure has been observed as a regression on NVIDIA A100 with CUDA 12.6.Multi-device inference may fail to initialize its NCCL communicator on Windows platforms, so
enqueueV3returns TensorRT error code 1 and distributed collective operations such asall_gatherdo not run. The failure has been observed with the distributed collective sample on Windows with eight NVIDIA A10 GPUs and CUDA 12.9. Another multi-device related problem was observed on Linux with eight NVIDIA A30 GPUs, where the NCCL communicator was not usable after setting a new optimization profile.On Windows on ARM, building a DLA network that contains a constant with no consumers may fail during engine build with an internal assertion from the DLA constant path. Such a constant serves no purpose and is unlikely to appear in a real network. As a workaround, remove any constant that no layer consumes.
On Windows on ARM, DLA networks that use
UINT8tensors may fail during engine build. Reported cases include a constant that produces aUINT8output, a network-levelUINT8output that is not produced byIIdentityLayerorICastLayer, a strongly typed network whose requestedUINT8output type does not match the inferredINT8type, and aDequantizeLinearnode with aUINT8input. As a workaround, insert anICastLayerto convertUINT8tensors to a supported type.On Windows on ARM, DLA does not support
BOOLconstants. A network that places aBOOLconstant on the DLA path fails during engine build with TensorRT error code 1 reporting an unsupported data type. Because Windows on ARM has no GPU fallback for DLA systems, the build cannot place the affected layer on the GPU instead. This restriction applies to constants rather than to theBOOLtype in general. An ElementWise comparison can run on DLA when it matches one of two patterns:Equal,Greater, orLessfollowed by aCastto FP16 or INT8; orGreaterorLessfollowed byNotand then aCastto FP16 or INT8. A comparison that does not match one of these patterns fails the build with a message that lists them.DLA requires that every quantized value be consumed by a dequantize (DQ) node. A strongly typed network that feeds quantized data directly into a node that is not a DQ node is rejected during engine build, with an error reporting that the node cannot be quantized by its input and that a dequantize node is needed before it. This is expected behavior rather than a defect. Follow every quantize (Q) node with a DQ node so that each quantized value is consumed by a DQ node.
On Windows on ARM, a cast that TensorRT inserts on the DLA path may fail the build with TensorRT error code 4, reporting that the layer is not supported on DLA while GPU fallback is not enabled. The ONNX parser inserts such a cast for operators including
TopKandGather, and a cast in the postprocessing part of a network can fail the same way. A standaloneCastmay also be unplaceable on DLA under the numeric-safety constraints that DLA applies. Windows on ARM does not support GPU fallback for DLA systems, so every layer must be DLA-capable. As a workaround, remove the cast from the ONNX model, or keep the affected subgraph off the DLA path.On Windows on ARM, a DLA network that applies
DequantizeLinearto an INT8 tensor and then reduces over the height and width dimensions may produceNaNoutput with no error reported. The failure has been observed at every batch size, including batch 1. As a workaround, reduce over the channel dimension only, or supply an FP16 input in place of the INT8 input, both of which produce correct results.On Windows on ARM, building a DLA network with a large batch count may fail during engine build. TensorRT reports that DLA compilation failed with error code 6 and then raises an internal error, because a separate scratch allocation is reserved for each batch and the total exceeds the memory available to DLA. The failure has been observed with a reduction network at 240 batches. As a workaround, lower the batch count.
On Windows on ARM, a strongly typed DLA network may show an intermittent accuracy drop. The drop has been observed when a model runs as part of a larger suite and does not reproduce when the same model is run on its own, so a single re-run is not sufficient to confirm accuracy.
On Windows on ARM, TensorRT will synchronize the device when the address for an input or output tensor changes. This works around a bug in cuDLA that can cause memory corruption.
A network that applies FP8 quantization to the output of an attention block may produce incorrect results when the key length changes between inference calls. In this case, shape-dependent parameters from a previous call may be reused for a call with a different key length. The issue has been observed on NVIDIA RTX 5090 with CUDA 13.4 and was not introduced in this release. As a workaround, remove the FP8 quantization (Q/DQ) applied to the attention output. This may reduce performance for the downstream operations.
On NVIDIA B300, building a long-sequence FP8 attention network with a CUDA 12.x build of TensorRT may fail during tactic selection, reporting that no implementation could be found for the attention node. TensorRT does not fuse FP8 attention into a single kernel on this combination, and the unfused intermediate is too large to allocate. The failure has been observed with a causal attention network at sequence length 79200 and is not new in TensorRT 11.5.0. As a workaround, use a CUDA 13.x build of TensorRT, where the same network builds.
On Windows on ARM,
trtexecdoes not load the TensorRT plugin library, so parsing an ONNX model that requires a standard plugin fails with a message that the plugin was not found in the plugin registry. The plugins are present in the package and onlytrtexecfails to register them, so an application that loadsnvinfer_plugin.dlland callsinitLibNvInferPluginsis unaffected. Refer to Extending TensorRT with Custom Layers. As a workaround, build the engine with Polygraphy, which registers the plugins itself.
Performance
On NVIDIA DGX Spark, strongly typed ONNX networks may show about 10%–20% higher GPU time and use up to 76% more execution-context device memory than with TensorRT 11.4.1. GPUs with discrete memory are unaffected.
On NVIDIA B300 and NVIDIA GB300, a strongly typed FLUX.1-dev single-transformer attention network at batch 4 may use about 33% more execution-context device memory in TensorRT 11.5.0 than in TensorRT 11.4.0, growing from 1296 MiB to 1728 MiB. This will be addressed in a future release.
On NVIDIA B300, a rounding correctness fix in the CUDA 12.9 build of TensorRT 11.5.0 may increase GPU time by about 12% for strongly typed FP16 LSTM networks compared with TensorRT 11.4.0.
On Windows with NVIDIA Vera Rubin, every process that builds an engine may spend about 160 seconds in graph construction before the build itself begins, because TensorRT compiles its builder kernel library for the GPU at startup instead of loading a prebuilt copy. The cost is paid by each new process regardless of model size and applies to engine build only, so inference is unaffected. Linux is not affected.
On NVIDIA RTX PRO 6000 Blackwell (compute capability 12.0), a strongly typed FP16 ViT-Base patch-16 384 network (
vit_base_patch16_384_strongType_fp16.onnx) may show about 6% to 7% higher GPU time in TensorRT 11.3.0 than in TensorRT 11.2.1. The slowdown is associated with tactic selection for the MLPfc2MatMul layers and does not reproduce on NVIDIA Ada Lovelace architecture GPUs.On NVIDIA GH200 480 GB SBSA (AArch64), FP8 causal attention may show about 11% higher GPU compute time in TensorRT 11.3.0 than in TensorRT 11.2.1. The slowdown is associated with the CUDA 13.4 device compiler: fused multi-head attention kernel source is unchanged, but the compiled device code is slower. Engine structure and tactic selection are otherwise the same.
A non-zero tiling optimization level set through
IBuilderConfig::setTilingOptimizationLevel()might introduce engine build failures for some networks on L4 GPUs.Engines built with
BuilderFlag::kREFITorkREFIT_IDENTICALhave performance regressions compared with non-refit engines where convolution layers are present within a branch or loop, and the precision is FP16/INT8.Some Maxine™ AR Facial Landmark Detection models (
landmark_quality_68_dynamic_strongtype_fp16andlandmark_quality_126_dynamic_strongtype_fp16) may exhibit approximately 7–9% higher GPU inference time in TensorRT 11.2.x than in TensorRT 11.1.0 on NVIDIA RTX PRO 6000 Blackwell Max-Q. The change was made for a functional fix, and the absolute impact is typically about 0.1 ms on workloads of roughly 1.7–1.9 ms.On NVIDIA Jetson AGX Orin, DLA networks that mix INT8 and FP16 may show higher GPU compute time when built with TensorRT 11.4.1 than the equivalent weakly typed networks built with TensorRT 10.3. The impact depends on whether the model was converted with the
--target_dlaoption of the NVIDIA TensorRT Model Optimizer ONNX AutoCast tool:Without
--target_dla: ResNet-50 sparse, AHC, and PredictionNet chopped 1 may be more than 30% slower. YOLOv3-Tiny, Super Resolution, and SSD-MobileNet-v1 (batch 4) may be 5% to 30% slower.With
--target_dla: ResNet-50 sparse and AHC may be more than 30% slower. Super Resolution and SSD-MobileNet-v1 (batch 4) may be 5% to 30% slower. PredictionNet chopped 1 and YOLOv3-Tiny are not affected.
Refer to Strongly Typed Networks for DLA for migration guidance, and try the conversion both with and without
--target_dlato determine which performs better for your model.