Is this page helpful?

Migrating TensorRT from 10.x to 11.x on Jetson/JetPack#

This page describes platform-specific migration considerations for TensorRT on NVIDIA Jetson platforms (JetPack) when upgrading from TensorRT 10.x to 11.x. For API-level migration details, refer to the C++, Python, and trtexec migration pages.

Platform eligibility for Jetson/JetPack is summarized in NVIDIA Jetson platform eligibility in the TensorRT Support Matrix.

TensorRT 11.4.1 Support Status on Jetson/JetPack#

TensorRT 11.4.1 supports NVIDIA JetPack 7.2.2 or later on Jetson AGX Orin and Jetson Thor. Install the TensorRT SBSA build: both are served by the Linux SBSA package, which is now a single package covering both DLA and non-DLA configurations, so there is no JetPack-specific package to select.

TensorRT 11.4.1 also restores DLA support, on Jetson AGX Orin only, and only under NVIDIA JetPack 7.2.2 or later. The SBSA build does not provide DLA outside NVIDIA JetPack, and Jetson Thor has no DLA. This support is generally available, and GPU fallback behaves as it did in TensorRT 10.x. DLA was unavailable in TensorRT 11.0 through 11.3, and TensorRT 11.4.0 supported DLA only on Windows on ARM. Refer to Working with DLA for the full platform and package breakdown, including DriveOS configurations where DLA is not supported.

Jetson Orin on NVIDIA JetPack 6.x is not supported in TensorRT 11.4.1. Those deployments must remain on a TensorRT 10.x release supported by their JetPack version, and must upgrade to JetPack 7.x before moving to TensorRT 11.4.1.

Note

TensorRT 11.0 through 11.4.0 did not support JetPack. Deployments that remain on one of those releases must stay on a TensorRT 10.x release supplied for their JetPack version until they upgrade to 11.4.1 or later.

Platform Eligibility#

Platform

TensorRT Version

Notes

Jetson AGX Orin (JetPack 7.2.2 or later)

11.4.1

Install the TensorRT SBSA build. DLA is supported, and is generally available, when running under JetPack 7.2.2 or later. The SBSA build does not provide DLA outside JetPack. Refer to Working with DLA.

Jetson Thor (JetPack 7.2.2 or later)

11.4.1

Install the TensorRT SBSA build. GPU inference only; Jetson Thor has no DLA.

Jetson Orin (JetPack 6.x)

Not supported in TensorRT 11.4.1

Remain on a TensorRT 10.x release supported by the installed JetPack version. Upgrade to JetPack 7.x before moving to TensorRT 11.4.1.

Who Should Prepare for Migration#

Prepare your Jetson application for a TensorRT 11.x upgrade if it does any of the following:

  • Uses implicit INT8 quantization (BuilderFlag::kINT8 with IInt8Calibrator, or --int8 with trtexec).

  • Relies on weakly typed builder flags (BuilderFlag::kFP16, BuilderFlag::kBF16, BuilderFlag::kFP8, or equivalent flags such as --fp16 with trtexec).

  • Uses IPluginV2DynamicExt or IPluginCreator.

  • References removed tactic sources (kCUBLAS, kCUBLAS_LT, kCUDNN).

If your application already uses strong typing, explicit quantization (Q/DQ nodes), and IPluginV3, the upgrade to TensorRT 11.x should require minimal code changes beyond recompilation.

Migration Checklist#

Apply the following steps when upgrading Jetson deployments to TensorRT 11.4.1 on JetPack 7.2.2 or later.

  1. Install the TensorRT SBSA build. Jetson AGX Orin and Jetson Thor are served by the Linux SBSA package, which covers both DLA and non-DLA configurations. DLA requires JetPack 7.2.2 or later on Jetson AGX Orin. Deployments on JetPack 6.x must upgrade to JetPack 7.x first; until then, remain on a TensorRT 10.x release supported by your JetPack version.

  2. Migrate from weak typing to strong typing. TensorRT 11.x removes all precision-enabling builder flags. Use ModelOpt AutoCast to convert ONNX models to mixed precision before building. Refer to the Strongly Typed Networks section for details.

  3. Migrate from implicit INT8 to explicit quantization. The IInt8Calibrator class and all subclasses have been removed. Use ModelOpt Quantization to produce models with Q/DQ nodes. Refer to the C++ or Python migration page for before/after examples.

  4. Migrate plugins from V2 to V3. IPluginV2DynamicExt and IPluginCreator are deprecated but still present, so V2 plugins continue to build with deprecation warnings. Migrate them to IPluginV3 with IPluginCreatorV3One. Refer to the C++ plugin migration section for a complete NonZero plugin example.

  5. Update V2 API calls to V2 replacements. Several methods (getDeviceMemorySize, setWeightStreamingBudget, setDeviceMemory) have been replaced with V2 versions that use int64_t or accept additional parameters. Refer to the V2 API replacements table.

  6. Rebuild all engines. Engines built with TensorRT 10.x using the BuilderFlag::kVERSION_COMPATIBLE setting are forward-compatible with the TensorRT 11.x runtime. However, to take advantage of 11.x optimizations and new features, rebuild engines with the 11.x builder.

Version Compatibility on Jetson#

For TensorRT 11.4.1 Jetson deployments, the following compatibility rules apply:

  • Engines built with TensorRT 10.x can run on the TensorRT 11.x runtime.

  • Engines built with TensorRT 8.x or 9.x are not compatible with the TensorRT 11.x runtime. TensorRT 8.x uses CUDA 11.x, which is not supported by TensorRT 11.x.

  • Backward compatibility is maintained within a major release. For example, a version-compatible engine built with TensorRT 11.5 runs on TensorRT 11.5 or any later 11.x release, but not on TensorRT 11.0 through 11.4, and not on TensorRT 10.x.

If your Jetson deployment currently uses TensorRT 8.x, you must first migrate to TensorRT 10.x using the 8.x to 10.x appendix, then follow this guide for the 10.x to 11.x migration.

CUDA and JetPack Dependency Changes#

Verify that your JetPack version includes a CUDA toolkit compatible with the TensorRT release you are installing. Refer to the TensorRT Support Matrix for the CUDA version required per platform, and to the JetPack release notes for the CUDA version your image ships.

TensorRT 11.x drops support for CUDA 11.x. Engines built against CUDA 11.x cannot run on the TensorRT 11.x runtime.

The toolchain row for NVIDIA JetPack AArch64 in the support matrix reports the versions TensorRT is built against, not the versions JetPack ships. JetPack 7.2.2 provides a newer compiler and C library than the matrix lists; TensorRT targets the lower versions deliberately so its packages remain compatible with that image. Build against the matrix values rather than the ones reported by the toolchain on your device.

DLA Considerations#

TensorRT 11.4.1 supports DLA on Jetson AGX Orin, and only under NVIDIA JetPack 7.2.2 or later. DLA is delivered through the Linux SBSA package, so installing the SBSA build under a qualifying JetPack is what makes DLA available. There is no separate DLA package to select, and the SBSA build does not provide DLA outside JetPack. Jetson Thor has no DLA; install the same SBSA build there for GPU inference.

DLA on JetPack is generally available, and GPU fallback behaves as it did in TensorRT 10.x. Refer to GPU Fallback Mode for how to enable it.

DLA builds use strongly typed networks. Precision is determined by network tensor types instead of the per-precision builder flags used in TensorRT 10.x, so a DLA model migrating from 10.x needs the same strong-typing conversion as the rest of your network. Refer to Strongly Typed Networks for DLA.

If DLA is required on JetPack 6.x, remain on TensorRT 10.7, which was the last 10.x release that supported DLA.

DLA support is platform- and package-specific. For the platforms and packages that support DLA, including DriveOS configurations, refer to the shared statement in Working with DLA and Migrating TensorRT from 10.x to 11.x on NVIDIA DriveOS.

Cross-Compilation#

If you cross-compile TensorRT applications for Jetson on an x86 host, keep the host TensorRT SDK version aligned with the TensorRT version on the target JetPack / SBSA image. Mixing TensorRT 10.x headers on the host with 11.x libraries on the target (or vice versa) causes compilation or linker errors due to removed APIs.

Recommendation for Edge Deployments#

Tip

For Jetson applications targeting TensorRT 11.4.1, install the TensorRT SBSA build for Jetson AGX Orin or Jetson Thor, then test the migration in a staging environment before deploying to production devices. Verify model accuracy, inference latency, DLA offload (on Jetson AGX Orin, if used), and memory usage against your TensorRT 10.x baseline.

Deployments on JetPack 6.x should continue to use the TensorRT 10.x release supported by their JetPack version until they move to JetPack 7.x.

Customers still using TensorRT 10.x should begin migrating to strong typing, explicit quantization, and IPluginV3 to reduce the scope of changes when upgrading to TensorRT 11.4.1. The TensorRT team provides mitigation plans for all removed APIs.