Is this page helpful?

Migrating TensorRT from 10.x to 11.x on Jetson/JetPack#

This page describes platform-specific migration considerations for TensorRT on NVIDIA Jetson platforms (JetPack) when upgrading from TensorRT 10.x to 11.x. For API-level migration details, refer to the C++, Python, and trtexec migration pages.

Platform eligibility for Jetson/JetPack is summarized in NVIDIA Jetson platform eligibility in the TensorRT Support Matrix.

TensorRT 11.2.1 support status on Jetson/JetPack#

TensorRT 11.2.1 does not support NVIDIA JetPack. Jetson deployments must remain on a TensorRT 10.x release supported by their JetPack version. The migration checklist and API guidance on this page apply when a future JetPack release adds TensorRT 11.x support, or when you are preparing a codebase for that upgrade while continuing to build and deploy with TensorRT 10.x today.

Platform Eligibility#

Platform

TensorRT Version

Notes

Jetson Thor (JetPack 7.x)

Not supported in TensorRT 11.2.1

Remain on the TensorRT 10.x release supplied for your JetPack version until JetPack adds TensorRT 11.x support.

Jetson Orin (JetPack 6.x)

10.x

Remain on the TensorRT 10.x release supplied for JetPack 6.x. Orin DLA is not supported by TensorRT 11.x at this time, but will be supported in a future 11.x release.

Who Should Prepare for Migration#

Prepare your Jetson application for a future TensorRT 11.x upgrade if it does any of the following:

  • Uses implicit INT8 quantization (BuilderFlag::kINT8 with IInt8Calibrator, or --int8 with trtexec).

  • Relies on weakly typed builder flags (BuilderFlag::kFP16, BuilderFlag::kBF16, BuilderFlag::kFP8, or equivalent flags such as --fp16 with trtexec).

  • Uses IPluginV2DynamicExt or IPluginCreator.

  • References removed tactic sources (kCUBLAS, kCUBLAS_LT, kCUDNN).

If your application already uses strong typing, explicit quantization (Q/DQ nodes), and IPluginV3, the eventual upgrade to TensorRT 11.x should require minimal code changes beyond recompilation.

Migration Checklist#

When JetPack adds TensorRT 11.x support, apply the following steps:

  1. Migrate from weak typing to strong typing. TensorRT 11.x removes all precision-enabling builder flags. Use ModelOpt AutoCast to convert ONNX models to mixed precision before building. Refer to the Strongly Typed Networks section for details.

  2. Migrate from implicit INT8 to explicit quantization. The IInt8Calibrator class and all subclasses have been removed. Use ModelOpt Quantization to produce models with Q/DQ nodes. Refer to the C++ or Python migration page for before/after examples.

  3. Migrate plugins from V2 to V3. IPluginV2DynamicExt and IPluginCreator have been removed. All plugins must use IPluginV3 with IPluginCreatorV3One. Refer to the C++ plugin migration section for a complete NonZero plugin example.

  4. Update V2 API calls to V2 replacements. Several methods (getDeviceMemorySize, setWeightStreamingBudget, setDeviceMemory) have been replaced with V2 versions that use int64_t or accept additional parameters. Refer to the V2 API replacements table.

  5. Rebuild all engines. Engines built with TensorRT 10.x using the BuilderFlag::kVERSION_COMPATIBLE setting are forward-compatible with the TensorRT 11.x runtime. However, to take advantage of 11.x optimizations and new features, rebuild engines with the 11.x builder.

Version Compatibility on Jetson#

When JetPack supports TensorRT 11.x, the following compatibility rules apply:

  • Engines built with TensorRT 10.x can run on the TensorRT 11.x runtime.

  • Engines built with TensorRT 8.x or 9.x are not compatible with the TensorRT 11.x runtime. TensorRT 8.x uses CUDA 11.x, which is not supported by TensorRT 11.x.

  • Backward compatibility is maintained within a major release. For example, a version-compatible engine built with TensorRT 11.5 runs on TensorRT 11.5 or any later 11.x release, but not on TensorRT 11.0 through 11.4, and not on TensorRT 10.x.

If your Jetson deployment currently uses TensorRT 8.x, you must first migrate to TensorRT 10.x using the 8.x to 10.x appendix, then follow this guide for the 10.x to 11.x migration.

CUDA and JetPack Dependency Changes#

When JetPack adds TensorRT 11.x support, verify that the JetPack release includes a CUDA toolkit compatible with that TensorRT version. Refer to the JetPack release notes and the TensorRT Support Matrix for supported CUDA versions.

TensorRT 11.x drops support for CUDA 11.x. Engines built against CUDA 11.x cannot run on the TensorRT 11.x runtime.

DLA Considerations#

Do not deploy a TensorRT 11.2.1 runtime or TensorRT 11.2.1 engines on JetPack. Continue to use the TensorRT 10.x version supplied for your JetPack release. If your deployment requires DLA, use TensorRT 10.7.

Cross-Compilation#

If you cross-compile TensorRT applications for Jetson on an x86 host, keep the host TensorRT SDK version aligned with the TensorRT version on the target JetPack image. Mixing TensorRT 10.x headers on the host with 11.x libraries on the target (or vice versa) causes compilation or linker errors due to removed APIs. For TensorRT 11.2.1, cross-compilation targets remain on TensorRT 10.x until JetPack adds TensorRT 11.x support.

Recommendation for Edge Deployments#

Tip

For Jetson applications in production on TensorRT 11.2.1, remain on the TensorRT 10.x release supported by your JetPack version. When JetPack adds TensorRT 11.x support, test the migration in a staging environment before deploying to production devices. Verify model accuracy, inference latency, and memory usage against your TensorRT 10.x baseline.

Customers still using TensorRT 10.x should begin migrating to strong typing, explicit quantization, and IPluginV3 to reduce the scope of changes when JetPack adds TensorRT 11.x support. The TensorRT team provides mitigation plans for all removed APIs.