TensorRT Release Notes
NVIDIA TensorRT is a C++ library that facilitates high performance inference on NVIDIA GPUs. It is designed to work in connection with deep learning frameworks that are commonly used for training. TensorRT focuses specifically on running an already trained network quickly and efficiently on a GPU for the purpose of generating a result; also known as inferencing. These release notes describe the key features, software enhancements and improvements, and known issues for the TensorRT 8.6.1 product package.
For previously released TensorRT documentation, refer to the TensorRT Archives.
1. TensorRT Release 8.x.x
1.1. TensorRT Release 8.6.1
These Release Notes are applicable to workstation, server, and NVIDIA JetPack™ users unless appended specifically with (not applicable for Jetson platforms).
For previously released TensorRT documentation, refer to the NVIDIA TensorRT Archived Documentation.
1.2. TensorRT Release 8.6.0 Early Access (EA)
These Release Notes are applicable to workstation, server, and NVIDIA JetPack™ users unless appended specifically with (not applicable for Jetson platforms).
This EA release is for early testing and feedback. For production use of TensorRT, continue to use TensorRT 8.5.3.
For previously released TensorRT documentation, refer to the NVIDIA TensorRT Archived Documentation.
1.3. TensorRT Release 8.5.3
These Release Notes are applicable to workstation, server, and NVIDIA JetPack™ users unless appended specifically with (not applicable for Jetson platforms).
For previously released TensorRT documentation, refer to the NVIDIA TensorRT Archived Documentation.
Deprecated API Lifetime
Refer to the API documentation (C++, Python) for how to update your code to remove the use of deprecated features.
Announcements
- In the next TensorRT release, cuDNN, cuBLAS, and cuBLASLt tactic sources will be turned off by default in builder profiling. TensorRT plans to remove the cuDNN, cuBLAS, and cuBLASLt dependency in future releases. Use the PreviewFeature flag kDISABLE_EXTERNAL_TACTIC_SOURCES_FOR_CORE_0805 to evaluate the functional and performance impact of disabling cuBLAS and cuDNN and report back to TensorRT if there are critical regressions in your use cases.
- TensorRT Python wheel files before TensorRT 8.5, such as TensorRT 8.4, were published to the NGC PyPI repo. Starting with TensorRT 8.5, Python wheels will instead be published to upstream PyPI. This will make it easier to install TensorRT because it requires no prerequisite steps. Also, the name of the Python package for installation has changed from nvidia-tensorrt to just tensorrt.
- The C++ and Python API documentation in previous releases was included inside the tar file packaging. This release no longer bundles the documentation inside the tar file since the online documentation can be updated post release and avoids encountering mistakes found in stale documentation inside the packages.
1.4. TensorRT Release 8.5.2
These Release Notes are applicable to workstation, server, and NVIDIA JetPack™ users unless appended specifically with (not applicable for Jetson platforms).
For previously released TensorRT documentation, refer to the NVIDIA TensorRT Archived Documentation.
Deprecated API Lifetime
Refer to the API documentation (C++, Python) for how to update your code to remove the use of deprecated features.
Announcements
- In the next TensorRT release, CUDA toolkit 10.2 support will be dropped.
- TensorRT 8.5 will be the last release supporting NVIDIA Kepler (SM 3.x) devices. Support for Maxwell (SM 5.x) devices will be dropped in TensorRT 9.0.
- In the next TensorRT release, cuDNN, cuBLAS, and cuBLASLt tactic sources will be turned off by default in builder profiling. TensorRT plans to remove the cuDNN, cuBLAS, and cuBLASLt dependency in future releases. Use the PreviewFeature flag kDISABLE_EXTERNAL_TACTIC_SOURCES_FOR_CORE_0805 to evaluate the functional and performance impact of disabling cuBLAS and cuDNN and report back to TensorRT if there are critical regressions in your use cases.
- TensorRT Python wheel files before TensorRT 8.5, such as TensorRT 8.4, were published to the NGC PyPI repo. Starting with TensorRT 8.5, Python wheels will instead be published to upstream PyPI. This will make it easier to install TensorRT because it requires no prerequisite steps. Also, the name of the Python package for installation has changed from nvidia-tensorrt to just tensorrt.
- The C++ and Python API documentation in previous releases was included inside the tar file packaging. This release no longer bundles the documentation inside the tar file since the online documentation can be updated post release and avoids encountering mistakes found in stale documentation inside the packages.
1.5. TensorRT Release 8.5.1
These Release Notes are applicable to workstation, server, and NVIDIA JetPack™ users unless appended specifically with (not applicable for Jetson platforms).
For previously released TensorRT documentation, refer to the NVIDIA TensorRT Archived Documentation.
Deprecated API Lifetime
Refer to the API documentation (C++, Python) for how to update your code to remove the use of deprecated features.
Announcements
- In the next TensorRT release, CUDA toolkit 10.2 support will be dropped.
- TensorRT 8.5 will be the last release supporting NVIDIA Kepler (SM 3.x) devices. Support for Maxwell (SM 5.x) devices will be dropped in TensorRT 9.0.
- In the next TensorRT release, cuDNN, cuBLAS, and cuBLASLt tactic sources will be turned off by default in builder profiling. TensorRT plans to remove the cuDNN, cuBLAS, and cuBLASLt dependency in future releases. Use the PreviewFeature flag kDISABLE_EXTERNAL_TACTIC_SOURCES_FOR_CORE_0805 to evaluate the functional and performance impact of disabling cuBLAS and cuDNN and report back to TensorRT if there are critical regressions in your use cases.
- TensorRT Python wheel files before TensorRT 8.5, such as TensorRT 8.4, were published to the NGC PyPI repo. Starting with TensorRT 8.5, Python wheels will instead be published to upstream PyPI. This will make it easier to install TensorRT because it requires no prerequisite steps. Also, the name of the Python package for installation has changed from nvidia-tensorrt to just tensorrt.
- The C++ and Python API documentation in previous releases was included inside the tar file packaging. This release no longer bundles the documentation inside the tar file since the online documentation can be updated post release and avoids encountering mistakes found in stale documentation inside the packages.
1.6. TensorRT Release 8.4.3
These Release Notes are applicable to workstation, server, and NVIDIA JetPack™ users unless appended specifically with (not applicable for Jetson platforms).
For previously released TensorRT documentation, refer to the NVIDIA TensorRT Archived Documentation.
1.7. TensorRT Release 8.4.2
These Release Notes are applicable to workstation, server, and NVIDIA JetPack™ users unless appended specifically with (not applicable for Jetson platforms).
For previously released TensorRT documentation, refer to the NVIDIA TensorRT Archived Documentation.
1.8. TensorRT Release 8.4.1
These Release Notes are applicable to workstation, server, and NVIDIA JetPack™ users unless appended specifically with (not applicable for Jetson platforms).
For previously released TensorRT documentation, refer to the NVIDIA TensorRT Archived Documentation.
1.9. TensorRT Release 8.4.0 Early Access (EA)
These Release Notes are also applicable to workstation, server, and NVIDIA JetPack™ users unless appended specifically with (not applicable for Jetson platforms).
For previously released TensorRT documentation, refer to the NVIDIA TensorRT Archived Documentation.
1.10. TensorRT Release 8.2.5
These Release Notes are also applicable to workstation, server, and NVIDIA JetPack™ users unless appended specifically with (not applicable for Jetson platforms).
For previously released TensorRT documentation, refer to the NVIDIA TensorRT Archived Documentation.
Deprecated API Lifetime
- APIs deprecated before TensorRT 8.0 will be removed in TensorRT 9.0.
- APIs deprecated in TensorRT 8.0 will be retained until at least 8/2022.
- APIs deprecated in TensorRT 8.2 will be retained until at least 11/2022.
Refer to the API documentation (C++, Python) for how to update your code to remove the use of deprecated features.
1.11. TensorRT Release 8.2.4
These Release Notes are also applicable to workstation, server, and NVIDIA JetPack™ users unless appended specifically with (not applicable for Jetson platforms).
For previously released TensorRT documentation, refer to the NVIDIA TensorRT Archived Documentation.
Deprecated API Lifetime
- APIs deprecated before TensorRT 8.0 will be removed in TensorRT 9.0.
- APIs deprecated in TensorRT 8.0 will be retained until at least 8/2022.
- APIs deprecated in TensorRT 8.2 will be retained until at least 11/2022.
Refer to the API documentation (C++, Python) for how to update your code to remove the use of deprecated features.
1.12. TensorRT Release 8.2.3
These release notes are applicable to workstation, server, and NVIDIA JetPack™ users unless appended specifically with (not applicable for Jetson platforms).
This release includes several fixes from the previous TensorRT 8.x.x release as well as the following additional changes. For previous TensorRT documentation, refer to the NVIDIA TensorRT Archived Documentation.
Deprecated API Lifetime
- APIs deprecated before TensorRT 8.0 will be removed in TensorRT 9.0.
- APIs deprecated in TensorRT 8.0 will be retained until at least 8/2022.
- APIs deprecated in TensorRT 8.2 will be retained until at least 11/2022.
Refer to the API documentation (C++, Python) for how to update your code to remove the use of deprecated features.
1.13. TensorRT Release 8.2.2
These release notes are applicable to workstation, server, and NVIDIA JetPack™ users unless appended specifically with (not applicable for Jetson platforms).
This release includes several fixes from the previous TensorRT 8.x.x release as well as the following additional changes. For previous TensorRT documentation, refer to the NVIDIA TensorRT Archived Documentation.
Deprecated API Lifetime
- APIs deprecated before TensorRT 8.0 will be removed in TensorRT 9.0.
- APIs deprecated in TensorRT 8.0 will be retained until at least 8/2022.
- APIs deprecated in TensorRT 8.2 will be retained until at least 11/2022.
Refer to the API documentation (C++, Python) for how to update your code to remove the use of deprecated features.
1.14. TensorRT Release 8.2.1
These release notes are applicable to workstation, server, and NVIDIA JetPack™ users unless appended specifically with (not applicable for Jetson platforms).
This release includes several fixes from the previous TensorRT 8.x.x release as well as the following additional changes. For previous TensorRT documentation, refer to the NVIDIA TensorRT Archived Documentation.
Deprecated API Lifetime
- APIs deprecated prior to TensorRT 8.0 will be removed in TensorRT 9.0.
- APIs deprecated in TensorRT 8.0 will be retained until at least 8/2022.
- APIs deprecated in TensorRT 8.2 will be retained until at least 11/2022.
Refer to the API documentation (C++, Python) for how to update your code to remove the use of deprecated features.
1.15. TensorRT Release 8.2.0 Early Access (EA)
These release notes are applicable to workstation, server, and JetPack users unless appended specifically with (not applicable for Jetson platforms).
This release includes several fixes from the previous TensorRT 8.x.x release as well as the following additional changes. For previous TensorRT documentation, refer to the NVIDIA TensorRT Archived Documentation.
1.16. TensorRT Release 8.0.3
These release notes are applicable to workstation, server, and JetPack users unless appended specifically with (not applicable for Jetson platforms).
This release includes several fixes from the previous TensorRT 8.x.x release as well as the following additional changes. For previous TensorRT documentation, refer to the NVIDIA TensorRT Archived Documentation.
1.17. TensorRT Release 8.0.2
These release notes are applicable to workstation, server, and JetPack users unless appended specifically with (not applicable for Jetson platforms).
This release includes several fixes from the previous TensorRT 8.x.x release as well as the following additional changes. For previous TensorRT documentation, refer to the NVIDIA TensorRT Archived Documentation.
1.18. TensorRT Release 8.0.1
This is the TensorRT 8.0.1 release notes and is applicable to x86 Linux and Windows users, as well as PowerPC and ARM Server Base System Architecture (SBSA) users on Linux only.
These release notes are applicable to workstation, server, and JetPack users unless appended specifically with (not applicable for Jetson platforms).
This release includes several fixes from the previous TensorRT 8.x.x release as well as the following additional changes. For previous TensorRT documentation, refer to the NVIDIA TensorRT Archived Documentation.
1.19. TensorRT Release 8.0.0 Early Access (EA)
This is the TensorRT 8.0.0 Early Access (EA) release notes and is applicable to Linux x86 users.
These release notes are applicable to workstation, server, and JetPack users unless appended specifically with (not applicable for Jetson platforms).
This release includes several fixes from the previous TensorRT 7.x.x release as well as the following additional changes. For previous TensorRT documentation, refer to the NVIDIA TensorRT Archived Documentation.
2. TensorRT Release 7.x.x
2.1. TensorRT Release 7.2.3
This is the TensorRT 7.2.3 GA release notes for Windows and Linux x86 users. For NVIDIA Jetson Linux for Tegra users, TensorRT 7.2.3 is an Early Access (EA) release specifically for MLPerf Inference. For production use of TensorRT, we recommend using the TensorRT 7.1.3 GA.
This release includes several fixes from the previous TensorRT 7.x.x release as well as the following additional changes. For previous TensorRT documentation, refer to the NVIDIA TensorRT Archived Documentation.
Key Features And Enhancements
- Updated the list of supported TensorFlow ops. Refer to Supported Ops for more information.
2.2. TensorRT Release 7.2.2
These are the TensorRT 7.2.2 release notes and are applicable to Windows and Linux x86 users.
These release notes are applicable to workstation, server, and JetPack users unless appended specifically with (not applicable for Jetson platforms).
This release includes several fixes from the previous TensorRT 7.x.x release as well as the following additional changes. For previous TensorRT documentation, refer to the NVIDIA TensorRT Archived Documentation.
2.3. TensorRT Release 7.2.1
These are the TensorRT 7.2.1 release notes and are applicable to Linux x86, Windows x64 and Linux ARM Server Base System Architecture (SBSA) users.
These release notes are applicable to workstation, server, and JetPack users unless appended specifically with (not applicable for Jetson platforms).
This release includes several fixes from the previous TensorRT 7.x.x release as well as the following additional changes. For previous TensorRT documentation, refer to the NVIDIA TensorRT Archived Documentation.
2.4. TensorRT Release 7.2.0
This is the TensorRT 7.2.0 release notes. We recommend PowerPC users download the TensorRT 7.2.0 build for production use. For Linux and JetPack users, TensorRT 7.2.0 is a Release Candidate (RC). As an RC release, this is a Preview for early testing and feedback. For production use of TensorRT for Linux and JetPack users, we recommend downloading TensorRT 7.1.3. The RC release is subject to change based on ongoing performance tuning and functional testing. For feedback, submit a bug on the NVIDIA Developer website.
These release notes are applicable to workstation, server, and JetPack users unless appended specifically with (not applicable for Jetson platforms).
This release includes several fixes from the previous TensorRT 7.x.x release as well as the following additional changes. For previous TensorRT documentation, refer to the NVIDIA TensorRT Archived Documentation.
Key Features And Enhancements
Compatibility
- TensorRT 7.2.0 has been tested with the following:
- This TensorRT release supports CUDA 10.2 for Jetson and 11.0 update 1 for x86 and PowerPC.