Support Matrix#
These support matrices provide an overview of the supported platforms, features, and hardware capabilities of the TensorRT APIs, parsers, and layers.
Hardware and Precision#
The following table lists NVIDIA hardware and the precision modes that each hardware supports.
GPU Architecture |
Example Devices |
FP32 |
FP16 |
BF16 |
INT8 WoQ GEMM |
INT4 WoQ GEMM [2] |
FP8 GEMM |
FP4 GEMM |
|
|---|---|---|---|---|---|---|---|---|---|
12.0 |
NVIDIA Blackwell |
NVIDIA RTX 5090 |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
8.9 |
NVIDIA Ada |
NVIDIA RTX 4090 |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
No |
8.6 |
NVIDIA Ampere |
NVIDIA RTX 3090 |
Yes |
Yes |
Yes |
Yes |
Yes |
No |
No |
7.5 |
NVIDIA Turing |
NVIDIA RTX 2080Ti |
Yes [1] |
Yes |
No |
No |
No |
No |
No |
Note
Engines built with TensorRT-RTX are by default portable across devices with compute capabilities 8.6, 8.9, and 12.0, if the model precisions are supported on the target architecture.
By default, TensorRT-RTX does not create portable engines that are compatible with Turing when targeting other compute capabilities, as this can impact performance. Instead of relying on multi-device portable engines that include Turing support, developers are encouraged to build optimized, device-specific engines for Turing using appropriate API flags.
Compute Capability Per Platform#
The section lists the supported compute capability based on the platform.
Compute Capability |
Platform |
|
|---|---|---|
Linux x86-64 |
Windows x64 |
|
12.0 |
Yes |
Yes |
8.9 |
Yes |
Yes |
8.6 |
Yes |
Yes |
7.5 |
Yes |
Yes |
Software Versions Per Platform#
Platform |
Compiler Version |
Tested CUDA Version |
TensorRT-RTX Python API [3] |
TensorRT-RTX C++ API |
|---|---|---|---|---|
|
Yes |
Yes |
||
|
Yes |
Yes |
Footnotes