Support Compatibility

Checking Available GPUs

The GPUs available on TensorRT-Cloud are subject to change dynamically as we enable more configurations. To figure out what is currently supported, run:

trt-cloud info

This will produce output similar to:

[I] Available GPUs on Windows:
[I]   RTX2070
[I]   RTX3070
[I]   RTX4090
[I] Available GPUs on Linux:
[I]   A100
[I]   H100

Note

TensorRT-Cloud does not support the full cross-combination of runtime GPUs and OS. For example, in the above output, engines may be built for inference on NVIDIA RTX 4090 on Windows, but not NVIDIA RTX 4090 on Linux.

Planned GPU Support

Over the next few minor releases, we will be adding support for the following platforms:

• NVIDIA GeForce GTX 1630

• NVIDIA GeForce GTX 1660 Ti

• NVIDIA GTX 1650 Super

• NVIDIA GTX 1660 Super

• NVIDIA 2060 SUPER

• NVIDIA 2080 SUPERo

• NVIDIA Quadro RTX 4000

• NVIDIA Quadro RTX 6000

• NVIDIA Quadro RTX 8000

• NVIDIA RTX A2000

• NVIDIA RTX A5000

• NVIDIA RTX A6000

• NVIDIA 4080 SUPER

• NVIDIA RTX 4000 SFF Ada Generation

• NVIDIA RTX 4500 Ada Generation

• NVIDIA RTX 6000 Ada Generation

• NVIDIA Geforce RTX 2060

• NVIDIA Geforce RTX 2080 Ti

• NVIDIA Geforce RTX 3050

• NVIDIA GeForce RTX 3090 Ti

• NVIDIA GeForce RTX 4060 Ti 8GB

• NVIDIA GeForce RTX 4070

If you require any other SKU or GPUs to be supported, contact tensorrt-cloud-contact@nvidia.com.