Support Compatibility
Checking Available GPUs
The GPUs available on TensorRT-Cloud are subject to change dynamically as we enable more configurations. To figure out what is currently supported, run:
trt-cloud info
This will produce output similar to:
[I] OS GPU Tags Command [I] ------- --------- ------ ---------------------------- [I] Linux A100 --os=linux --gpu=A100 [I] Linux H100 --os=linux --gpu=H100 [I] Windows RTX2070 --os=windows --gpu=RTX2070 [I] Windows RTX3070 --os=windows --gpu=RTX3070 [I] Windows RTX4060TI --os=windows --gpu=RTX4060TI [I] Windows RTX4070 --os=windows --gpu=RTX4070 [I] Windows RTX4090 --os=windows --gpu=RTX4090
Note
TensorRT-Cloud does not support the full cross-combination of runtime GPUs and OS. For example, in the above output, engines may be built for inference on NVIDIA RTX 4090 on Windows, but not NVIDIA RTX 4090 on Linux.
Planned GPU Support
Over the next few minor releases, we will be adding support for the following platforms:
NVIDIA GeForce GTX 1630
NVIDIA GeForce GTX 1660 Ti
NVIDIA GTX 1650 Super
NVIDIA GTX 1660 Super
NVIDIA 2060 SUPER
NVIDIA 2080 SUPERo
NVIDIA Quadro RTX 4000
NVIDIA Quadro RTX 6000
NVIDIA Quadro RTX 8000
NVIDIA RTX A2000
NVIDIA RTX A5000
NVIDIA RTX A6000
NVIDIA 4080 SUPER
NVIDIA RTX 4000 SFF Ada Generation
NVIDIA RTX 4500 Ada Generation
NVIDIA RTX 6000 Ada Generation
NVIDIA Geforce RTX 2060
NVIDIA Geforce RTX 2080 Ti
NVIDIA Geforce RTX 3050
NVIDIA GeForce RTX 3090 Ti
NVIDIA GeForce RTX 4060 Ti 8GB
NVIDIA GeForce RTX 4070
If you require any other SKU or GPUs to be supported, contact tensorrt-cloud-contact@nvidia.com.