Support Matrix#
Hardware#
NVIDIA NIM for NV-CLIP runs on any supported NVIDIA GPU. For further information, refer to the Supported Models section.
Software#
Linux operating systems (Ubuntu 22.04 or later recommended)
NVIDIA Driver >= 535
NVIDIA Docker >= 23.0.1
Supported Models#
These models are optimized using TRT and are available as pre-built, optimized engines on NGC and should use the Embeddings Endpoint.
NV-CLIP NIM#
Optimized Configurations#
The GPU Memory and Disk Space values are in GB; Disk Space is for both the container and the model; Profile is for what the model is optimized.
GPU |
GPU Memory |
Precision |
# of GPUs |
Disk Space |
|---|---|---|---|---|
H100 SXM |
80 |
FP16 |
1 |
19.3 |
H100 NVL |
80 |
FP16 |
1 |
19.3 |
H100 PCIe |
80 |
FP16 |
1 |
19.3 |
A100 SXM |
80 |
FP16 |
1 |
19.3 |
A100 PCIe |
80 |
FP16 |
1 |
19.3 |
L40S |
48 |
FP16 |
1 |
19.3 |
L4 |
48 |
FP16 |
1 |
19.3 |
A10G |
24 |
FP16 |
1 |
19.3 |
A6000 Ada |
48 |
FP16 |
1 |
19.3 |
RTX 4090 |
24 |
FP16 |
1 |
19.3 |
RTX 5080-WSL (Beta) |
16 |
FP16 |
1 |
19.3 |
RTX 5090-WSL (Beta) |
32 |
FP16 |
1 |
19.3 |
GH 200 |
480 |
FP16 |
1 |
11.5 |
Non-optimized Configuration#
NV-CLIP NIM supports non-optimized configurations on GPUs other than these listed.