Support Matrix#
This documentation describes the software and hardware that Riva NMT NIM supports.
Hardware#
NVIDIA Riva NMT NIM is supported on NVIDIA GPUs with Compute Capability > 7.0. Avoid exceeding the available memory when selecting models to deploy; 16+ GB VRAM is recommended.
GPUs Supported#
GPU |
Precision |
|---|---|
A30, A100 |
FP16 |
H100 |
FP16 |
A2, A10, A16, A40 |
FP16 |
L4, L40, GeForce RTX 40xx |
FP16 |
GeForce RTX 50xx |
FP16 |
Software#
Linux operating systems (Ubuntu 22.04 or later recommended)
NVIDIA Driver >= 535
NVIDIA Docker >= 23.0.1
Supported Models#
Riva NMT NIM supports the following models.
NIM automatically downloads the prebuilt model if it is available on the target GPU (GPUs with Compute Capability >= 8.0) or generates an optimized model on-the-fly using RMIR model on other GPUs (Compute Capability > 7.0).
Model |
Publisher |
WSL Support |
|---|---|---|
NVIDIA |
❌ |
Note
All models use FP16 precision.
Riva Translate 1.6b#
Riva Translate 1.6b is a Megatron model designed to translate text between language pairs (from one language to another).
To use this model, set CONTAINER_ID to riva-translate-1_6b. Refer to Launching the NIM for details.
Model Profiles#
Model Profile |
CPU Memory (GB) |
GPU Memory (GB) |
|---|---|---|
|
4.6 |
9.5 |
Supported Languages#
The Riva Translate 1.6b model supports translation for any-to-any language pair from below 36 languages.
Simplified Chinese (
zh-CN)Traditional Chinese (
zh-TW)Russian (
ru)German (
de)European Spanish (
es-ES)LATAM Spanish (
es-US)French (
fr)Danish (
da)Greek (
el)Finnish (
fi)Hungarian (
hu)Italian (
it)Lithuanian (
lt)Latvian (
lv)Dutch (
nl)Norwegian (
no)Polish (
pl)European Portuguese (
pt-PT)Brazilian Portuguese (
pt-BR)Romanian (
ro)Slovak (
sk)Swedish (
sv)Japanese (
ja)Hindi (
hi)Korean (
ko)Estonian (
et)Slovenian (
sl)Bulgarian (
bg)Ukrainian (
uk)Croatian (
hr)Arabic (
ar)Vietnamese (
vi)Turkish (
tr)Indonesian (
id)Czech (
cs)Thai (
th)