NVIDIA NMT NIM Support Matrix#
This page describes the software and hardware that the NVIDIA NMT NIM supports.
Hardware#
The NVIDIA NMT NIM requires an NVIDIA GPU with a Compute Capability of 8.0 or higher and at least 16 GB of VRAM. Ensure that the models you deploy do not exceed the available GPU memory.
GPU |
Precision |
|---|---|
A30, A100 |
FP16 |
H100 |
FP16 |
A2, A10, A16, A40 |
FP16 |
L4, L40, GeForce RTX 40xx |
FP16 |
GeForce RTX 50xx |
FP16 |
Software#
Linux operating systems (Ubuntu 22.04 or later recommended)
NVIDIA Driver >= 535
NVIDIA Docker >= 23.0.1
Supported Models#
The NVIDIA NMT NIM microservice supports the following Nemotron NMT model.
The microservice requires an NVIDIA GPU with Compute Capability >= 8.0. It automatically downloads a pre-built model when available for the target GPU or generates an optimized model on-the-fly using the RMIR model.
Model |
Publisher |
WSL Support |
|---|---|---|
NVIDIA |
❌ |
Note
All models use FP16 precision.
Riva Translate 1.6b#
Riva Translate 1.6b is a Megatron model designed to translate text between language pairs (from one language to another).
To use this model, set CONTAINER_ID to riva-translate-1_6b. Refer to Step 1: Deploy the NMT NIM Microservice for details.
Model Profiles#
Model Profile |
CPU Memory (GB) |
GPU Memory (GB) |
|---|---|---|
|
4.6 |
9.5 |
Supported Languages#
The Riva Translate 1.6b model supports translation for any-to-any language pair from below 36 languages.
Simplified Chinese (
zh-CN)Traditional Chinese (
zh-TW)Russian (
ru)German (
de)European Spanish (
es-ES)LATAM Spanish (
es-US)French (
fr)Danish (
da)Greek (
el)Finnish (
fi)Hungarian (
hu)Italian (
it)Lithuanian (
lt)Latvian (
lv)Dutch (
nl)Norwegian (
no)Polish (
pl)European Portuguese (
pt-PT)Brazilian Portuguese (
pt-BR)Romanian (
ro)Slovak (
sk)Swedish (
sv)Japanese (
ja)Hindi (
hi)Korean (
ko)Estonian (
et)Slovenian (
sl)Bulgarian (
bg)Ukrainian (
uk)Croatian (
hr)Arabic (
ar)Vietnamese (
vi)Turkish (
tr)Indonesian (
id)Czech (
cs)Thai (
th)