Support Matrix for NVIDIA NIM for Image OCR (NeMo Retriever OCR)#
This documentation describes the software and hardware that NVIDIA NIM for Image OCR (NeMo Retriever OCR) supports.
System Requirements#
Use a supported NVIDIA GPU system with a current NVIDIA driver and NVIDIA Container Toolkit. For validated data center systems, see the NVIDIA Certified Systems Catalog.
Model#
Publisher |
Model |
Model ID |
Variants |
Model Card |
|---|---|---|---|---|
NVIDIA |
NeMo Retriever OCR v2 |
|
English, multilingual |
The multilingual variant is the default. Set NIM_ENGINE_MODEL_VARIANT=english to serve the English variant.
Supported Hardware and Memory Footprint#
The NIM supports the following validated GPU SKUs & Precision. Where available, the approximate GPU memory sizes are the startup measurements from the latency configuration after forward buffers are allocated. Runtime GPU memory use can increase with larger batches, additional OCR engines, CUDA graph capture shapes, and input-dependent working buffers.
GPU SKU |
GPU |
English approximate GPU memory size (GiB) |
Multilingual approximate GPU memory size (GiB) |
Precision |
|---|---|---|---|---|
NVIDIA-A100-SXM4-80GB |
NVIDIA A100 SXM4 80GB |
1.79 |
2.88 |
FP16 |
NVIDIA-A10G |
NVIDIA A10G |
1.63 |
2.72 |
FP16 |
NVIDIA-B200 |
NVIDIA B200 |
2.17 |
3.26 |
FP16 |
NVIDIA-GB10 |
NVIDIA GB10 Grace Blackwell Superchip, such as NVIDIA DGX Spark systems |
N/A |
N/A |
FP16 |
NVIDIA-GB200 |
NVIDIA GB200 |
2.23 |
3.33 |
FP16 |
NVIDIA-H100-80GB-HBM3 |
NVIDIA H100 80GB HBM3 |
2.07 |
3.17 |
FP16 |
NVIDIA-H100-NVL |
NVIDIA H100 NVL |
2.07 |
3.17 |
FP16 |
NVIDIA-H200 |
NVIDIA H200 |
2.07 |
3.17 |
FP16 |
NVIDIA-L4 |
NVIDIA L4 |
1.57 |
2.66 |
FP16 |
NVIDIA-L40S |
NVIDIA L40S |
1.80 |
2.89 |
FP16 |
NVIDIA-RTX-PRO-6000-Blackwell-Server-Edition |
NVIDIA RTX PRO 6000 Blackwell Server Edition |
1.94 |
3.03 |
FP16 |
Note
GB10 uses unified system memory. GPU memory usage is reported as N/A because the usual VRAM measurements are not accurate for this architecture. For more information, refer to Guidance for Reporting Memory Resources with Unified Memory Architecture.
Throughput mode defaults to two OCR engines and a batch size of 16, which requires more GPU memory than latency mode.
Software#
NVIDIA Driver#
Install a current NVIDIA driver that supports the CUDA version packaged with the NIM. If the container fails to start, verify that the driver and NVIDIA Container Toolkit are current.
NVIDIA Container Toolkit#
Your Docker environment must support NVIDIA GPUs. For installation instructions, see the NVIDIA Container Toolkit documentation.