Is this page helpful?

Release Notes for NVIDIA NIM for Image OCR (NeMo Retriever OCR)#

This documentation contains the release notes for NVIDIA NIM for Image OCR (NeMo Retriever OCR).

Release 2.0.1#

This patch release includes the following updates:

  • Adds the com.nvidia.nim.model_download_protocol=native-v1 container label for NIM Operator deployment compatibility (requires NIM Operator >= 3.1.2).

  • The non-default RUST_LOG configuration is removed so that logging is controlled by the product default or explicit user configuration.

Release 2.0.0#

This release represents a major runtime upgrade for the nvidia/nemotron-ocr-v2 NIM that includes a new purpose-built CUDA OCR runtime. The release supports English and multilingual OCR variants and provides the choice of both latency-optimized and throughput-optimized runtime configurations.

Highlights#

This release contains the following key changes:

  • Changed the OCR inference endpoint from /v1/infer to /v1/ocr.

  • Updated the GenAI-Perf benchmarking example to use the /v1/ocr endpoint.

  • Replaced model-profile selection with explicit runtime settings. No manual profile selection steps are required.

  • Added new environment variables. For details, refer to Environment Variables for NVIDIA NIM for Image OCR (NeMo Retriever OCR).

    • Added NIM_PERFORMANCE_MODE — Supports 0 (latency-oriented default) or 1 (throughput-oriented).

    • Added NIM_ENGINE_MODEL_VARIANT — Supports english or multilingual (default).

    • Added controls for OCR engine count, batching, CUDA graph capture, stage-specific warmup, GPU preprocessing, and recognizer memory optimization.

    • Added Hugging Face and NGC model download providers.

    • Added model-download-only and precompile-only startup modes for staging artifacts and precompiled CUDA kernels.

    • Added optional KServe V2 gRPC, HTTPS, queue, timeout, and structured logging configuration.

Support Matrix Updates#

For details, refer to Support Matrix for NVIDIA NIM for Image OCR (NeMo Retriever OCR).

Compatibility Updates#

Most environment variables beginning with NIM_TRITON_ are not supported by the 2.0.0 runtime. The runtime emits a deprecation warning when an alias is used. For new deployments, update your code to use the current environment variables.

The following deprecated compatibility aliases remain available for migration.

Environment variable

Deprecated compatibility aliases

NIM_SERVER_MAX_WAIT_MS

NIM_TRITON_DYNAMIC_BATCHING_MAX_QUEUE_DELAY_MICROSECONDS

NIM_ENGINE_COUNT

NIM_TRITON_MODEL_INSTANCE_COUNT

NIM_PIPELINE_MAX_BATCH_SIZE

NIM_TRITON_DATA_MAX_BATCH_SIZE, NIM_TRITON_MAX_BATCH_SIZE, NIM_TRITON_MODEL_MAX_BATCH_SIZE

NIM_SERVER_MAX_QUEUE_SIZE

NIM_TRITON_MAX_QUEUE_SIZE, NIM_TRITON_DYNAMIC_BATCHING_MAX_QUEUE_SIZE

NIM_SERVER_REQUEST_TIMEOUT_S

NIM_TRITON_REQUEST_TIMEOUT

NIM_PIPELINE_TIMING_TELEMETRY

NIM_TRITON_ENABLE_PIPELINE_TIMING

Note

Compatibility alias values are parsed by using the units of the current environment variable. In particular, the value of NIM_TRITON_DYNAMIC_BATCHING_MAX_QUEUE_DELAY_MICROSECONDS is interpreted as milliseconds by the 2.0.0 runtime.

All Known Issues#

The known issues for NeMo Retriever OCR are the following:

  • Helm nim.modelName is not propagated. Use the appropriate runtime model-selection environment variable directly. For OCR, use envVars.NIM_ENGINE_MODEL_VARIANT.

  • Use persistence.storageClass not persistence.class.

  • Do not combine persistence.enabled=true with hostPath.enabled=true. Currently, the chart mounts the PVC and ignores hostPath.

  • Performance benchmarking has not yet completed.

Previous Versions#