Is this page helpful?

Environment Variables for NVIDIA NIM for Image OCR (NeMo Retriever OCR)#

Use this documentation to configure NVIDIA NIM for Image OCR (NeMo Retriever OCR).

Binary Environment Variables#

Name	Default	Description
`LOG_FORMAT`	`pretty`	The log output format. One of: `pretty`, `json`, `compact`.
`NIM_PERFORMANCE_MODE`	`false`	True to use throughput-oriented values for unset server and pipeline variables; false to use latency-oriented values. For details, see Select Latency or Throughput Mode.
`RUST_LOG`	`error`	The tracing filter. One of: `error`, `info`.
`SHOW_CONFIG`	`false`	True to print environment-variable help and exit.

Server Environment Variables#

Name	Default	Description
`NIM_SERVER_BIND_ADDR`	`0.0.0.0:8000`	The HTTP listen address (host:port).
`NIM_SERVER_GRPC_BIND_ADDR`	-	The optional KServe V2 gRPC listen address (host:port). Empty disables gRPC.
`NIM_SERVER_GRPC_MAX_DECODING_MESSAGE_BYTES`	-	The maximum inbound KServe gRPC message size in bytes. Empty uses the effective HTTP body limit.
`NIM_SERVER_MAX_QUEUE_SIZE`	`1024`	The maximum number of requests in the batcher queue.
`NIM_SERVER_MAX_WAIT_MS`	`0`	The maximum time in milliseconds to accumulate requests before dispatching a batch. Throughput mode changes the default to `10`.
`NIM_SERVER_REQUEST_TIMEOUT_S`	`120`	The request timeout in seconds.
`NIM_SERVER_TLS_CERT_PATH`	-	The path to a PEM certificate chain for HTTPS. When set with `NIM_SERVER_TLS_KEY_PATH`, enables TLS. The NIM validates both files before model initialization.
`NIM_SERVER_TLS_KEY_PATH`	-	The path to a PEM private key for HTTPS. Must be set together with with `NIM_SERVER_TLS_CERT_PATH`. The NIM fails to start if either file is missing or invalid.

Pipeline Environment Variables#

Name	Default	Description
`NIM_ENGINE_COUNT`	`1`	The number of parallel OCR engine instances. Throughput mode changes the default to `2`.
`NIM_PIPELINE_MAX_BATCH_SIZE`	`1`	The maximum images per engine. Throughput mode changes the default to `16`.
`NIM_PIPELINE_OCR_EMPTY_RESULT_PADDED_RETRY`	`false`	True to enable one padded retry when the first OCR pass returns no text detections.
`NIM_PIPELINE_TIMING_TELEMETRY`	-	True to enable pipeline timing telemetry.
`NIM_PIPELINE_USE_GPU_PREPROCESS`	`true`	True to use GPU image preprocessing.
`NIM_PIPELINE_VRAM_TELEMETRY`	-	True to enable VRAM telemetry.
`NIM_SERVER_MODE`	`latency`	The server scheduling mode. One of: `latency`, `throughput`. Throughput mode changes the default to `throughput`.

Engine Environment Variables#

Name	Default	Description
`HF_TOKEN`	-	The Hugging Face token for model download when `NIM_ENGINE_MODEL_DOWNLOAD_PROVIDER=hf`. Passed through to the model downloader. For details, refer to Get Started With NVIDIA NIM for Image OCR (NeMo Retriever OCR).
`NGC_API_KEY`	-	The NGC API key for model download when `NIM_ENGINE_MODEL_DOWNLOAD_PROVIDER=ngc`. Passed through to the model downloader. For details, refer to Get Started With NVIDIA NIM for Image OCR (NeMo Retriever OCR).
`NIM_ENGINE_CUDA_GRAPH_CAPTURE`	`full`	The default CUDA graph capture scope for OCR stages that inherit it. One of: `full`, `warmup`, `none`.
`NIM_ENGINE_CUDA_GRAPH_CAPTURE_SHAPES`	-	A comma-separated list of OCR detector batch sizes for CUDA graph capture.
`NIM_ENGINE_DEVICES`	-	The CUDA-visible device ordinal for OCR engine placement. If empty, the NIM uses visible device `0`.
`NIM_ENGINE_MODEL_DOWNLOAD_ONLY`	`false`	True to download model artifacts and exit before CUDA warmup or server startup. Requires `HF_TOKEN` when `NIM_ENGINE_MODEL_DOWNLOAD_PROVIDER=hf`, or `NGC_API_KEY` when `NIM_ENGINE_MODEL_DOWNLOAD_PROVIDER=ngc`.
`NIM_ENGINE_MODEL_DOWNLOAD_PROVIDER`	`auto`	The model download provider. One of: `auto`, `ngc`, `hf`, `huggingface`, `hugging-face`. For `auto`, the runtime selects Hugging Face if `HF_TOKEN` is set or NGC if `NGC_API_KEY` is set. If both credentials are set, Hugging Face takes precedence. If neither credential is set, and the model artifacts are not already present, model download fails.
`NIM_ENGINE_MODEL_NAME`	`nvidia/nemotron-ocr-v2`	The model identifier for Hugging Face or NGC.
`NIM_ENGINE_MODEL_PATH`	`/model/ocr`	The model directory in the container.
`NIM_ENGINE_MODEL_VARIANT`	`multilingual`	The OCR model variant. One of: `english`, `multilingual`. Starting in release 2.0.0, the earlier `NIM_OCR_MODEL_VERSION` variable is not supported.
`NIM_ENGINE_OCR_DETECTOR_CUDA_GRAPH_CAPTURE`	`inherit`	The detector CUDA graph capture override. One of: `inherit`, `full`, `partial`, `max`, `warmup`, `none`.
`NIM_ENGINE_OCR_DETECTOR_WARMUP`	`inherit`	Detector warmup override. Supported values are `inherit`, `full`, `partial`, `max`, and `none`.
`NIM_ENGINE_OCR_RECOGNIZER_CUDA_GRAPH_CAPTURE`	`none`	The recognizer CUDA graph capture mode for region chunks. One of: `inherit`, `full`, `partial`, `max`, `warmup`, `none`.
`NIM_ENGINE_OCR_RECOGNIZER_CUDA_GRAPH_CAPTURE_SHAPES`	-	A comma-separated list of recognizer region chunk sizes for CUDA graph capture.
`NIM_ENGINE_OCR_RECOGNIZER_LOGITS_ELISION`	`true`	True to decode recognizer chunks directly into reduced token outputs instead of materializing global logits.
`NIM_ENGINE_OCR_RECOGNIZER_WARMUP`	`none`	The recognizer warmup mode for region chunks. One of: `inherit`, `full`, `partial`, `max`, `none`.
`NIM_ENGINE_OCR_RECOGNIZER_WARMUP_SHAPES`	-	A comma-separated list of recognizer region chunk sizes that override recognizer warmup.
`NIM_ENGINE_OCR_RELATIONAL_CUDA_GRAPH_CAPTURE`	`none`	The relational transformer CUDA graph capture override. One of: `inherit`, `full`, `warmup`, `none`.
`NIM_ENGINE_OCR_RELATIONAL_WARMUP`	`none`	The relational transformer warmup override. One of: `inherit`, `full`, `none`.
`NIM_ENGINE_PRECISION`	`fp16`	The inference precision. The OCR runtime supports only `fp16`.
`NIM_ENGINE_PRECOMPILE_CACHE_DIR`	-	The root directory for precompiled CUDA artifacts.
`NIM_ENGINE_PRECOMPILE_ONLY`	`false`	True to compile and cache CUDA artifacts, then exit without starting the HTTP server.
`NIM_ENGINE_WARMUP_MODE`	`full`	The warmup scope for enabled OCR stages. One of: `full`, `partial`, `none`.
`NIM_ENGINE_WARMUP_SHAPES`	-	A comma-separated list of OCR detector batch sizes that override detector warmup.