Is this page helpful?

Environment Variables for NVIDIA NIM for Nemotron Parse#

Use this documentation to configure NVIDIA NIM for Nemotron Parse.

Set SHOW_CONFIG=1 when you launch the container to print the user-facing environment-variable help and exit before model loading.

Binary Environment Variables#

Unless otherwise stated, boolean variables accept 1, true, or TRUE to enable a setting, and 0, false, or FALSE to disable it. Use SHOW_CONFIG=1 for the help command. Invalid boolean values and integers outside the stated ranges cause startup validation to fail.

Name	Default	Description
`LOG_FORMAT`	`pretty`	The log output format. One of: `pretty`, `json`, `compact`.
`NIM_PERFORMANCE_MODE`	`false`	Changes the default decoder slot capacity from `1` to `128` when enabled and `NIM_PIPELINE_MAX_BATCH_SIZE` is unset. An explicit capacity takes precedence.
`RUST_LOG`	`error`	The tracing filter. For example, `info`, `debug`, or a per-module filter such as `parse=debug,tower=error`.
`SHOW_CONFIG`	`0`	Set to `1` to print user-facing environment-variable help and exit without loading the model or starting the server.

Server Environment Variables#

Name	Default	Description
`NIM_SERVER_BIND_ADDR`	`0.0.0.0:8000`	The HTTP listen address (host:port).
`NIM_SERVER_FAIL_FAST`	`false`	True to reject new requests when reserved and active work reaches the decoder slot capacity set by `NIM_PIPELINE_MAX_BATCH_SIZE`. This can occur before the queue is full. Returns HTTP `503` or gRPC `RESOURCE_EXHAUSTED`.
`NIM_SERVER_GRPC_BIND_ADDR`	unset	The optional KServe V2 gRPC listen address (host:port), such as `0.0.0.0:8001`. Unset or empty disables gRPC. Use a different port from HTTP and publish that port when using Docker.
`NIM_SERVER_GRPC_MAX_DECODING_MESSAGE_BYTES`	unset	The maximum inbound gRPC message size in bytes. A positive integer sets the limit. Unset, `0`, or a value that cannot be parsed uses `104857600` bytes (100 MiB). The limit includes message overhead and does not change the HTTP body limit.
`NIM_SERVER_MAX_QUEUE_SIZE`	`1024`	The maximum number of requests waiting in the shared runtime queue. Supported range: `1` through `1048576`. A full queue returns HTTP `503` or gRPC `RESOURCE_EXHAUSTED`. Increasing queue depth does not increase decoder capacity.
`NIM_SERVER_REQUEST_TIMEOUT_S`	`3600`	The work deadline in seconds, including queue wait and inference. Supported range: `1` through `3600`. HTTP upload and multipart parsing occur before the deadline starts. Expiry returns HTTP `504` or gRPC `DEADLINE_EXCEEDED`.
`NIM_SERVER_TLS_CERT_PATH`	unset	The path to a readable PEM certificate chain for HTTPS. Must be set together with `NIM_SERVER_TLS_KEY_PATH`. Both files are validated before model loading. These settings apply to HTTP only; gRPC uses plaintext.
`NIM_SERVER_TLS_KEY_PATH`	unset	The path to a readable PEM private key for HTTPS. Must be set together with `NIM_SERVER_TLS_CERT_PATH`. A partial pair, missing or invalid file, or incompatible certificate/key pair causes startup to fail. Leave both TLS variables unset to use HTTP.

Pipeline Environment Variables#

Name	Default	Description
`NIM_PIPELINE_MAX_BATCH_SIZE`	`1`	The maximum number of concurrent consumer decoder slots. Supported range: `1` through `256`. When unset, the default is `128` with `NIM_PERFORMANCE_MODE=1` and `1` otherwise. An explicit value overrides performance mode. Higher capacity uses more GPU memory.
`NIM_PIPELINE_TIMING_TELEMETRY`	unset	`2` to enable diagnostic timing logs for telemetry. Unset, `0`, or `1` leaves detailed timing disabled. Requires `RUST_LOG=info`. Diagnostic mode can affect performance.
`NIM_PIPELINE_VRAM_TELEMETRY`	`0`	`1` to enable DEBUG logs of process GPU-memory usage after each decoder iteration; `0` to disable. Also requires `RUST_LOG=debug`. Startup memory checkpoints are emitted separately regardless of this setting. Refer to Observability for examples.

Engine Environment Variables#

Name	Default	Description
`HF_TOKEN`	unset	The Hugging Face access token used to download the supported checkpoint from `nvidia/Nemotron-3.5-Parse`. A complete, verified local checkpoint does not require a token. For details, refer to Model Download Providers.
`NGC_API_KEY`	unset	The NGC API key used to download the supported checkpoint from NGC. A complete, verified local checkpoint does not require a key. For details, refer to Model Download Providers.
`NIM_ENGINE_MODEL_DOWNLOAD_ONLY`	`0`	Set to `1` to download and verify the model files, then exit before CUDA warmup or server startup. Use this setting to stage weights for an offline deployment.
`NIM_ENGINE_MODEL_DOWNLOAD_PROVIDER`	`auto`	The model download provider. One of: `auto`, `ngc`, `hf`, `huggingface`, or `hugging-face`. With `auto`, the NIM uses Hugging Face when `HF_TOKEN` is set, and otherwise uses NGC when `NGC_API_KEY` is set. For details, refer to Model Download Providers.
`NIM_ENGINE_MODEL_PATH`	`/model/parse`	The checkpoint directory in the container. The NIM verifies existing files and downloads missing files here. It requires the supported `config.json`, `model.safetensors`, and `tokenizer.json`; incompatible nonempty files cause startup to fail. Refer to Get Started for the pinned checkpoint and staging instructions.
`NIM_ENGINE_PRECOMPILE_CACHE_DIR`	`/opt/cache`	The root directory for persistent CUDA artifacts. Mount a writable directory here to reuse compiled artifacts. An empty value disables persistent caching.
`NIM_ENGINE_PRECOMPILE_ONLY`	`0`	Set to `1` to warm CUDA artifacts and exit before opening the server listeners. Requires a GPU and inference warmup, which is enabled by default.