Environment Variables for NVIDIA NIM for Object Detection#

Use this documentation to learn about the environment variables for NVIDIA NIM for Object Detection.

To print environment-variable help, including defaults and currently set values, and exit without starting the server, run the container with SHOW_CONFIG=1.

Binary Variables#

Variable	Default	Description
`LOG_FORMAT`	`pretty`	The log output format. Supported values are `pretty`, `json`, and `compact`.
`NIM_PERFORMANCE_MODE`	`false`	Selects throughput-oriented values when enabled and latency-oriented values when disabled for unset server and pipeline variables.
`RUST_LOG`	`error`	Sets the tracing log filter.
`SHOW_CONFIG`	`false`	When enabled, prints environment-variable help and exits.

The unset or false value is equivalent to NIM_PERFORMANCE_MODE=0. When NIM_PERFORMANCE_MODE=1, unset defaults change as follows.

Variable	`NIM_PERFORMANCE_MODE=0`	`NIM_PERFORMANCE_MODE=1`
`NIM_SERVER_MAX_WAIT_MS`	`0`	`10`
`NIM_ENGINE_COUNT`	`1`	`2`
`NIM_PIPELINE_MAX_BATCH_SIZE`	`1`	`16`
`NIM_SERVER_MODE`	`latency`	`throughput`

Explicitly set variables are not overridden by NIM_PERFORMANCE_MODE.

Server Variables#

Variable	Default	Description
`NIM_SERVER_BIND_ADDR`	`0.0.0.0:8000`	The HTTP listen address in `host:port` format.
`NIM_SERVER_GRPC_BIND_ADDR`	unset	The optional KServe V2 gRPC listen address in `host:port` format. Empty disables gRPC.
`NIM_SERVER_GRPC_MAX_DECODING_MESSAGE_BYTES`	unset	The maximum inbound KServe gRPC message size in bytes. Empty uses the effective HTTP body limit.
`NIM_SERVER_MAX_QUEUE_SIZE`	`1024`	The maximum number of requests in the batcher queue.
`NIM_SERVER_MAX_WAIT_MS`	`0`	The maximum time in milliseconds to accumulate requests before dispatching a batch. `NIM_PERFORMANCE_MODE=1` changes the default to `10`.
`NIM_SERVER_REQUEST_TIMEOUT_S`	`120`	The per-request timeout in seconds.
`NIM_SERVER_TLS_CERT_PATH`	unset	The path to a PEM certificate chain for HTTPS. Must be set together with `NIM_SERVER_TLS_KEY_PATH`. The NIM validates both files before model initialization.
`NIM_SERVER_TLS_KEY_PATH`	unset	The path to a PEM private key for HTTPS. Must be set together with `NIM_SERVER_TLS_CERT_PATH`. The NIM fails to start if either file is missing or invalid.

Pipeline Variables#

Variable	Default	Description
`NIM_ENGINE_COUNT`	`1`	The number of engine replicas for the selected object detection model. `NIM_PERFORMANCE_MODE=1` changes the default to `2`.
`NIM_PIPELINE_MAX_BATCH_SIZE`	`1`	The maximum number of images per engine forward pass. Supported values are `1` through `255`. This is the primary VRAM control. `NIM_PERFORMANCE_MODE=1` changes the default to `16`.
`NIM_PIPELINE_TIMING_TELEMETRY`	unset	Enables pipeline timing telemetry when set to true.
`NIM_PIPELINE_USE_GPU_PREPROCESS`	`1`	Enables GPU image preprocessing when set to true.
`NIM_PIPELINE_VRAM_TELEMETRY`	unset	Enables VRAM telemetry when set to true.
`NIM_SERVER_MODE`	`latency`	The server scheduling mode. Supported values are `latency` and `throughput`. `NIM_PERFORMANCE_MODE=1` changes the default to `throughput`.

Engine Variables#

Variable	Default	Description
`HF_TOKEN`	unset	Token required for Hugging Face model download. When the provider is `auto`, this credential causes the runtime to select Hugging Face.
`NGC_API_KEY`	unset	API key required for NGC model download. When the provider is `auto`, this credential causes the runtime to select NGC if `HF_TOKEN` is not set.
`NIM_ENGINE_CUDA_GRAPH_CAPTURE`	unset	CUDA graph capture mode. When unset, the runtime falls back to `warmup`. Supported values are `full`, `warmup`, and `none`.
`NIM_ENGINE_CUDA_GRAPH_CAPTURE_SHAPES`	unset	Comma-separated batch sizes for CUDA graph capture. Overrides `NIM_ENGINE_CUDA_GRAPH_CAPTURE`.
`NIM_ENGINE_DEVICES`	unset	CUDA-visible device ordinal used by the runtime. Unset uses visible device `0`.
`NIM_ENGINE_MODEL_DOWNLOAD_ONLY`	`false`	Downloads model artifacts and exits before CUDA warmup or server startup.
`NIM_ENGINE_MODEL_DOWNLOAD_PROVIDER`	`auto`	Model download provider. Supported values are `auto`, `hf`, `huggingface`, `hugging-face`, and `ngc`. The values `hf`, `huggingface`, and `hugging-face` explicitly select Hugging Face. The value `ngc` explicitly selects NGC.
`NIM_ENGINE_MODEL_NAME`	`nvidia/nemotron-page-elements-v3`	Model ID to load. The NIM container loads this Page Elements model by default. To load Table Structure, set this variable to `nvidia/nemotron-table-structure-v1` and set `NIM_ENGINE_MODEL_PATH=/model/table-structure`. This value determines class labels and the postprocessing variant.
`NIM_ENGINE_MODEL_PATH`	`/model/page-elements`	In-container model artifact path for the selected model. The NIM container uses this Page Elements path by default. To switch models, set this variable explicitly together with `NIM_ENGINE_MODEL_NAME` (for Table Structure, use `/model/table-structure`).
`NIM_ENGINE_PRECISION`	`fp16`	Inference precision. Use `fp16` for supported deployments.
`NIM_ENGINE_PRECOMPILE_CACHE_DIR`	unset	Root directory for precompiled cuDNN plans and kernel cubins.
`NIM_ENGINE_PRECOMPILE_ONLY`	`0`	Precompiles runtime artifacts and exits before the server starts.
`NIM_ENGINE_WARMUP_MODE`	unset	Startup warmup mode. When unset, the runtime falls back to `full`. Supported values are `full`, `partial`, and `none`.
`NIM_ENGINE_WARMUP_SHAPES`	unset	Comma-separated batch sizes to warm up. Overrides `NIM_ENGINE_WARMUP_MODE`.

With NIM_ENGINE_MODEL_DOWNLOAD_PROVIDER=auto, the runtime selects Hugging Face when HF_TOKEN is set. Otherwise, it selects NGC when NGC_API_KEY is set. If both credentials are set, Hugging Face takes precedence. If neither credential is set and the model artifacts are not already present, model download fails. For explicit selection, use hf with HF_TOKEN or ngc with NGC_API_KEY.

NIM_PIPELINE_MAX_BATCH_SIZE controls the tensor buffer size and therefore GPU memory usage. Changing NIM_ENGINE_WARMUP_MODE or NIM_ENGINE_CUDA_GRAPH_CAPTURE changes startup work and graph coverage, but it does not reduce the tensor buffer allocation.

NIM_ENGINE_WARMUP_MODE=explicit and NIM_ENGINE_CUDA_GRAPH_CAPTURE=partial are not supported. To pin specific batch sizes, use NIM_ENGINE_WARMUP_SHAPES=<sizes> and NIM_ENGINE_CUDA_GRAPH_CAPTURE_SHAPES=<sizes>. Do not set explicit warmup or capture shapes together with NIM_ENGINE_WARMUP_MODE=none.

Compatibility With Previous Releases#

Most environment variables beginning with NIM_TRITON_ are not supported by the 2.0.0 runtime. For new deployments, update your code to use the current environment variables.

The following deprecated compatibility aliases remain available for migration.

Previous Variable	Use Instead	Notes
`NIM_TRITON_DATA_MAX_BATCH_SIZE`, `NIM_TRITON_MAX_BATCH_SIZE`, `NIM_TRITON_MODEL_MAX_BATCH_SIZE`	`NIM_PIPELINE_MAX_BATCH_SIZE`	Controls the maximum number of images per engine forward pass.
`NIM_TRITON_PIPELINE_MAX_BATCH_SIZE`	`NIM_SERVER_MAX_QUEUE_SIZE`	Controls the batcher request queue depth.
`NIM_TRITON_MODEL_INSTANCE_COUNT`	`NIM_ENGINE_COUNT`	Controls the number of engine replicas for the selected model.
`NIM_TRITON_DYNAMIC_BATCHING_MAX_QUEUE_DELAY_MICROSECONDS`	`NIM_SERVER_MAX_WAIT_MS`	Controls the request batching wait window in milliseconds. The 2.0.0 runtime interprets `NIM_TRITON_DYNAMIC_BATCHING_MAX_QUEUE_DELAY_MICROSECONDS` values as milliseconds despite the variable name. For example, a previous value of `10000` changes from 10 ms to 10 seconds. To preserve a 10 ms wait window, set `NIM_SERVER_MAX_WAIT_MS=10`.
`NIM_TRITON_REQUEST_TIMEOUT`	`NIM_SERVER_REQUEST_TIMEOUT_S`	Controls the per-request timeout in seconds.
`NIM_TRITON_ENABLE_PIPELINE_TIMING`	`NIM_PIPELINE_TIMING_TELEMETRY`	Enables timing telemetry logs.
`NIM_OBJECT_DETECTION_MODEL_PATH`	`NIM_ENGINE_MODEL_PATH`	Controls the model artifact path.

The 2.0.0 runtime rejects the removed NIM_ENGINE_CUDA_DEVICE and NIM_ENGINE_CUDA_DEVICES variables at startup. Use NIM_ENGINE_DEVICES instead.

Other NIM_TRITON_* variables, including Triton model-control, memory-pool, GPU-decoding, asynchronous execution, flush-interval, idle-bytes, and rate-limit settings, do not configure this runtime.