> This page is for version 1.0.0 (Latest) (default). > For other versions, use one of these documentation indexes: > - 1.0.0 (Latest) (default): https://docs.nvidia.com/nim/harmonizer/1.0.0/llms.txt > For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.nvidia.com/nim/harmonizer/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nim/harmonizer/_mcp/server. # Environment Variables and Configuration for Harmonizer NIM > Configure Harmonizer NIM GPU selection, inference transport, model cache, runtime environment variables, and performance settings. This page contains a reference for configuring the Harmonizer NIM container. ## GPU Selection Harmonizer NIM uses one GPU per container. Passing `--gpus all` is acceptable on a single-GPU host. On a multi-GPU host, expose one target GPU with either `--gpus` or `CUDA_VISIBLE_DEVICES`. For example: ```bash docker run --gpus '"device=0"' ... ``` or: ```bash docker run -e CUDA_VISIBLE_DEVICES=0 --gpus all ... ``` ## Inference Protocol The container can serve either HTTP or gRPC. `NIM_INFERENCE_PROTOCOL` defaults to `http`. Set `NIM_INFERENCE_PROTOCOL=grpc` to start the gRPC entrypoint. The HTTP health and readiness routes remain available so deployment readiness checks work in either mode. Protocol selection does not restrict processing mode: both non-temporal and temporal requests are available over the selected transport. ## Shared Memory The Docker examples set `--shm-size=8g` as the recommended launch configuration. The codec workers and GPU coalescing queue use in-process threads and do not exchange request tensors through multiprocessing shared memory. ## Model Cache The container process runs as user ID `1000`. The host directory mounted at `/opt/nim/.cache` (`NIM_CACHE_PATH`) must therefore be writable by UID `1000`, otherwise startup fails while materializing model artifacts with “Permission denied” or “os error 13”. > **Warning** > > NFS-backed home directories are frequently not writable by UID `1000` and > can additionally fail with “Stale file handle”. Prefer a local-disk cache > path for direct Docker runs. Create a cache directory that UID `1000` can write to before launching: ```bash mkdir -p /tmp/harmonizer-nim-cache chmod 777 /tmp/harmonizer-nim-cache ``` Alternatively, run the container with `--user "$(id -u):$(id -g)"` so the process matches the owning UID of the mounted cache directory. `start_server.sh` places the Hugging Face and Torch asset caches, TorchInductor cache, and Triton cache below `NIM_CACHE_PATH`. Generated kernels are stored under `torchinductor` and `triton`. Persisting this bind mount lets later starts reuse model assets, generated kernels, and autotuning results. Model loading, Dynamo tracing and guard construction, CUDA-graph capture, and warmup still run on each process start. The image does not contain a compilation cache generated at build time. With version `1.3.0` or later of the `nim-wfm` Helm chart, set `model.nimCache` instead of setting `NIM_CACHE_PATH` in `env`. The chart sets `NIM_CACHE_PATH` to this value and mounts its `model-store` volume at the same path. When `persistence.enabled` is `true`, that volume is a persistent volume claim (PVC); with persistence, host-path storage, and NFS disabled, it is an ephemeral `emptyDir`. The Harmonizer Helm example uses `/model-store`, overriding the container default of `/opt/nim/.cache`. Refer to [Deploy Harmonizer NIM with Helm](/nim/harmonizer/helm) for a complete values file. A cache can be reused across supported GPU SKUs. TorchInductor and Triton cache lookups reuse compatible artifacts rather than blindly selecting an artifact from another SKU. The first start on a new SKU can still compile additional artifacts, take longer, and consume more space in the shared cache. Separating claims by SKU can provide more predictable warm-start time and capacity, but it is not required for cache correctness. ## General Environment Variables The following table describes the runtime configuration variables: | Name | Default Value | Description | | ------------------------------------------ | --------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `NGC_API_KEY` | unset | API key used to download assets from NGC. | | `NIM_CACHE_PATH` | `/opt/nim/.cache` | Path inside the container where model assets and the compiled-kernel cache are stored. Mount this path to persistent storage. | | `NIM_HTTP_API_PORT` | `8000` | HTTP API port inside the container. | | `NIM_GRPC_API_PORT` | `50051` | gRPC API port inside the container. | | `NIM_INFERENCE_PROTOCOL` | `http` | Selects one inference transport at startup. The default serves `POST /v1/infer` over HTTP; set to `grpc` to serve `harmonizer.v1.Harmonizer/Process` instead. Both selections support non-temporal and temporal request modes. HTTP health endpoints remain available in either selection. | | `NIM_FT_CHECKPOINT` | unset | BYOC checkpoint directory. Refer to [Deploy a Custom Harmonizer Checkpoint](/nim/harmonizer/bring-your-own-checkpoint). | | `HARMONIZER_PKL_MODEL_PATH` | `diffusion_harmonizer.pkl` within the BYOC directory when BYOC is set | Full in-container checkpoint path for a custom file name or location. Refer to [Deploy a Custom Harmonizer Checkpoint](/nim/harmonizer/bring-your-own-checkpoint). | | `HARMONIZER_PKL_OPTIMIZED_SPECIALIZATIONS` | GPU-specific | Fixed batch and frame shapes prepared at startup. The required shapes are `1x1`, `1x5`, and `4x1`. Refer to [Startup Time and Batch Specializations](#startup-time-and-batch-specializations). | | `BACKEND_TYPE` | `source-pkl` | Source `.pkl` backend selector. Accepted aliases are `source-pkl`, `harmonizer-pkl`, and `pkl`. | | `HARMONIZER_PKL_PNG_COMPRESSION_LEVEL` | `1` | PNG encoder compression level. Level `1` uses the fast lossless codec and is the recommended end-to-end default. Set to `0` only for encode-latency microbenchmarks. | | `HARMONIZER_PKL_GPU_QUEUE` | `true` | Enables opportunistic request coalescing. Concurrent requests with the same warmed tensor shape can share a GPU forward batch while request histories and output ordering remain isolated. Set to `false` to use the direct single-request path. | | `HARMONIZER_PKL_GPU_QUEUE_MAXSIZE` | `16` | Maximum number of pending GPU work items. This is an advanced benchmarking and SKU-tuning control. | ## Performance Tuning The optimized serving runtime ships fully enabled by default: fixed-shape optimized executors, compiled variational autoencoder (VAE) and diffusion transformer (DiT) paths, temporal current-slice decode, the request coalescing queue, and the fast PNG codec. These defaults are tuned per GPU SKU and are covered by the release quality gate, so most deployments do not need to change them. A large set of advanced `HARMONIZER_PKL_*` flags expose this tuning (executor specialization matrix, compile and CUDA-graph options, memory formats, cuDNN autotune, and coalescing-queue depth). They are intended for benchmarking and SKU bring-up rather than production configuration. Leave them at their defaults unless you are reproducing a specific benchmark configuration. ### Startup Time and Batch Specializations The NIM reports ready only after model loading, optimization, and warmup finish. Representative cold starts with the default GPU-specific configuration are shown below. These times begin when the inference server starts and end when the service becomes ready; model download and container scheduling are not included. | GPU | Approximate Startup Time | | ------------ | ------------------------ | | A100 80 GB | 9 minutes | | H100 80 GB | 6 minutes | | H200 | 5 minutes | | B200 | 4.5 minutes | | L40S | 4.5 minutes | | RTX 6000 Ada | 4 minutes | | RTX 4090 | 2 minutes | Startup time varies with host performance, storage, compiler caches, and the configured batch specializations. Persist `/opt/nim/.cache` to make subsequent starts faster. `HARMONIZER_PKL_OPTIMIZED_SPECIALIZATIONS` controls the fixed batch and frame shapes prepared during startup. Reducing the optional batch-size specializations shortens cold startup and reduces GPU memory used by optimized states. The tradeoff is lower peak throughput, primarily on faster GPUs under high concurrency. The impact on low-concurrency serving and slower GPUs is typically small. The minimum supported value is: ```bash HARMONIZER_PKL_OPTIMIZED_SPECIALIZATIONS=1x1,1x5,4x1 ``` These three specializations are required by the API serving modes. In particular, `4x1` is required for the four-frame prefill used by temporal requests with `context_length=4`. Omitting `1x1`, `1x5`, or `4x1` causes the NIM to fail during startup. All other batch-size specializations are optional throughput optimizations. ## Related Topics Continue with these related guides: * [Quickstart Guide](/nim/harmonizer/quickstart-guide) * [Deploy with Helm](/nim/harmonizer/helm)