> This page is for version 1.0.0 (Latest) (default). > For other versions, use one of these documentation indexes: > - 1.0.0 (Latest) (default): https://docs.nvidia.com/nim/alpamayo/1.0.0/llms.txt > For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.nvidia.com/nim/alpamayo/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nim/alpamayo/_mcp/server. # Configuring a NIM This page contains a reference for configuring the Alpamayo1.5 NIM container. ## GPU Selection Alpamayo1.5 NIM currently supports one GPU per container. You can pass `--gpus all` on a single-GPU host. On a multi-GPU host, expose one target GPU with either `--gpus` or `CUDA_VISIBLE_DEVICES`. For example: ```bash docker run --gpus '"device=0"' ... ``` or: ```bash docker run -e CUDA_VISIBLE_DEVICES=0 --gpus all ... ``` ## Shared Memory Set `--shm-size=16g` for direct Docker usage because PyTriton and TensorRT-LLM exchange multimodal request payloads through shared memory. Host IPC can also work when allowed by your deployment environment, but an explicit `--shm-size=16g` keeps the requirement visible in the launch command. ## Model Cache The container process runs as user ID `1000`. The host directory mounted at `/opt/nim/.cache` (`NIM_CACHE_PATH`) must therefore be writable by UID `1000`, otherwise startup fails while materializing model artifacts with `Permission denied` / `os error 13`. NFS-backed home directories are frequently not writable by UID `1000` and can additionally fail with `Stale file handle`. Prefer a local-disk cache path for direct Docker runs. Create a cache directory that UID `1000` can write to before launching: ```bash mkdir -p /tmp/alpamayo-nim-cache chmod 777 /tmp/alpamayo-nim-cache ``` Alternatively, run the container with `--user "$(id -u):$(id -g)"` so the process matches the owning UID of the mounted cache directory. ## Environment Variables The server accepts the following customer-facing variables. | Variable | Default | Notes | | -------------------------------------- | ----------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `NGC_API_KEY` | unset | API key used to download assets from NGC. | | `NIM_CACHE_PATH` | `/opt/nim/.cache` | Path inside the container where model assets and generated artifacts are cached. Mount this path to persistent storage. | | `NIM_PRECISION` | auto | Optional precision selector. Supported values are `bf16`, `fp8`, and `w4a16`. The runtime maps the requested precision to the matching manifest profile and then applies compute-capability and memory gates. Unlisted compatible GPUs default to K=1. | | `NIM_MODEL_PROFILE` | auto | Exact model profile name or hash override. Use only when you need to select a specific manifest profile. | | `NIM_HTTP_API_PORT` | `8000` | HTTP API port inside the container. | | `NIM_FT_CHECKPOINT` | unset | BYOC Alpamayo checkpoint directory. Refer to [Bring Your Own Checkpoint](/nim/alpamayo/bring-your-own-checkpoint) for the supported checkpoint layouts and calibration requirements. | | `NIM_GRPC_API_PORT` | `50051` | Port for the gRPC trajectory API. | | `NIM_ALPAMAYO_TRAJ_SAMPLES` | `1` | Comma-separated K values warmed and served by the container. Each configured value consumes startup time and retained CUDA graph/cache memory, so configure only the values your deployment needs. Unlisted GPUs may opt into values up to 32, but should start at K=1 and increase gradually; refer to [Support Matrix](/nim/alpamayo/support-matrix) for OOM guidance. | | `NIM_ALPAMAYO_SEED` | `42` | Default seed used when a request does not provide a `seed` value. HTTP and gRPC requests can override this per request. For repeatable K=1 trajectory inference in the same deployment, use a fixed `seed` with `top_k=1` and `top_p=1`. The seed controls diffusion sampling, and top-1 decoding removes VLM sampling variability. | | `NIM_ALPAMAYO_STARTUP_WARMUP` | `1` | Run packaged sample requests before readiness. Set to `0` to skip warmup inference; the backend still loads before readiness. | | `NIM_ALPAMAYO_CALIBRATION_SAMPLE_LIST` | unset | Comma-separated text manifest files. Each manifest contains one Alpamayo scene directory per line. Used when creating BYOC FP8/W4A16 ModelOpt VLM exports. | | `NIM_ALPAMAYO_ENABLE_TIMINGS` | unset | Set to `1` to log detailed request timing payloads and emit NVTX ranges. Refer to [Observability](/nim/alpamayo/observability) for profiling guidance. | | `NIM_ALPAMAYO_TIMINGS_DIR` | unset | Directory for timing artifacts used in offline analysis. The directory must be writable by the container user. | Startup warmup prepares the configured trajectory K values before readiness. The first client request can still populate request-path caches and be slower than steady state, so run one untimed request before collecting latency data.