Is this page helpful?

Configuration#

NIM VLM offers several configuration options using environment variables to control caching, logging, and model profiles.

Export the API Key#

To use your API key when starting the NIM container, you must make it available as an environment variable.

Export the variable in your shell (temporary), replacing VALUE with your actual API key:
```
export NGC_API_KEY=VALUE
```

Persist the variable (optional):

If using bash:

echo "export NGC_API_KEY=$NGC_API_KEY" >> ~/.bashrc

If using zsh:

echo "export NGC_API_KEY=$NGC_API_KEY" >> ~/.zshrc

Verify the variable is set:
```
echo "$NGC_API_KEY"
```

Important

For enhanced security, consider storing your token in a file and retrieving it as needed with cat. Alternatively, consider using a password manager.

Model Cache and Source#

An essential variable to configure on your host system is the cache path directory. This directory is mapped from the host machine to the container; assets (for example, model weights) are downloaded to this host directory and persist across container restarts. Configuring a local cache is highly recommended, as it avoids re-downloading large model files upon subsequent container restarts. You can name the environment variable containing the path to the local cache whatever you want.

Create the cache directory and export an environment variable:

export LOCAL_NIM_CACHE=~/.cache/nim
mkdir -p $LOCAL_NIM_CACHE
# Optionally add sticky bit to avoid issues writing to the cache if the
# container is running as a different user
chmod -R a+rwxt $LOCAL_NIM_CACHE

When you start the NIM container, you must map your host machine’s local cache directory ($LOCAL_NIM_CACHE) to the container’s internal cache path (/opt/nim/.cache) using a Docker volume mount, such as -v "$LOCAL_NIM_CACHE:/opt/nim/.cache". This mapping ensures that the large model weights downloaded by the container are saved to your host machine. Because containers are ephemeral, any data stored only inside the container is lost when it stops. By using a volume mount, subsequent container runs detect the existing model files in your local cache and skip the lengthy download process, allowing the NIM to start up faster.

Cache Directory Permissions#

The NIM container runs as a non-root user with GID 0 (root group). The cache directory on your host must be writable by GID 0:

export NIM_CACHE_PATH=/tmp/nim-cache
mkdir -p "$NIM_CACHE_PATH"
sudo chgrp -R 0 "$NIM_CACHE_PATH"
sudo chmod -R g+rwX "$NIM_CACHE_PATH"

Run the container with the cache mounted:

docker run --gpus all \
   -v "$NIM_CACHE_PATH:/opt/nim/.cache" \
   ...

To run as a custom user (for example, your host user), pass -u <uid>:0:

docker run --gpus all -u $(id -u):0 \
   -v "$NIM_CACHE_PATH:/opt/nim/.cache" \
   ...

Important

When using -u <uid>, you must include :0 to set GID 0 (for example, -u $(id -u):0). The container’s writable directories are group-owned by GID 0. Without it, the container will fail with PermissionError when writing to cache, config, or log paths.

Tip

To make this setting permanent across terminal sessions, you can add export LOCAL_NIM_CACHE=~/.cache/nim to your ~/.bashrc or ~/.zshrc profile.

Advanced Configurations#

For production deployments or specific organizational requirements, you can configure additional environment variables. NIM VLM supports settings such as the following (see the full Environment Variable Reference):

TLS/SSL Configuration (NIM_SSL_MODE, NIM_SSL_KEY_PATH, NIM_SSL_CERTS_PATH, NIM_SSL_CA_CERTS_PATH): Control whether SSL/TLS is enabled for API connections, and specify the locations of SSL private keys, certificates, and CA certificates.
Unified Structured Logging and Verbosity (NIM_JSONL_LOGGING, NIM_LOG_LEVEL): Enable structured JSONL log output and set the logging verbosity level for debugging or monitoring.
Manual Model Profile Overrides (NIM_MODEL_PROFILE): Manually select or override the model execution profile, allowing control over parameters like precision or hardware acceleration.