Skip to navigation

Configuring the NIM

View as Markdown

The following environment variables can be used to configure the NIM at runtime:

ENVRequiredDefaultNotes
NGC_API_KEYYes—Your NGC API key for model access.
NIM_HTTP_API_PORTNo8000The port for the HTTP API server.
USE_CUDA_IPCNoAuto-detectedEnables CUDA IPC for inter-process communication. If not explicitly set, this value will be auto-detected.
0: Disabled
1: Enabled
ENABLE_CUDA_MPSNo0Allows CUDA to use the multi process server. On some systems and configurations, this can produce increased performance. Values
0: Disabled
1: Enabled
NIM_ENABLE_OTELNoTrueEnables OpenTelemetry.
CUDA_VISIBLE_DEVICESNoAllA comma-separated list of GPU IDs to use
NIM_CACHE_PATHNo/opt/nim/.cacheThe default location in the NIM to use for caching.
NIM_LOG_LEVELNoDEFAULTControls NIM logging verbosity. Supported levels are [‘DEFAULT’, ‘DEBUG’, ‘INFO’, ‘WARNING’, ‘ERROR’, ‘CRITICAL’, ‘TRACE’]
NIM_TRITON_LOG_VERBOSENo0Controls logging verbosity of the Triton server component. Supported values are [0,1,2,3,4,5], with 0 being the least verbose and 5 being the most verbose option.

Additional Runtime Variables:

  • –tmpfs /tmp/ram,rw,size=2g: Can be used to configure /tmp/ram to use host memory space for potential speed up on some systems.

Host caching via LOCAL_NIM_CACHE

To persist model downloads across container restarts and align with other NIMs, bind mount a host cache directory using LOCAL_NIM_CACHE (see Quickstart for example commands). The directory you mount on the host becomes the in-container /opt/nim/.cache path, which is also the value of NIM_CACHE_PATH.

Notes on input resolution and variants

The current release exposes the model as cosmos-embed1 and bundles the 224p variant, which produces 256‑dimensional embeddings. Input videos using supported codecs and sizes are automatically resized by the NIM. Variant selection is not configurable in this release.