Is this page helpful?

Utility Scripts for Speech NIM Containers#

NIM includes a set of utility scripts to assist with NIM operation.

Utilities can be launched by adding the name of the desired utility to the Docker run command.

Refer to the Support Matrix section for setting valid values for CONTAINER_ID in the following examples.

List Model Profiles#

nim_list_model_profiles()#

Prints the system information detected by NIM, and the list of all profiles for the chosen NIM to the console. Profiles are categorized by whether or not they are compatible with the current system, based on the system information detected.

This function can also be called using its alias list-model-profiles.

Example#

export CONTAINER_ID=parakeet-0-6b-ctc-en-us
docker run -it --rm --gpus all --entrypoint nim_list_model_profiles \
    nvcr.io/nim/nvidia/$CONTAINER_ID:latest

SYSTEM INFO
- Free GPUs: <None>
- Non-free GPUs:
  -  [2330:10de] (0) NVIDIA H100 80GB HBM3 (H100 80GB) [current utilization: 15%]
MODEL PROFILES
- Compatible with system:
    - 6c9d2a0d1721...e687029e9 - ampereplus:enabled|diarizer:sortformer|gpu:a100|gpu_device:20b2|mode:all|model_type:prebuilt|name:parakeet-0-6b-ctc-en-us|vad:silero
    - be0a954aecfb...40c01c96cc5a04c4 - diarizer:sortformer|mode:all|model_type:rmir|name:parakeet-0-6b-ctc-en-us|vad:silero
- Incompatible with system:
    - 03637268c8af...fdab397fb33c82bf - ampereplus:disabled|diarizer:disabled|gpu:h100|gpu_device:2330|mode:str-thr|model_type:prebuilt|name:parakeet-0-6b-ctc-en-us|vad:default
    ...

export CONTAINER_ID=magpie-tts-multilingual
docker run -it --rm --gpus all --entrypoint nim_list_model_profiles \
    nvcr.io/nim/nvidia/$CONTAINER_ID:latest

SYSTEM INFO
- Free GPUs:
  -  [2331:10de] (0) NVIDIA H100 PCIe [current utilization: 0%]
MODEL PROFILES
- Compatible with system:
- Incompatible with system:
    - 40b4843dedfaab9216c650f5e81973df7e903f6d7a63924859064c4caba3b266 - {'model_type': 'prebuilt', 'name': 'magpie-tts-multilingual'}

export CONTAINER_ID=riva-translate-1_6b
docker run -it --rm --gpus all --entrypoint nim_list_model_profiles \
    nvcr.io/nim/nvidia/$CONTAINER_ID:latest

...
SYSTEM INFO
- Free GPUs:
  -  [2331:10de] (0) NVIDIA H100 PCIe [current utilization: 0%]
MODEL PROFILES
- Compatible with system:
- Incompatible with system:
    - 9f54ea088ebb1e656ee51b6311b4b8ad25e919faab129aa3ccbdd2b984570cd6 - {'model_type': 'prebuilt', 'name': 'riva-translate-1_6b'}
...

Download Model Profiles to NIM Cache#

nim_download_to_cache()#

Downloads selected or default model profile(s) to NIM cache. Can be used to pre-cache profiles prior to deployment. Requires NGC_API_KEY in environment.

This function can also be called using its alias download-to-cache.

--profiles [PROFILES ...], -p [PROFILES ...]#

Profile hashes to download. If none are provided, the optimal profile is downloaded. Multiple profiles can be specified separated by spaces.

--all#

Set to download all profiles to cache

--lora#

Set this to download default lora profile. This expects --profiles and --all arguments are not specified.

manifest-file <manifest_file>, -m <manifest_file>#

The manifest file path is an optional parameter that users can specify. It allows for the downloading of model profiles.

--model-cache-path <model-cache-path>#

The model cache path is an optional parameter that users can specify. This feature enables the modification of the default model_cache_path.

Example#

export CONTAINER_ID=parakeet-0-6b-ctc-en-us
export LOCAL_NIM_CACHE=$HOME/cache
docker run -it --rm --gpus all -e NGC_API_KEY \
    -v $LOCAL_NIM_CACHE:/opt/nim/.cache --entrypoint nim_download_to_cache \
    nvcr.io/nim/nvidia/$CONTAINER_ID:latest \
    -p 2ad57968fc4f7d13232be825ab85ea4782e96a73c8a6829a2fc27206ed0d8ff9

INFO 2025-08-06 07:04:13.827 download.py:82] Fetching contents for profile 2ad57968fc4f7d13232be825ab85ea4782e96a73c8a6829a2fc27206ed0d8ff9
INFO 2025-08-06 07:04:13.828 download.py:87] {
  "ampereplus": "enabled",
  "gpu": "a100",
  "gpu_device": "20b2",
  "mode": "ofl",
  "model_type": "prebuilt",
  "name": "parakeet-0-6b-ctc-en-us"
}

export CONTAINER_ID=magpie-tts-multilingual
export LOCAL_NIM_CACHE=$HOME/cache
docker run -it --rm --gpus all -e NGC_API_KEY \
    -v $LOCAL_NIM_CACHE:/opt/nim/.cache --entrypoint nim_download_to_cache \
    nvcr.io/nim/nvidia/$CONTAINER_ID:latest \
    -p 40b4843dedfaab9216c650f5e81973df7e903f6d7a63924859064c4caba3b266

...
"timestamp": "2025-03-15 06:53:57,256", "level": "INFO", "message": "Fetching contents for profile 40b4843dedfaab9216c650f5e81973df7e903f6d7a63924859064c4caba3b266"
"timestamp": "2025-03-15 06:53:57,257", "level": "INFO", "message": "{
  "model_type": "prebuilt",
  "name": "magpie-tts-multilingual"
}"
"timestamp": "2025-03-15 06:53:57,257", "level": "INFO", "message": "Using the profile specified by the user: 40b4843dedfaab9216c650f5e81973df7e903f6d7a63924859064c4caba3b266"
"timestamp": "2025-03-15 06:53:57,257", "level": "INFO", "message": "Downloading manifest profile: 40b4843dedfaab9216c650f5e81973df7e903f6d7a63924859064c4caba3b266"
...

export CONTAINER_ID=riva-translate-1_6b
export LOCAL_NIM_CACHE=$HOME/cache
docker run -it --rm --gpus all -e NGC_API_KEY \
    -v $LOCAL_NIM_CACHE:/opt/nim/.cache --entrypoint nim_download_to_cache \
    nvcr.io/nim/nvidia/$CONTAINER_ID:latest \
    -p 9f54ea088ebb1e656ee51b6311b4b8ad25e919faab129aa3ccbdd2b984570cd6

...
INFO 2025-05-13 10:24:23.985 download.py:81] Fetching contents for profile 9f54ea088ebb1e656ee51b6311b4b8ad25e919faab129aa3ccbdd2b984570cd6
INFO 2025-05-13 10:24:23.985 download.py:86] {
  "model_type": "prebuilt",
  "name": "riva-translate-1_6b"
}
INFO 2025-05-13 10:24:23.985 nim_sdk.py:285] Using the profile specified by the user: 9f54ea088ebb1e656ee51b6311b4b8ad25e919faab129aa3ccbdd2b984570cd6
INFO 2025-05-13 10:24:23.985 nim_sdk.py:299] Downloading manifest profile: 9f54ea088ebb1e656ee51b6311b4b8ad25e919faab129aa3ccbdd2b984570cd6
...

Create Model Store#

nim_create_model_store()#

Extracts files from a cached model profile and creates a properly formatted directory. If the profile is not already cached, it will be downloaded to the model cache. Downloading the profile requires NGC_API_KEY in environment.

This function can also be called using its alias create-model-store.

--profile <PROFILE>, -p <PROFILE>#

Profile hash to create a model directory of. Will be downloaded if not present.

--model-store <MODEL_STORE>, -m <MODEL_STORE>#

Directory path where model --profile will be extracted and copied to.

--model-cache-path <model-cache-path>#

The model cache path is an optional parameter that users can specify. This feature enables the modification of the default model_cache_path.

Example#

export CONTAINER_ID=parakeet-0-6b-ctc-en-us
export LOCAL_NIM_CACHE=$HOME/cache
docker run -it --rm --gpus all -e NGC_API_KEY \
    -v $LOCAL_NIM_CACHE:/opt/nim/.cache --entrypoint nim_create_model_store \
    nvcr.io/nim/nvidia/$CONTAINER_ID:latest \
    -p 2ad57968fc4f7d13232be825ab85ea4782e96a73c8a6829a2fc27206ed0d8ff9 \
    -m /tmp

...
INFO 2025-08-06 07:07:23.311 create_model_store.py:62] Fetching contents for profile 2ad57968fc4f7d13232be825ab85ea4782e96a73c8a6829a2fc27206ed0d8ff9
INFO 2025-08-06 07:07:23.312 nim_sdk.py:383] Using the default model_cache_path: /opt/nim/workspace
INFO 2025-08-06 07:07:23.312 nim_sdk.py:393] Creating model store at /tmp
...

export CONTAINER_ID=magpie-tts-multilingual
export LOCAL_NIM_CACHE=$HOME/cache
docker run -it --rm --gpus all -e NGC_API_KEY \
    -v $LOCAL_NIM_CACHE:/opt/nim/.cache --entrypoint nim_create_model_store \
    nvcr.io/nim/nvidia/$CONTAINER_ID:latest \
    -p 40b4843dedfaab9216c650f5e81973df7e903f6d7a63924859064c4caba3b266 \
    -m /tmp

...
"timestamp": "2025-03-15 06:55:34,455", "level": "INFO", "message": "Fetching contents for profile 40b4843dedfaab9216c650f5e81973df7e903f6d7a63924859064c4caba3b266"
"timestamp": "2025-03-15 06:55:34,455", "level": "INFO", "message": "Using the default model_cache_path: /opt/nim/workspace"
"timestamp": "2025-03-15 06:55:34,455", "level": "INFO", "message": "Creating model store at /tmp"
...

export CONTAINER_ID=riva-translate-1_6b
export LOCAL_NIM_CACHE=$HOME/cache
docker run -it --rm --gpus all -e NGC_API_KEY \
    -v $LOCAL_NIM_CACHE:/opt/nim/.cache --entrypoint nim_create_model_store \
    nvcr.io/nim/nvidia/$CONTAINER_ID:latest \
    -p 9f54ea088ebb1e656ee51b6311b4b8ad25e919faab129aa3ccbdd2b984570cd6 \
    -m /tmp

...
INFO 2025-05-13 10:25:13.833 create_model_store.py:57] Fetching contents for profile 9f54ea088ebb1e656ee51b6311b4b8ad25e919faab129aa3ccbdd2b984570cd6
INFO 2025-05-13 10:25:13.833 nim_sdk.py:368] Using the default model_cache_path: /opt/nim/workspace
INFO 2025-05-13 10:25:13.833 nim_sdk.py:378] Creating model store at /tmp
...

Check NIM Cache#

nim_check_cache_env()#

Checks if the NIM cache directory is present and can be written to.

This function can also be called using its alias nim-llm-check-cache-env.

Example#

export CONTAINER_ID=<container-id>
export LOCAL_NIM_CACHE=$HOME/cache
docker run -it --rm --gpus all -e NGC_API_KEY \
    -v /bad_path:/opt/nim/.cache --entrypoint nim_check_cache_env \
    nvcr.io/nim/nvidia/$CONTAINER_ID:latest

The NIM cache directory /opt/nim/.cache is read-only. Application may fail if the model is not already present in the cache.