Utilities#

NIM includes a set of utility scripts to assist with NIM operation.

Utilities can be launched by adding the name of the desired utility to the docker run command.

See the Supported Models section for setting valid values for CONTAINER_ID in below examples.

List Model Profiles#

nim_list_model_profiles()#

Prints the system information detected by NIM, and the list of all profiles for the chosen NIM to the console. Profiles are categorized by whether or not they are compatible with the current system, based on the system information detected.

This function can also be called using its alias list-model-profiles.

Example#

export CONTAINER_ID=parakeet-0-6b-ctc-en-us
docker run -it --rm --gpus all --entrypoint nim_list_model_profiles \
    nvcr.io/nim/nvidia/$CONTAINER_ID:latest

SYSTEM INFO
- Free GPUs: <None>
- Non-free GPUs:
  -  [2330:10de] (0) NVIDIA H100 80GB HBM3 (H100 80GB) [current utilization: 15%]
MODEL PROFILES                                                                                                                                                       08:34:09 [5/16757]
- Compatible with system:
    - 6c9d2a0d172114611a0c31f61a33dfed80b1f5ea94b00856f5fb44ce687029e9 - ampereplus:enabled|diarizer:sortformer|gpu:a100|gpu_device:20b2|mode:all|model_type:prebuilt|name:parakeet-0-6b-ctc-en-us|vad:silero
    - be0a954aecfb66242ae3ea58d5c2585b1059a4980a2b858288815d3425a49afc - diarizer:sortformer|mode:all|model_type:rmir|name:parakeet-0-6b-ctc-en-us|vad:silero
- Incompatible with system:
    - 03637268c8aff691045941ec4651ad41fdab397fb33c82bf34a75f3ace3771e2 - ampereplus:disabled|diarizer:disabled|gpu:h100|gpu_device:2330|mode:str-thr|model_type:prebuilt|name:parakeet-0-6b-ctc-en-us|vad:default
    - 0665ee684534d741879593247b97b2024f06a5a93932a8a2070c08cd9c6b9323 - ampereplus:disabled|diarizer:disabled|gpu:l40s|gpu_device:26b9|mode:str-thr|model_type:prebuilt|name:parakeet-0-6b-ctc-en-us|vad:default
    - 1f07a2762d7276e08d8fcec18ca485077e082941211c293e955c3fb59dc37231 - ampereplus:disabled|bs:1|diarizer:disabled|gpu:h100|gpu_device:2330|mode:str|model_type:prebuilt|name:parakeet-0-6b-ctc-en-us|vad:default
    - 22d083853a66db56d7f3e25e8b010463d2b564d5d19a893a19f34bf3cb716bbe - ampereplus:disabled|diarizer:disabled|gpu:l40s|gpu_device:26b9|mode:ofl|model_type:prebuilt|name:parakeet-0-6b-ctc-en-us|vad:default
    - 2672beec3331bedd2dbe0f74ba1e836483cf542c5656a27bba3ab8e149f94bcc - ampereplus:disabled|diarizer:disabled|gpu:h100|gpu_device:2330|mode:str|model_type:prebuilt|name:parakeet-0-6b-ctc-en-us|vad:default
    - 276f9ffd02d07c186504aa6c050f24449853451751eea0a07e406dea2f36b431 - ampereplus:disabled|diarizer:disabled|gpu:h100|gpu_device:2330|mode:all|model_type:prebuilt|name:parakeet-0-6b-ctc-en-us|vad:default
    - 301d5edf26b76add72a2cacf09f219cca9e4baa22400b4ac70fa16b84a54ef82 - ampereplus:disabled|diarizer:disabled|gpu:a100|gpu_device:20b2|mode:str|model_type:prebuilt|name:parakeet-0-6b-ctc-en-us|vad:default
    - 38ca5f18305addde525c1ddbce20054854450e05cf9e0c2eedf5faae6766219f - ampereplus:disabled|diarizer:disabled|gpu:h100|gpu_device:2330|mode:ofl|model_type:prebuilt|name:parakeet-0-6b-ctc-en-us|vad:default
    - 392aa45cfdc4614c99e4c3ce4ebf46ee15b91593b89c19250e2df811a0333b34 - ampereplus:enabled|diarizer:sortformer|gpu:a100|gpu_device:20b2|mode:ofl|model_type:prebuilt|name:parakeet-0-6b-ctc-en-us|vad:silero
    - 491620d74740ab3ba789b481685d35275d1fb48bc218fb10f5ee79b474f0cc4c - ampereplus:disabled|diarizer:disabled|gpu:a100|gpu_device:20b2|mode:str-thr|model_type:prebuilt|name:parakeet-0-6b-ctc-en-us|vad:default
    - 66f84e06cd1d7b84aa5bc11e9586259600830f9f502676999a184910c85ce501 - ampereplus:disabled|bs:1|diarizer:disabled|gpu:l40s|gpu_device:26b9|mode:ofl|model_type:prebuilt|name:parakeet-0-6b-ctc-en-us|vad:default
    - 7cd4f6369dd01bb8e219853278eebd11b2acdab26bad226c7d2c4eccbaa006e5 - ampereplus:enabled|bs:1|diarizer:disabled|gpu:a100|gpu_device:20b2|mode:str|model_type:prebuilt|name:parakeet-0-6b-ctc-en-us|vad:default
    - 8187511a3f72e5c2456d68096d3bfea790d37ca96df0f403a23acd7949d8209d - ampereplus:enabled|bs:1|diarizer:disabled|gpu:a100|gpu_device:20b2|mode:ofl|model_type:prebuilt|name:parakeet-0-6b-ctc-en-us|vad:default
    - 9343a7c31afe146f9f872e87f79275d68dc5c4f79b6f7d7a01b5134cd0103536 - diarizer:disabled|mode:ofl|model_type:rmir|name:parakeet-0-6b-ctc-en-us|vad:default
    - 966231e2385df9000a569b4bc9e4354e4db6ec2600b967e0cc6aa93c89bcfa81 - ampereplus:enabled|diarizer:sortformer|gpu:a100|gpu_device:20b2|mode:str-thr|model_type:prebuilt|name:parakeet-0-6b-ctc-en-us|vad:silero
    - 9dca193fe501ab9bae9feaacd7928c85515de8c4c1028a7fde6a56445966fe48 - ampereplus:disabled|diarizer:disabled|gpu:a100|gpu_device:20b2|mode:ofl|model_type:prebuilt|name:parakeet-0-6b-ctc-en-us|vad:default
    - a562b374e7a5fe015d94ba017c6c381003ec9af5d02da1ac16557d611840f76f - diarizer:sortformer|mode:str|model_type:rmir|name:parakeet-0-6b-ctc-en-us|vad:silero
    - a98c2a7eb5bd0ffe9d0e7d094f2900b967c0892ce8f1ac0bc17935e5d13f9a9c - diarizer:disabled|mode:str-thr|model_type:rmir|name:parakeet-0-6b-ctc-en-us|vad:default
    - b7a0a946de3f2af3e80078c9409427cddaca5d346dce5ee87619954557d1fce6 - ampereplus:disabled|bs:1|diarizer:disabled|gpu:h100|gpu_device:2330|mode:ofl|model_type:prebuilt|name:parakeet-0-6b-ctc-en-us|vad:default
    - b9c146e51c70c4e008538ba2e820dd3ee9c26d4bd14045ef6c071923cbd49b21 - bs:1|diarizer:disabled|mode:ofl|model_type:rmir|name:parakeet-0-6b-ctc-en-us|vad:default
    - bb45923122d1f7751ba72a6f45219d72555bede135d7dda3e160e919418b505b - ampereplus:disabled|diarizer:disabled|gpu:l40s|gpu_device:26b9|mode:all|model_type:prebuilt|name:parakeet-0-6b-ctc-en-us|vad:default
    - c8cec9ffd1c65ab54f328c0434ce7feddd680c38ef5591cdb26a55676440d63f - ampereplus:disabled|bs:1|diarizer:disabled|gpu:l40s|gpu_device:26b9|mode:str|model_type:prebuilt|name:parakeet-0-6b-ctc-en-us|vad:default
    - cb06f68005e2faa9940ca32016c22a6c9647493bb32e8b7440c01c96cc5a04c4 - diarizer:disabled|mode:all|model_type:rmir|name:parakeet-0-6b-ctc-en-us|vad:default
    - d83b059c9c3e521b0eaf72cac325945a21fec7075e4620f644101dc885cea764 - diarizer:sortformer|mode:ofl|model_type:rmir|name:parakeet-0-6b-ctc-en-us|vad:silero
    - dc4d413c27977abed0eb1ee2fdccc83e1b17134f3294206421b89f5e073a971d - bs:1|diarizer:disabled|mode:str|model_type:rmir|name:parakeet-0-6b-ctc-en-us|vad:default
    - dd802845c30a675f9ef4827eea4a7386197cd48cfc52fb12b404d5402d7da74f - ampereplus:disabled|diarizer:disabled|gpu:l40s|gpu_device:26b9|mode:str|model_type:prebuilt|name:parakeet-0-6b-ctc-en-us|vad:default
    - def4261bde203638c6fd2a1c156740295f4f9a07a595129f74fabc0e947635a3 - diarizer:disabled|mode:str|model_type:rmir|name:parakeet-0-6b-ctc-en-us|vad:default
    - e0c8c9372c1b44bdae7a1805de1463d84918606f2ca0459ca1471a714a090416 - diarizer:sortformer|mode:str-thr|model_type:rmir|name:parakeet-0-6b-ctc-en-us|vad:silero
    - e2cfad391dd7def2a27180287ba370010c4b96471a47668e73ef15b208d61711 - ampereplus:enabled|diarizer:sortformer|gpu:a100|gpu_device:20b2|mode:str|model_type:prebuilt|name:parakeet-0-6b-ctc-en-us|vad:silero
    - e30c84f6e358f569562ef6bda9acca51cce55cd80bf9505d548dce16dc972373 - ampereplus:disabled|diarizer:disabled|gpu:a100|gpu_device:20b2|mode:all|model_type:prebuilt|name:parakeet-0-6b-ctc-en-us|vad:default

Download Model Profiles to NIM Cache#

nim_download_to_cache()#

Downloads selected or default model profile(s) to NIM cache. Can be used to pre-cache profiles prior to deployment. Requires NGC_API_KEY in environment.

This function can also be called using its alias download-to-cache.

--profiles [PROFILES ...], -p [PROFILES ...]#: Profile hashes to download. If none are provided, the optimal profile is downloaded. Multiple profiles can be specified separated by spaces.

--all#: Set to download all profiles to cache

--lora#: Set this to download default lora profile. This expects --profiles and --all arguments are not specified.

manifest-file <manifest_file>, -m <manifest_file>#: The manifest file path is an optional parameter that users can specify. It allows for the downloading of model profiles.

--model-cache-path <model-cache-path>#: The model cache path is an optional parameter that users can specify. This feature enables the modification of the default model_cache_path.

Example#

export CONTAINER_ID=parakeet-0-6b-ctc-en-us
export LOCAL_NIM_CACHE=$HOME/cache
docker run -it --rm --gpus all -e NGC_API_KEY \
    -v $LOCAL_NIM_CACHE:/opt/nim/.cache --entrypoint nim_download_to_cache \
    nvcr.io/nim/nvidia/$CONTAINER_ID:latest \
    -p 2ad57968fc4f7d13232be825ab85ea4782e96a73c8a6829a2fc27206ed0d8ff9

INFO 2025-08-06 07:04:13.827 download.py:82] Fetching contents for profile 2ad57968fc4f7d13232be825ab85ea4782e96a73c8a6829a2fc27206ed0d8ff9
INFO 2025-08-06 07:04:13.828 download.py:87] {
  "ampereplus": "enabled",
  "gpu": "a100",
  "gpu_device": "20b2",
  "mode": "ofl",
  "model_type": "prebuilt",
  "name": "parakeet-0-6b-ctc-en-us"
}

Create Model Store#

nim_create_model_store()#

Extracts files from a cached model profile and creates a properly formatted directory. If the profile is not already cached, it will be downloaded to the model cache. Downloading the profile requires NGC_API_KEY in environment.

This function can also be called using its alias create-model-store.

--profile <PROFILE>, -p <PROFILE>#: Profile hash to create a model directory of. Will be downloaded if not present.

--model-store <MODEL_STORE>, -m <MODEL_STORE>#: Directory path where model --profile will be extracted and copied to.

--model-cache-path <model-cache-path>#: The manifest file path is an optional parameter that users can specify. This feature enables the modification of the default model_cache_path.

Example#

export CONTAINER_ID=parakeet-0-6b-ctc-en-us
export LOCAL_NIM_CACHE=$HOME/cache
docker run -it --rm --gpus all -e NGC_API_KEY \
    -v $LOCAL_NIM_CACHE:/opt/nim/.cache --entrypoint nim_create_model_store \
    nvcr.io/nim/nvidia/$CONTAINER_ID:latest \
    -p 2ad57968fc4f7d13232be825ab85ea4782e96a73c8a6829a2fc27206ed0d8ff9 \
    -m /tmp

...
INFO 2025-08-06 07:07:23.311 create_model_store.py:62] Fetching contents for profile 2ad57968fc4f7d13232be825ab85ea4782e96a73c8a6829a2fc27206ed0d8ff9
INFO 2025-08-06 07:07:23.312 nim_sdk.py:383] Using the default model_cache_path: /opt/nim/workspace
INFO 2025-08-06 07:07:23.312 nim_sdk.py:393] Creating model store at /tmp
...

Check NIM Cache#

nim_check_cache_env()#

Checks if the NIM cache directory is present and can be written to.

This function can also be called using its alias nim-llm-check-cache-env.

Example#

export CONTAINER_ID=parakeet-0-6b-ctc-en-us
export LOCAL_NIM_CACHE=$HOME/cache
docker run -it --rm --gpus all -e NGC_API_KEY \
    -v /bad_path:/opt/nim/.cache --entrypoint nim_check_cache_env \
    nvcr.io/nim/nvidia/$CONTAINER_ID:latest \

The NIM cache directory /opt/nim/.cache is read-only. Application may fail if the model is not already present in the cache.