Is this page helpful?

Air Gap Deployment#

NVIDIA NIM for Vision Language Models (VLMs) supports serving models in an air gap system (also known as air wall, air-gapping, or disconnected network). In an air gap system, you can run a NIM with no internet connection, and with no connection to the NGC registry.

Before you use this documentation, review all prerequisites and instructions in About Get Started.

For air gap deployment, see offline cache.

Air Gap Deployment (offline cache option)#

NIM supports serving models in an air gap system. If NIM detects a previously loaded profile in the cache, it serves that profile from the cache.

Prerequisites#

Before deploying in an air gap environment, you need to download the model profiles on a system with internet access. Follow these steps:

Step 1: Set up environment variables on the connected system

# Choose a container name for bookkeeping
export CONTAINER_NAME=vlm-download-container

# Set your NIM image name (replace with actual values from NGC)
export IMG_NAME=<your-nim-image-name>

# Choose a path on your system to cache the downloaded models
export LOCAL_NIM_CACHE=~/.cache/nim
mkdir -p "$LOCAL_NIM_CACHE"

Step 2: Download models to the cache on the connected system

First, launch the NIM container to access the download utilities:

docker run -it --rm --name=$CONTAINER_NAME \
  --runtime=nvidia \
  --gpus all \
  --shm-size=16GB \
  -e NGC_API_KEY=$NGC_API_KEY \
  -v "$LOCAL_NIM_CACHE:/opt/nim/.cache" \
  -u $(id -u) \
  $IMG_NAME bash

Inside the container, list available profiles and download the one you need:

# List available model profiles
list-model-profiles

# download a profile matching the air-gapped profile (replace with actual profile ID)
download-to-cache --profile 09e2f8e68f78ce94bf79d15b40a21333cea5d09dbe01ede63f6c957f4fcfab7b

Exit the container after the download completes:

exit

Air-Gapped System Deployment#

Step 3: Transfer the cache to the air-gapped system

Copy the entire cache directory from the connected system to your air-gapped system.

Step 4: Set up environment variables on the air-gapped system

# Choose a container name for bookkeeping
export CONTAINER_NAME=vlm-airgap-container

# Set your NIM image name (same as used for download)
export IMG_NAME=<your-nim-image-name>

# Path to the transferred cache directory on the air-gapped system
export AIR_GAP_NIM_CACHE=~/.cache/air-gap-nim-cache

# Ensure the directory exists and copy the transferred cache
mkdir -p "$AIR_GAP_NIM_CACHE"
cp -r <path-to-transferred-cache>/* "$AIR_GAP_NIM_CACHE"

Step 5: Launch NIM on the air-gapped system

# replace NIM_MODEL_PROFILE value with actual profile ID
docker run -it --rm --name=$CONTAINER_NAME \
  --runtime=nvidia \
  --gpus all \
  --shm-size=16GB \
  -e NIM_MODEL_PROFILE=09e2f8e68f78ce94bf79d15b40a21333cea5d09dbe01ede63f6c957f4fcfab7b \
  -v "$AIR_GAP_NIM_CACHE:/opt/nim/.cache" \
  -u $(id -u) \
  -p 8000:8000 \
  $IMG_NAME

Verification#

Once the NIM is running on the air-gapped system, verify the deployment by checking the available models:

curl -X GET 'http://0.0.0.0:8000/v1/models'

Model-Free NIM with a Local Path to Model Files#

When using a model-free NIM with a local model path, no download occurs at startup. If the model directory is already on the air-gapped system, you can run without download-to-cache or create-model-store and without any API keys.

For more information on configuring the model-free NIM image, refer to Configuring the model-free NIM image.

  1. Set the container name and model repository path:

    export CONTAINER_NAME=nim-airgap
export MODEL_REPO=/path/to/model-repo
export NIM_MODEL_FREE_IMAGE=<model-free-nim-image>

  2. Run the NIM container with the local model path:

    docker run -it --rm --name="$CONTAINER_NAME" \
  --gpus all \
  --shm-size=16GB \
  -p 8000:8000 \
  -e NIM_PASSTHROUGH_ARGS="" \
  -e NIM_SERVED_MODEL_NAME=my-model \
  -e NIM_DISABLE_MODEL_DOWNLOAD=1 \
  -e NIM_MODEL_PATH=/model-repo \
  -v "$MODEL_REPO:/model-repo" \
  ${NIM_MODEL_FREE_IMAGE}

Mounted Path Permissions#

The container’s default user, nvs (GID 1000), might not be able to read files mounted from the host. In that case, the container can fail to start with a Permission denied error.

Grant the container user read access through the group that owns the model directory. Determine the directory’s group ID using stat -f %g /data/models/<model-directory>, then pass that value using Docker flag --group-add (for example, --group-add 1048). The model directory and its contents must grant that group read permission, and every parent directory in the mount path must grant it execute (traversal) permission.

Avoid chmod 777 /data/models/<model-directory>, which grants excessive access. Prefer granting only the required group permissions, such as chmod -R g+rX /data/models/<model-directory>, after confirming that the owning group is the intended one.

Verification#

Once the NIM is running on the air-gapped system, verify the deployment by checking the available models:

curl -X GET 'http://0.0.0.0:8000/v1/models'