Air Gap Deployment#
NVIDIA NIM for Vision Language Models (VLMs) supports serving models in an air gap system (also known as air wall, air-gapping, or disconnected network). In an air gap system, you can run a NIM with no internet connection, and with no connection to the NGC registry.
Before you use this documentation, review all prerequisites and instructions in Get Started with NIM, and Serving models from local assets.
For air gap deployment, see offline cache.
Air Gap Deployment (offline cache option)#
NIM supports serving models in an air gap system. If NIM detects a previously loaded profile in the cache, it serves that profile from the cache.
Prerequisites#
Before deploying in an air gap environment, you need to download the model profiles on a system with internet access. Follow these steps:
Step 1: Set up environment variables on the connected system
# Choose a container name for bookkeeping
export CONTAINER_NAME=vlm-download-container
# Set your NIM image name (replace with actual values from NGC)
export IMG_NAME=<your-nim-image-name>
# Choose a path on your system to cache the downloaded models
export LOCAL_NIM_CACHE=~/.cache/nim
mkdir -p "$LOCAL_NIM_CACHE"
Step 2: Download models to the cache on the connected system
First, launch the NIM container to access the download utilities:
docker run -it --rm --name=$CONTAINER_NAME \
--runtime=nvidia \
--gpus all \
--shm-size=16GB \
-e NGC_API_KEY=$NGC_API_KEY \
-v "$LOCAL_NIM_CACHE:/opt/nim/.cache" \
-u $(id -u) \
$IMG_NAME bash
Inside the container, list available profiles and download the one you need:
# List available model profiles
list-model-profiles
# download a profile matching the air-gapped profile (replace with actual profile ID)
download-to-cache --profile 09e2f8e68f78ce94bf79d15b40a21333cea5d09dbe01ede63f6c957f4fcfab7b
Exit the container after the download completes:
exit
Air-Gapped System Deployment#
Step 3: Transfer the cache to the air-gapped system
Copy the entire cache directory from the connected system to your air-gapped system.
Step 4: Set up environment variables on the air-gapped system
# Choose a container name for bookkeeping
export CONTAINER_NAME=vlm-airgap-container
# Set your NIM image name (same as used for download)
export IMG_NAME=<your-nim-image-name>
# Path to the transferred cache directory on the air-gapped system
export AIR_GAP_NIM_CACHE=~/.cache/air-gap-nim-cache
# Ensure the directory exists and copy the transferred cache
mkdir -p "$AIR_GAP_NIM_CACHE"
cp -r <path-to-transferred-cache>/* "$AIR_GAP_NIM_CACHE"
Step 5: Launch NIM on the air-gapped system
# replace NIM_MODEL_PROFILE value with actual profile ID
docker run -it --rm --name=$CONTAINER_NAME \
--runtime=nvidia \
--gpus all \
--shm-size=16GB \
-e NIM_MODEL_PROFILE=09e2f8e68f78ce94bf79d15b40a21333cea5d09dbe01ede63f6c957f4fcfab7b \
-v "$AIR_GAP_NIM_CACHE:/opt/nim/.cache" \
-u $(id -u) \
-p 8000:8000 \
$IMG_NAME
Verification#
Once the NIM is running on the air-gapped system, verify the deployment by checking the available models:
curl -X GET 'http://0.0.0.0:8000/v1/models'
You should see output similar to what’s described in the Run Inference section of the Get Started guide.