Air-Gapped Environments#

NVIDIA NIM for large language models (LLMs) supports serving models in an air-gapped system (also known as air wall, air-gapping, or disconnected network). In an air-gapped system, you can run a NIM with no internet connection and with no connection to the NGC registry or Hugging Face Hub.

You have two options for air-gapped deployment: accessing NGC through proxy and mirrored local model registries.

Proxy Support#

If your cluster has a proxy that requires custom certificates, you can add them in a config map and mount them into the NIM cache job. You can use self-signed or custom CA certificates.

  1. Create a ConfigMap with the certificates:

    $ kubectl create configmap -n nim-service ca-certs --from-file=<path-to-cert-file-1> --from-file=<path-to-cert-file-2>

  2. When you create the NIM Cache job and NIM Service deployment, specify the name of the config map and the path to mount them in the container, and specify the proxy information:

    spec:
  proxy:
    httpsProxy: "https://<node-ip>:<port>"                 # address of a proxy server that should be used for outbound HTTPS requests
    httpProxy: "http://<node-ip>:<port>"                   # address of a proxy server that should be used for outbound HTTP requests
    noProxy: "http://example.com, http://example2.com"     # comma-separated list of domain names, IP addresses, or IP ranges for which proxying should be bypassed
    certConfigMap: "ca-certs"                              # the ConfigMap name that holds your CA certificates

    Note

    Refer to Prerequisites for more information on using NIM Cache.

  3. For OpenShift Container Platform installations, you must enable certificate injection.

    1. Create a file, such as ca-inject.yaml, with contents like the following example:

      apiVersion: v1
data: {}
kind: ConfigMap
metadata:
  labels:
    config.openshift.io/inject-trusted-cabundle: "true"
  name: ca-inject-cm
  namespace: nim-service

    2. Apply the manifest:

      $ oc apply -n nim-service -f ca-inject.yaml

For more information, refer to Certificate injection using Operators in the OpenShift Container Platform documentation.

On this page