Preparations

This page prepares your environment for installing the NVIDIA Run:ai control plane and cluster: setting up your installer machine, creating the namespaces and TLS/CA secrets each install needs, and retrieving software artifacts and registry access.

System Requirements

Complete the System Requirements section before starting. It covers:

• Infrastructure Requirements - Hardware, operating system, Kubernetes distribution, storage, and permissions required in your Kubernetes environment.
• Networking Setup - Load balancer and ingress requirements, FQDNs, TLS certificates and private keys, and a local CA chain if your certificates are privately signed. The exact requirements depend on your deployment model, routing configuration, and the workload types you plan to support.
• Third-Party Services - The third-party services deployed as part of the installation.
• External Access to Critical Services - The inbound and outbound connectivity required to reach those services and NVIDIA registries.

Your deployment model determines how the installation steps are applied. In a same-cluster deployment, the control plane and NVIDIA Run:ai cluster are installed on the same Kubernetes cluster. In a multi-cluster deployment, control plane setup is completed once, and cluster setup is repeated for each NVIDIA Run:ai cluster.

General Setup

This section covers what you set up once, before installing anything: your installer machine and the registry credentials needed to pull NVIDIA Run:ai images. None of it depends on your deployment model or on whether you are installing the control plane, the cluster, or both.

Installer Machine

The machine you run Helm from needs:

• At least 50GB of free space
• Docker installed
• Helm 3.14 or later

Registry Credentials

NVIDIA Run:ai Support-Entitled Release installations require an NGC Personal Key to authenticate to the NVIDIA Run:ai container registry (nvcr.io), for connected installs. Generate a Personal Key from the NGC Personal Keys page.

Personal Keys begin with nvapi-. NGC authenticates using the literal username $oauthtoken with the key as the password. The Personal Key is used during installation, to pull the NVIDIA Run:ai container images. Use this key to create the runai-reg-creds secret described in Software Artifacts below.

For connected installs, NVIDIA Run:ai’s telemetry for Support-Entitled Release installations is not associated with your organization by default. To associate it with your NCA ID, generate an NGC Service API Key. In the Service field, select both NVIDIA Run:ai Self-Hosted and NGC Catalog. The Personal Key is used during installation. The Service API Key is used at runtime, after installation, by the NVIDIA Run:ai control plane to report telemetry. Pass the resulting key as telemetryService.telemetryAuth.token during installation, or reference a secret you create yourself beforehand via telemetryService.telemetryAuth.existingSecret/secretKey. Create the runai-backend namespace first, as described in Control Plane Namespace, then create the secret:

kubectl create secret generic <SECRET_NAME> \
  --namespace runai-backend \
  --from-literal=<KEY_NAME>=<SERVICE_API_KEY>

Control Plane Setup

This section covers what you set up for the NVIDIA Run:ai control plane: its namespace and TLS certificate and local CA secrets. A deployment has only one control plane, whether same-cluster or multi-cluster. Set it up on the control plane’s own Kubernetes cluster - the same Kubernetes cluster as the first NVIDIA Run:ai cluster in a same-cluster deployment, or a dedicated Kubernetes cluster in a multi-cluster deployment. See Deployment Model.

Control Plane Namespace

The NVIDIA Run:ai control plane uses a namespace or project (OpenShift) called runai-backend. Use the following to create the namespace/project:

kubectl create namespace runai-backend

Control Plane Certificate

This certificate secures the NVIDIA Run:ai control plane FQDN you prepared in FQDN and DNS. Before creating the secret below, see TLS Certificates to confirm which certificate you need and why.

• Kubernetes - You must have a TLS certificate that is associated with the FQDN for HTTPS access. Create a Kubernetes Secret named runai-backend-tls in the runai-backend namespace and include the path to the TLS --cert and its corresponding private --key by running the following:

  • Replace /path/to/fullchain.pem with the actual path to your TLS certificate.
  • Replace /path/to/private.pem with the actual path to your private key.

  kubectl create secret tls runai-backend-tls -n runai-backend \
    --cert /path/to/fullchain.pem \
    --key /path/to/private.pem

• OpenShift - NVIDIA Run:ai uses the OpenShift default Ingress router for serving. The TLS certificate configured for this router must be issued by a trusted CA. For more details, see the OpenShift documentation on configuring certificates.

Control Plane Local CA

If your TLS certificates are signed by a private or local CA rather than a publicly trusted one, you need to provide that CA’s certificate chain as part of installation. See Local Certificate Authority (CA) for details.

1. Add the public key to the runai-backend namespace:

kubectl -n runai-backend create secret generic runai-ca-cert \
    --from-file=runai-ca.pem=<ca_bundle_path>

2. When installing the control plane, make sure the following flag is added to the helm command --set global.customCA.enabled=true. See Install control plane.

Cluster Setup

This section covers what you set up for each NVIDIA Run:ai cluster: its namespace, TLS certificates, host-based routing, and local CA secrets. Set it up on the Kubernetes cluster where the NVIDIA Run:ai cluster will run - the same Kubernetes cluster as the control plane in a same-cluster deployment, or a separate Kubernetes cluster per NVIDIA Run:ai cluster in a multi-cluster deployment. In a multi-cluster deployment, repeat this section once per cluster. See Deployment Model.

Cluster Namespace

The NVIDIA Run:ai cluster must be installed in a namespace or project (OpenShift) called runai. Use the following to create the namespace/project:

kubectl create ns runai

Pod Security Admission

NVIDIA Run:ai supports restricted policy for Pod Security Admission (PSA) on OpenShift only. Other Kubernetes distributions are only supported with privileged policy.

For NVIDIA Run:ai on OpenShift to run with PSA restricted policy:

• Label the runai namespace as described in Pod Security Admission with the following labels:

  pod-security.kubernetes.io/audit=privileged
  pod-security.kubernetes.io/enforce=privileged
  pod-security.kubernetes.io/warn=privileged

• The workloads submitted through NVIDIA Run:ai should comply with the restrictions of PSA restricted policy. This can be enforced using Policies.

Cluster TLS Certificates

These certificates secure the NVIDIA Run:ai cluster, workspaces, training, distributed inference, and standard inference FQDNs you prepared in FQDN and DNS. Before creating the secrets below, see TLS Certificates to confirm which certificates you need and why. The workspaces, training, and distributed inference wildcard certificate is only needed for the default host-based routing - see Host-Based Routing (Ingress Resource) below for the matching Ingress setup. If you are using path-based routing instead, skip that certificate.

Cluster Domain Certificates (Single-Domain)

• Kubernetes - To enable secure communication between the NVIDIA Run:ai control plane and the cluster, configure a TLS certificate associated with the cluster’s main domain (e.g. runai.mycorp.local). This certificate should be stored as a secret named runai-cluster-domain-tls-secret in the runai namespace.

  • Replace /path/to/fullchain.pem with the actual path to your TLS certificate.
  • Replace /path/to/private.pem with the actual path to your private key.

  kubectl create secret tls runai-cluster-domain-tls-secret -n runai \
    --cert /path/to/fullchain.pem \
    --key /path/to/private.pem

• OpenShift - NVIDIA Run:ai uses the OpenShift default Ingress router for serving. The TLS certificate configured for this router must be issued by a trusted CA. For more details, see the OpenShift documentation on configuring certificates.

Workspaces, Training, and Distributed Inference Wildcard Certificate

• Kubernetes - To allow secure access to workspace, training, and distributed inference workloads via subdomains, configure a wildcard TLS certificate that matches the cluster domain (e.g. *.runai.mycorp.local). This certificate should be stored as a secret named runai-cluster-domain-star-tls-secret in the runai namespace.

  • Replace /path/to/fullchain.pem with the actual path to your TLS certificate.

  • Replace /path/to/private.pem with the actual path to your private key.

    kubectl create secret tls runai-cluster-domain-star-tls-secret -n runai \
      --cert /path/to/fullchain.pem \
      --key /path/to/private.pem
• OpenShift - A wildcard TLS certificate for Workspace and Training workloads is not required. OpenShift Routes handle TLS termination for workspace and training endpoints using the platform’s built-in routing and certificate management.

Standard Inference Wildcard Certificate

• Kubernetes - To securely expose inference services over HTTPS, configure a wildcard TLS certificate for the inference domain (e.g. *.runai-inference.mycorp.local). This certificate should be stored as a secret named runai-cluster-inference-tls-secret in the knative-serving namespace.

  • Replace /path/to/fullchain.pem with the actual path to your TLS certificate.
  • Replace /path/to/private.pem with the actual path to your private key.

  kubectl create secret tls runai-cluster-inference-tls-secret -n knative-serving \
      --cert /path/to/fullchain.pem \
      --key /path/to/private.pem

• OpenShift - A wildcard TLS certificate for Inference workloads is not required. OpenShift Routes handle TLS termination for inference endpoints using the platform’s built-in routing and certificate management.

Host-Based Routing (Ingress Resource)

Host-based routing binds together the configured domains (FQDN), TLS certificates, and ingress rules to expose workloads externally. With NVIDIA Run:ai host-based routing, workloads are exposed using subdomains, so each workload is assigned its own URL. For example:

https://<project>-<workload>.<CLUSTER_URL>

Host-based routing relies on:

• The FQDN structure defined earlier
• The TLS certificates configured for those domains

1. Ensure that:

   • Wildcard DNS records are configured (see FQDN and DNS)
   • TLS certificates are created and stored as Kubernetes secrets (see Cluster TLS Certificates above)

2. Create the ingress resource

   apiVersion: networking.k8s.io/v1
   kind: Ingress
   metadata:
     name: runai-cluster-domain-star-ingress
     namespace: runai
   spec:
     rules:
     - host: '*.<CLUSTER_URL>'
     tls:
     - hosts:
       - '*.<CLUSTER_URL>'
       secretName: runai-cluster-domain-star-tls-secret

3. Run the following:

   kubectl apply -f <filename>

Cluster Local CA

If your TLS certificates are signed by a private or local CA rather than a publicly trusted one, you need to provide that CA’s certificate chain as part of installation. See Local Certificate Authority (CA) for details.

1. Add the public key to the required namespace:

kubectl -n runai create secret generic runai-ca-cert \
    --from-file=runai-ca.pem=<ca_bundle_path>
kubectl label secret runai-ca-cert -n runai run.ai/cluster-wide=true run.ai/name=runai-ca-cert --overwrite

2. When installing the cluster, make sure the following flag is added to the helm command --set global.customCA.enabled=true. See Install the cluster.

Software Artifacts

This section describes how to prepare the software artifacts required for installing the NVIDIA Run:ai control plane and cluster. The steps depend on:

• Platform distribution - Kubernetes or OpenShift
• On-premise environment type - connected or air-gapped (i.e. not connected)

Kubernetes

kubectl create secret docker-registry runai-reg-creds \
--docker-server=https://nvcr.io \
--docker-username='$oauthtoken' \
--docker-password=<NGC_API_KEY> \
--docker-email=<EMAIL> \
--namespace=runai-backend

cat > airgap-load.yaml <<'EOF'
settings:
  bundlePath: runai-airgap-<RUNAI_VERSION>.tar.gz
  workingDir: ./airgap-work
  valuesDir: ./airgap-values
registry:
  url: <REGISTRY_URL>
  username: <REGISTRY_USERNAME>
  passwordPath: <REGISTRY_PASSWORD_PATH>
EOF
./runai-installer airgap load \
  --config airgap-load.yaml

OpenShift

oc create secret docker-registry runai-reg-creds \
--docker-server=https://nvcr.io \
--docker-username='$oauthtoken' \
--docker-password=<NGC_API_KEY> \
--docker-email=<EMAIL> \
--namespace=runai-backend

cat > airgap-load.yaml <<'EOF'
settings:
  bundlePath: runai-airgap-<RUNAI_VERSION>.tar.gz
  workingDir: ./airgap-work
  valuesDir: ./airgap-values
registry:
  url: <REGISTRY_URL>
  username: <REGISTRY_USERNAME>
  passwordPath: <REGISTRY_PASSWORD_PATH>
EOF
./runai-installer airgap load \
  --config airgap-load.yaml

Next Steps

With your environment prepared, continue to Prerequisites and Features to install the GPU Operator, Prometheus, and any workload-specific features (such as distributed training or inference) the NVIDIA Run:ai cluster needs.