Multi-Cluster Installation

For a control plane installation on Kubernetes, configure:

1. Define the installation:

   • install.mode - Set to runai_control_plane_only to install the control plane only.
   • install.connectivity - Set to connected, for an installation with outbound internet access.

   install:
     mode: runai_control_plane_only
     connectivity: connected

2. Configure the registry:

   • registry.type - Set to ngc.
   • registry.tokenPath - For NVIDIA Run:ai Support-Entitled Release installations, provide the path to your NGC Personal Key. Open Source installations do not require registry credentials. See Registry Credentials.

   registry:
     type: ngc
     tokenPath: /path/to/ngc-token # Support-Entitled Release only

3. Define the control plane:

   • controlPlane.domain - The FQDN for the NVIDIA Run:ai UI and API. See Domains and TLS Certificates.
   • controlPlane.tls - The certificate and private key for the control plane FQDN, as certPath and keyPath. See Domains and TLS Certificates.
   • controlPlane.version - The control plane version to install. Omit it to install the latest supported version.
   • controlPlane.customHelmValues - Merges Helm values into the control plane chart, for settings not otherwise exposed by the installer. See Advanced Control Plane Configurations for the full set of keys.
     • supportEntitled - For NVIDIA Run:ai Support-Entitled Release, must be set to true.
     • telemetryService.enabled - For NVIDIA Run:ai Support-Entitled Release, must be set to true to enable telemetry.
     • telemetryService.telemetryAuth - Optional, Support-Entitled Release only. Associates NVIDIA Run:ai’s telemetry with your NCA ID - by default, telemetry is not associated with your organization. See Registry Credentials for how to generate the NGC Service API Key. Provide it one of two ways:
       • token: <SERVICE_API_KEY> - Sets the Service API Key directly; the chart creates and manages the Kubernetes secret for you. Stored in plaintext in the configuration file, so protect it accordingly, or use existingSecret instead.
       • existingSecret: <SECRET_NAME> / secretKey: <KEY_NAME> - References a secret you create yourself beforehand - the installer does not create or manage it.
     • Third-party services - Add the image pull secret, the external PostgreSQL connection, and the hardened images and settings of the control plane third-party services. See Gov-Ready Third-Party Services for the values. Gov-ready
     • global.fipsMode - Enables FIPS mode, so that the NVIDIA Run:ai services use only FIPS-validated cryptography. Set to only. Gov-ready

   controlPlane:
     enabled: true
     domain: <DOMAIN>
     tls:
       certPath: /path/to/cp-cert.pem
       keyPath: /path/to/cp-key.pem
     version: "" # Omit to install the latest supported version
     customHelmValues:
       supportEntitled: true # Support-Entitled Release only
       telemetryService: # Support-Entitled Release only
         enabled: true
         telemetryAuth: # Optional
           token: <SERVICE_API_KEY>
       # Gov-ready: add the image pull secret, external PostgreSQL, and hardened control plane values, see Gov-Ready Third-Party Services
       global:
         fipsMode: only # Gov-ready

4. Configure external access to the control plane:

   • externalConnection - Configures how traffic reaches the NVIDIA Run:ai control plane UI/API from outside the cluster, through a Kubernetes Ingress Controller or Gateway API implementation already running on your cluster. See Networking Setup.

   externalConnection:
     type: ingress
     ingress:
       className: <YOUR_INGRESS_CLASS>

The following example combines these settings to install the NVIDIA Run:ai control plane:

install:
  mode: runai_control_plane_only
  connectivity: connected
registry:
  type: ngc
  tokenPath: /path/to/ngc-token # Support-Entitled Release only
controlPlane:
  enabled: true
  domain: <DOMAIN>
  tls:
    certPath: /path/to/cp-cert.pem
    keyPath: /path/to/cp-key.pem
  version: ""
  customHelmValues:
    supportEntitled: true # Support-Entitled Release only
    # Gov-ready: add the image pull secret, external PostgreSQL, and hardened control plane values, see Gov-Ready Third-Party Services
    global:
      fipsMode: only # Gov-ready
externalConnection:
  type: ingress
  ingress:
    className: <YOUR_INGRESS_CLASS>

For a control plane installation on OpenShift, configure:

1. Define the installation:

   • install.mode - Set to runai_control_plane_only to install the control plane only.
   • install.connectivity - Set to connected, for an installation with outbound internet access.
   • install.kubernetesDistribution - Set to openshift.

   install:
     mode: runai_control_plane_only
     connectivity: connected
     kubernetesDistribution: openshift

2. Configure the registry:

   • registry.type - Set to ngc.
   • registry.tokenPath - For NVIDIA Run:ai Support-Entitled Release installations, provide the path to your NGC Personal Key. Open Source installations do not require registry credentials. See Registry Credentials.

   registry:
     type: ngc
     tokenPath: /path/to/ngc-token # Support-Entitled Release only

3. Define the control plane:

   • controlPlane.domain - The FQDN for the NVIDIA Run:ai UI and API. See Domains and TLS Certificates.
   • controlPlane.version - The control plane version to install. Omit it to install the latest supported version.
   • controlPlane.customHelmValues - Merges Helm values into the control plane chart, for settings not otherwise exposed by the installer. See Advanced Control Plane Configurations for the full set of keys.
     • supportEntitled - For NVIDIA Run:ai Support-Entitled Release, must be set to true.
     • telemetryService.enabled - For NVIDIA Run:ai Support-Entitled Release, must be set to true to enable telemetry.
     • telemetryService.telemetryAuth - Optional, Support-Entitled Release only. Associates NVIDIA Run:ai’s telemetry with your NCA ID - by default, telemetry is not associated with your organization. See Registry Credentials for how to generate the NGC Service API Key. Provide it one of two ways:
       • token: <SERVICE_API_KEY> - Sets the Service API Key directly; the chart creates and manages the Kubernetes secret for you. Stored in plaintext in the configuration file, so protect it accordingly, or use existingSecret instead.
       • existingSecret: <SECRET_NAME> / secretKey: <KEY_NAME> - References a secret you create yourself beforehand - the installer does not create or manage it.
     • Third-party services - Add the image pull secret, the external PostgreSQL connection, and the hardened images and settings of the control plane third-party services. See Gov-Ready Third-Party Services for the values. Gov-ready
     • global.fipsMode - Enables FIPS mode, so that the NVIDIA Run:ai services use only FIPS-validated cryptography. Set to only. Gov-ready

   controlPlane:
     enabled: true
     domain: <DOMAIN>
     version: "" # Omit to install the latest supported version
     customHelmValues:
       supportEntitled: true # Support-Entitled Release only
       telemetryService: # Support-Entitled Release only
         enabled: true
         telemetryAuth: # Optional
           token: <SERVICE_API_KEY>
       # Gov-ready: add the image pull secret, external PostgreSQL, and hardened control plane values, see Gov-Ready Third-Party Services
       global:
         fipsMode: only # Gov-ready

The following example combines these settings to install the NVIDIA Run:ai control plane on OpenShift:

install:
  mode: runai_control_plane_only
  connectivity: connected
  kubernetesDistribution: openshift
registry:
  type: ngc
  tokenPath: /path/to/ngc-token # Support-Entitled Release only
controlPlane:
  enabled: true
  domain: <DOMAIN>
  version: ""
  customHelmValues:
    supportEntitled: true # Support-Entitled Release only
    # Gov-ready: add the image pull secret, external PostgreSQL, and hardened control plane values, see Gov-Ready Third-Party Services
    global:
      fipsMode: only # Gov-ready

For a cluster installation on Kubernetes, configure:

1. Define the installation:

   • install.mode - Set to runai_cluster_only to install the cluster component only.
   • install.connectivity - Set to connected, for an installation with outbound internet access.

   install:
     mode: runai_cluster_only
     connectivity: connected

2. Configure the registry:

   • registry.type - Set to ngc.
   • registry.tokenPath - For NVIDIA Run:ai Support-Entitled Release installations, provide the path to your NGC Personal Key. Open Source installations do not require registry credentials. See Registry Credentials.

   registry:
     type: ngc
     tokenPath: /path/to/ngc-token # Support-Entitled Release only

3. Define the cluster:

   • cluster.name - The name of the NVIDIA Run:ai cluster as it appears in the control plane.
   • cluster.domain - The FQDN for this cluster. See Domains and TLS Certificates. Required when the cluster component is installed on a different Kubernetes cluster than the control plane. The installer also uses this domain to reach the control plane when registering the cluster.
   • cluster.controlPlaneUrl - The control plane’s FQDN, reachable over HTTPS from this cluster.
   • cluster.controlPlaneCredentials - Identifies this cluster to the control plane. Use the control plane administrator username and password generated during control plane installation - the installer uses these to register the cluster and fetch its bootstrap token automatically, in the same run. See Retrieve Control Plane Credentials.
   • cluster.workloadRouting - host (the default) or path. This determines which certificate you need for cluster.tls below.
   • cluster.tls - A certificate and private key for the cluster domain, as certPath and keyPath. See Domains and TLS Certificates for which certificate you need.
   • cluster.version - The cluster version to install. Omit it to install the latest supported version.
   • cluster.customHelmValues - Merges Helm values into the cluster chart, for settings not otherwise exposed by the installer. See Advanced Cluster Configurations for the full set of keys.
     • Third-party services - Add the image pull secret and the hardened images of the cluster third-party services. See Gov-Ready Third-Party Services for the values. Gov-ready
     • global.fipsMode - Enables FIPS mode, so that the NVIDIA Run:ai services use only FIPS-validated cryptography. Set to only. Gov-ready

   cluster:
     enabled: true
     name: <CLUSTER_NAME>
     domain: <CLUSTER_DOMAIN>
     workloadRouting: host # Default; determines the certificate cluster.tls needs
     controlPlaneUrl: https://<DOMAIN>
     controlPlaneCredentials:
       username: <ADMIN_EMAIL>
       passwordPath: /path/to/cp-password
     tls:
       certPath: /path/to/cluster-a-cert.pem
       keyPath: /path/to/cluster-a-key.pem
     version: "" # Omit to install the latest supported version
     customHelmValues:
       # Gov-ready: add the image pull secret and hardened cluster values, see Gov-Ready Third-Party Services
       global:
         fipsMode: only # Gov-ready

4. Enable the prerequisites:

   • prerequisites.gpuOperator - Required for every installation. See Prerequisites and Features for more details.
     • mode - Set to install.
     • version - Omit to install the latest version, or see Support Matrix for the supported range if you want to set it explicitly. GPU fractions (memory fractions, dynamic memory fractions, or compute modes) require 26.7.1 or later, installed automatically when omitted.
     • customHelmValues - Merges Helm values into the GPU Operator’s own Helm chart. Set driver.version to r615 or later if you plan to use GPU fractions, since GPU Operator 26.7.1 does not default to this driver version. Set cdi.nriPluginEnabled to false, since GPU memory fractions do not function while the NRI plugin is enabled. See GPU Memory Fractions.
     • GPU Operator values - Add the Helm values required for a Gov-ready GPU Operator to customHelmValues. See Install the Gov-Ready GPU Operator for the values. Gov-ready
   • prerequisites.prometheus - Required for every installation. See Prerequisites and Features for more details.
     • mode - Set to install.
     • version - Omit to install the latest version, or see Support Matrix for the supported range.
     • customHelmValues - Merges Helm values into Prometheus’s own Helm chart.

   prerequisites:
     gpuOperator:
       mode: install
       version: ""
       customHelmValues: {} # Gov-ready: add the GPU Operator Helm values
     prometheus:
       mode: install
       version: ""
       customHelmValues: {}
   Features

   The minimum configuration does not include features such as Knative, Kubeflow, MPI Operator, LWS, NIM Operator, Dynamo, or Grove. These are skipped by default.

   If your workloads require any of these features, add the corresponding features configuration. See Prerequisites and Features.

   Some features require additional networking configuration. For example, externally exposed standard inference with Knative requires an additional inference wildcard FQDN and TLS certificate. See Networking Setup.

5. Configure external access to the cluster:

   • externalConnection - Configures how traffic reaches the NVIDIA Run:ai cluster endpoint from outside the cluster, through a Kubernetes Ingress Controller or Gateway API implementation already running on your cluster. This is separate from cluster.workloadRouting, which controls how the workloads you submit are exposed. See Networking Setup.

   externalConnection:
     type: ingress
     ingress:
       className: <YOUR_INGRESS_CLASS>

The following example combines these settings to install the NVIDIA Run:ai cluster component on a separate Kubernetes cluster, and has the installer register it with the control plane using the control plane administrator username and password:

install:
  mode: runai_cluster_only
  connectivity: connected
registry:
  type: ngc
  tokenPath: /path/to/ngc-token # Support-Entitled Release only
cluster:
  enabled: true
  name: <CLUSTER_NAME>
  domain: <CLUSTER_DOMAIN>
  workloadRouting: host
  controlPlaneUrl: https://<DOMAIN>
  controlPlaneCredentials:
    username: <ADMIN_EMAIL>
    passwordPath: /path/to/cp-password
  tls:
    certPath: /path/to/cluster-a-cert.pem
    keyPath: /path/to/cluster-a-key.pem
  version: ""
  customHelmValues:
    # Gov-ready: add the image pull secret and hardened cluster values, see Gov-Ready Third-Party Services
    global:
      fipsMode: only # Gov-ready
prerequisites:
  gpuOperator:
    mode: install
    version: ""
    customHelmValues: {} # Gov-ready: add the GPU Operator Helm values
  prometheus:
    mode: install
    version: ""
    customHelmValues: {}
externalConnection:
  type: ingress
  ingress:
    className: <YOUR_INGRESS_CLASS>

For a cluster installation on OpenShift, configure:

1. Define the installation:

   • install.mode - Set to runai_cluster_only to install the cluster component only.
   • install.connectivity - Set to connected, for an installation with outbound internet access.
   • install.kubernetesDistribution - Set to openshift.

   install:
     mode: runai_cluster_only
     connectivity: connected
     kubernetesDistribution: openshift

2. Configure the registry:

   • registry.type - Set to ngc.
   • registry.tokenPath - For NVIDIA Run:ai Support-Entitled Release installations, provide the path to your NGC Personal Key. Open Source installations do not require registry credentials. See Registry Credentials.

   registry:
     type: ngc
     tokenPath: /path/to/ngc-token # Support-Entitled Release only

3. Define the cluster:

   • cluster.name - The name of the NVIDIA Run:ai cluster as it appears in the control plane.
   • cluster.domain - The FQDN for this cluster. See Domains and TLS Certificates. Required when the cluster component is installed on a different OpenShift cluster than the control plane. The installer also uses this domain to reach the control plane when registering the cluster.
   • cluster.controlPlaneUrl - The control plane’s FQDN, reachable over HTTPS from this cluster.
   • cluster.controlPlaneCredentials - Identifies this cluster to the control plane. Use the control plane administrator username and password generated during control plane installation - the installer uses these to register the cluster and fetch its bootstrap token automatically, in the same run. See Retrieve Control Plane Credentials.
   • cluster.version - The cluster version to install. Omit it to install the latest supported version.
   • cluster.customHelmValues - Merges Helm values into the cluster chart, for settings not otherwise exposed by the installer. See Advanced Cluster Configurations for the full set of keys.
     • Third-party services - Add the image pull secret and the hardened images of the cluster third-party services. See Gov-Ready Third-Party Services for the values. Gov-ready
     • global.fipsMode - Enables FIPS mode, so that the NVIDIA Run:ai services use only FIPS-validated cryptography. Set to only. Gov-ready

   cluster:
     enabled: true
     name: <CLUSTER_NAME>
     domain: <CLUSTER_DOMAIN>
     controlPlaneUrl: https://<DOMAIN>
     controlPlaneCredentials:
       username: <ADMIN_EMAIL>
       passwordPath: /path/to/cp-password
     version: "" # Omit to install the latest supported version
     customHelmValues:
       # Gov-ready: add the image pull secret and hardened cluster values, see Gov-Ready Third-Party Services
       global:
         fipsMode: only # Gov-ready

4. Enable the prerequisites:

   • prerequisites.gpuOperator - Required for every installation. See Prerequisites and Features for more details.
     • mode - Set to install. The installer installs the GPU Operator through OperatorHub (OLM).
     • version - Omit to install the latest version, or see Support Matrix for the supported range if you want to set it explicitly. GPU fractions (memory fractions, dynamic memory fractions, or compute modes) require 26.7.1 or later, installed automatically when omitted.
     • openshift.customResource - Merges values into the GPU Operator’s ClusterPolicy. customHelmValues is not supported on OpenShift. Set driver.version to r615 or later if you plan to use GPU fractions, since GPU Operator 26.7.1 does not default to this driver version. See GPU Memory Fractions and Per-Component OpenShift Fields.
     • GPU Operator values - Add the values required for a Gov-ready GPU Operator to openshift.customResource. See Install the Gov-Ready GPU Operator for the values. Gov-ready
   • prerequisites.prometheus - See Prerequisites and Features for more details.
     • mode - Set to skip, to use the cluster’s built-in monitoring stack. install is rejected on OpenShift.

   prerequisites:
     gpuOperator:
       mode: install
       version: ""
       openshift:
         customResource: # Gov-ready: add the GPU Operator values
           driver:
             version: <DRIVER_VERSION>
     prometheus:
       mode: skip
   Features

   The minimum configuration does not include features such as Knative, Kubeflow, MPI Operator, LWS, NIM Operator, Dynamo, or Grove. These are skipped by default.

   If your workloads require any of these features, add the corresponding features configuration. See Prerequisites and Features.

   Some features need OpenShift-specific configuration. For example, Knative, required for standard inference, installs through the OpenShift Serverless operator and requires cluster.inference.wildcardDomain, but no wildcard TLS certificate. Features that install through OperatorHub, such as Knative and LWS, are customized through openshift.customResource rather than customHelmValues. See Per-Component OpenShift Fields.

The following example combines these settings to install the NVIDIA Run:ai cluster component on a separate OpenShift cluster, and has the installer register it with the control plane using the control plane administrator username and password:

install:
  mode: runai_cluster_only
  connectivity: connected
  kubernetesDistribution: openshift
registry:
  type: ngc
  tokenPath: /path/to/ngc-token # Support-Entitled Release only
cluster:
  enabled: true
  name: <CLUSTER_NAME>
  domain: <CLUSTER_DOMAIN>
  controlPlaneUrl: https://<DOMAIN>
  controlPlaneCredentials:
    username: <ADMIN_EMAIL>
    passwordPath: /path/to/cp-password
  version: ""
  customHelmValues:
    # Gov-ready: add the image pull secret and hardened cluster values, see Gov-Ready Third-Party Services
    global:
      fipsMode: only # Gov-ready
prerequisites:
  gpuOperator:
    mode: install
    version: ""
    openshift:
      customResource: # Gov-ready: add the GPU Operator values
        driver:
          version: <DRIVER_VERSION>
  prometheus:
    mode: skip