> This page is for Self-Hosted, version v3.0 (default).
> For other versions, use one of these documentation indexes:
> - v3.0 (default): https://docs.nvidia.com/run-ai/self-hosted/v3.0/llms.txt
> - v2.26: https://docs.nvidia.com/run-ai/self-hosted/v2.26/llms.txt
> - v2.25: https://docs.nvidia.com/run-ai/self-hosted/v2.25/llms.txt
> - v2.24: https://docs.nvidia.com/run-ai/self-hosted/v2.24/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/run-ai/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/run-ai/_mcp/server.

# High Availability

This guide outlines the best practices for configuring the NVIDIA Run:ai platform to ensure high availability and maintain service continuity during system failures or under heavy load. The goal is to reduce downtime and eliminate single points of failure by leveraging Kubernetes best practices alongside NVIDIA Run:ai specific configuration options. The NVIDIA Run:ai platform relies on two fundamental high availability strategies:

* **Use of system nodes** - Assigning multiple dedicated nodes for critical system services ensures control, resource isolation, and enables system-level scaling.
* **Replication of core and third-party services** - Configuring multiple replicas of essential services, including both platform and third-party components, distributes workloads and reduces single points of failure. If a component fails on one node, requests can seamlessly route to another instance.

## System Nodes

The NVIDIA Run:ai platform allows you to dedicate specific nodes (system nodes) exclusively for core platform services. This approach provides improved operational isolation and easier resource management.

Ensure that at least **three system nodes** are configured to support high availability. If you use only a single node for core services, horizontally scaled components will not be distributed, resulting in a single point of failure. See [NVIDIA Run:ai system nodes](/run-ai/self-hosted/documentation/installation-and-configuration/config/node-roles) for more details. This practice applies to both the NVIDIA Run:ai cluster and control plane.

## Service Replicas <a href="#undefined" id="undefined"></a>

### Control Plane Service Replicas

The NVIDIA Run:ai control plane runs in the `runai-backend` namespace and consists of multiple Kubernetes [Deployments](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/) and [StatefulSets](https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/). To achieve high availability, it is recommended to configure multiple replicas during installation or upgrade using Helm flags.

In addition, the control plane supports autoscaling for certain services to handle variable load and improve system resiliency. Autoscaling can be enabled or configured during installation or upgrade using Helm flags.

#### Deployments

By default, NVIDIA Run:ai deployments are deployed with a single replica. To achieve high availability, it is recommended to configure multiple replicas for core NVIDIA Run:ai services. For more information, see [Advanced control plane configuration](/run-ai/self-hosted/documentation/installation-and-configuration/config/advanced-control-plane-configuration).

#### StatefulSets

NVIDIA Run:ai uses the following third-party components which are managed as Kubernetes StatefulSets. For more information, see [Advanced control plane configurations](/run-ai/self-hosted/documentation/installation-and-configuration/config/advanced-control-plane-configuration):

* **PostgreSQL** - The internal PostgreSQL cannot be scaled horizontally. To connect NVIDIA Run:ai to an external PostgreSQL service which can be configured for high availability, see [Using an External PostgreSQL Instance](/run-ai/self-hosted/documentation/installation-and-configuration/system-requirements/third-party-services/standard-third-party-services#using-an-external-postgresql-instance).
*   **Thanos** - To enable Thanos autoscaling, use the following NVIDIA Run:ai control plane helm flags:

    ```bash
    --set thanos.query.autoscaling.enabled=true \
    --set thanos.query.autoscaling.maxReplicas=2 \
    --set thanos.query.autoscaling.minReplicas=2 
    ```
*   **Keycloak** **-** By default, Keycloak sets a minimum of 3 pods and will scale to more on transaction load. To scale Keycloak, use the following NVIDIA Run:ai control plane helm flags:

    ```bash
    --set keycloakx.autoscaling.enabled=true
    ```

### Cluster Services Replicas <a href="#nvidia-run-ai-services-replicas" id="nvidia-run-ai-services-replicas"></a>

By default, NVIDIA Run:ai cluster services are deployed with a single replica. To achieve high availability, it is recommended to configure multiple replicas for core NVIDIA Run:ai services. For more information, see [NVIDIA Run:ai services replicas](/run-ai/self-hosted/documentation/installation-and-configuration/config/advanced-cluster-configuration#nvidia-run-ai-services-replicas).