Deploy NVIDIA RAG Blueprint on Kubernetes with Helm from the repository#

Use the following documentation to deploy the NVIDIA RAG Blueprint by using the helm chart from the repository.

The following are the core services that you install:

  • RAG server

  • Ingestor server

Prerequisites#

  1. Verify that you meet the prerequisites specified in prerequisites.

  2. Clone the RAG Blueprint Git repository to get access to the Helm chart source files.

  3. Verify that you have installed the NVIDIA NIM Operator. If not, install it by running the following code:

    helm repo add nvidia https://helm.ngc.nvidia.com/nvidia \
  --username='$oauthtoken' \
  --password=$NGC_API_KEY
helm repo update
helm install nim-operator nvidia/k8s-nim-operator -n nim-operator --create-namespace

    For more details, see instructions here.

  4. Install the ECK (Elastic Cloud on Kubernetes) operator. Elasticsearch is the default vector database; the ECK operator manages Elasticsearch on Kubernetes.

    helm repo add elastic https://helm.elastic.co
helm repo update
helm install elastic-operator elastic/eck-operator -n elastic-system --create-namespace

    For verification and Elasticsearch-specific Helm settings, see Vector database configuration.

Important

Consider the following before you deploy the RAG Blueprint:

  • Ensure that you have at least 200GB of available disk space per node for NIM model caches

  • First-time deployment takes 60-70 minutes as models download without visible progress indicators

For monitoring commands, refer to Deploy on Kubernetes with Helm - Prerequisites.

Deploy the RAG Helm chart from the repository#

If you are working directly with the source Helm chart, and you want to customize components individually, use the following procedure.

  1. Change directory to deploy/helm/ by running the following code.

    cd deploy/helm/

  2. Create a namespace for the deployment by running the following code.

    kubectl create namespace rag

  3. Configure Helm repo additions by editing and then running the following code.

    helm repo add nvidia-nim https://helm.ngc.nvidia.com/nim/nvidia/ --username='$oauthtoken' --password=$NGC_API_KEY
helm repo add nim https://helm.ngc.nvidia.com/nim/ --username='$oauthtoken' --password=$NGC_API_KEY
helm repo add nemo-microservices https://helm.ngc.nvidia.com/nvidia/nemo-microservices --username='$oauthtoken' --password=$NGC_API_KEY
helm repo add baidu-nim https://helm.ngc.nvidia.com/nim/baidu --username='$oauthtoken' --password=$NGC_API_KEY
helm repo add bitnami https://charts.bitnami.com/bitnami
helm repo add elastic https://helm.elastic.co
helm repo add seaweed https://seaweedfs.github.io/seaweedfs/helm
helm repo add otel https://open-telemetry.github.io/opentelemetry-helm-charts
helm repo add zipkin https://zipkin.io/zipkin-helm
helm repo add prometheus https://prometheus-community.github.io/helm-charts

  4. Update Helm chart dependencies by running the following code.

    helm dependency update nvidia-blueprint-rag

  5. Install the chart by running the following code.

    helm upgrade --install rag -n rag nvidia-blueprint-rag/ \
--set imagePullSecret.password=$NGC_API_KEY \
--set ngcApiSecret.password=$NGC_API_KEY

    Note

    Refer to NIM Model Profile Configuration for using non-default NIM LLM profile.

    For RTX PRO 6000 hardware, see the RTX PRO 6000 setup prerequisites in the Nemotron 3 Super deployment guide.

  6. Follow the remaining instructions in Deploy on Kubernetes with Helm (including verification examples that reflect the default Elasticsearch vector database; optional Milvus adds different pods and services—refer to Vector database configuration) for more information.