Deploy NeMo Evaluator Using Parent Helm Chart#

You can deploy the NeMo Evaluator microservice using the parent Helm chart, which manages all required dependencies automatically. This is the recommended approach for most users.

By default, this deployment enables only the Evaluator and Data Store microservices, which are the minimal required dependencies for Evaluator. All other microservices are explicitly disabled.

Why Use the Parent Chart?

Dependency Management: The parent chart ensures all required services are installed and networked correctly.
Simplicity: One command, one values file, minimal manual configuration.
Support: This is the officially supported and tested deployment method.

Prerequisites#

Dependencies

Kubernetes cluster with Helm installed
NVIDIA GPU Cloud (NGC) API key for pulling images
(Optional) External services if not using the chart-provided dependencies:
- NIM: Used for model inference.
- NeMo Data Store
- Argo Workflows: Used for Evaluation job orchestration.
- Milvus: Used for Retriever and RAG evaluations.
- PostgreSQL: The persistent data store for NeMo Evaluator.

Kubernetes

Create NGC Image pull credentials. This includes:
- A generic secret containing your NGC API key, referenced by evaluator.existingSecret
- An image pull secret, referenced by evaluator.existingImagePullSecret

Download the Parent Helm Chart#

Download the NeMo Microservices Helm Chart.

helm fetch https://helm.ngc.nvidia.com/nvidia/nemo-microservices/charts/nemo-microservices-helm-chart-${parent_helm_chart_latest_version}.tgz \
   --username='$oauthtoken' \
   --password=$NGC_API_KEY

Prepare Your Values File#

Create a values-evaluator-only.yaml file to enable only the Evaluator and its required dependencies. For more advanced scenarios (for example, enabling Milvus, Argo, Ingress), see the advanced example below.

Minimal (Evaluator Only)

existingSecret: ngc-api # This value depends on your secret name
existingImagePullSecret: nvcrimagepullsecret # This value depends on your secret name

evaluator:
  enabled: true
  milvus:
    enabled: false
  argoServiceAccount:
    create: true
    name: workflow-executor
data-store:
  enabled: true

# Disable all other microservices
entity-store:
  enabled: false
nim:
  enabled: false
customizer:
  enabled: false
guardrails:
  enabled: false
deployment-management:
  enabled: false
nim-proxy:
  enabled: false
nemo-operator:
  enabled: false
nim-operator:
  enabled: false
studio-ui:
  enabled: false
studio-api:
  enabled: false
volcano:
  enabled: false
ingress:
  enabled: true

Advanced (Milvus, Argo, Ingress)

existingSecret: ngc-api # This value depends on your secret name
existingImagePullSecret: nvcrimagepullsecret # This value depends on your secret name

evaluator:
  enabled: true
  milvus:
    enabled: true
    standalone:
      persistence:
        enabled: true
        persistentVolumeClaim:
          size: 50Gi
          storageClass: "standard"
  argoServiceAccount:
    create: true
    name: workflow-executor
data-store:
  enabled: true

# Disable all other microservices
entity-store:
  enabled: false
nim:
  enabled: false
customizer:
  enabled: false
guardrails:
  enabled: false
deployment-management:
  enabled: false
nim-proxy:
  enabled: false
nemo-operator:
  enabled: false
nim-operator:
  enabled: false
studio-ui:
  enabled: false
studio-api:
  enabled: false
volcano:
  enabled: false
ingress:
  enabled: true

If you want to use external services for any dependency, set its enabled value to false and provide the appropriate external endpoint in your values file.

Install with Helm#

Run the following command to install the platform with your custom values file:

helm install nemo-platform \
  ./platform \
  -f values-evaluator-only.yaml \
  -f custom-values.yaml # optional

Replace ./platform with the path to your parent chart if different.

Post-Installation Steps#

Port-forward the Evaluator service:

kubectl -n <NAMESPACE> \
  port-forward \
  service/<release-name>-nemo-evaluator \
  7331:7331

Replace <NAMESPACE> and <release-name> as appropriate.
Verify the deployment:
- Launch an evaluation job using the Evaluator API.
- Check logs and service status as needed.

Troubleshooting#

Secret Ownership/Label Errors#

If you encounter an error like:

Error: INSTALLATION FAILED: Unable to continue with install: Secret "nvcrimagepullsecret" in namespace "default" exists and cannot be imported into the current release: invalid ownership metadata; label validation error: missing key "app.kubernetes.io/managed-by": must be set to "Helm"; annotation validation error: missing key "meta.helm.sh/release-name": must be set to "nemo-platform"; annotation validation error: missing key "meta.helm.sh/release-namespace": must be set to "default"

Make sure you have set existingSecret and existingImagePullSecret in your values file as shown above.
Do not let Helm try to create or import secrets you created manually.

ImagePullBackOff or manifest unknown#

If pods fail to start and you encounter events like:

Failed to pull image "nvcr.io/nvidia/nemo-microservices/evaluator:${parent_helm_chart_latest_version}": Error response from daemon: manifest for nvcr.io/nvidia/nemo-microservices/evaluator:${parent_helm_chart_latest_version} not found: manifest unknown: manifest unknown

Check that the image tag exists in NGC.

Override the image tag in your values file if needed.

evaluator:
  image:
    tag: "${parent_helm_chart_latest_version}"  # Change to a valid tag if needed

Ensure your image pull secret is correct and referenced properly.