Deploy NeMo Evaluator Using Parent Helm Chart#

You can deploy the NeMo Evaluator microservice using the parent Helm chart, which manages all required dependencies automatically. This is the recommended approach for most users.

By default, this deployment enables only the Evaluator and Data Store microservices, which are the minimal required dependencies for Evaluator. All other microservices are explicitly disabled.

Why Use the Parent Chart?

Dependency Management: The parent chart ensures all required services are installed and networked correctly.
Simplicity: One command, one values file, minimal manual configuration.
Support: This is the officially supported and tested deployment method.

Prerequisites#

Dependencies

Kubernetes cluster with Helm installed
NVIDIA GPU Cloud (NGC) API key for pulling images
(Optional) External services if not using the chart-provided dependencies:
- NIM: Used for model inference.
- NeMo Data Store
- Argo Workflows: Used for Evaluation job orchestration.
- Milvus: Used for Retriever and RAG evaluations.
- PostgreSQL: The persistent data store for NeMo Evaluator.

Kubernetes

Create NGC Image pull credentials. This includes:
- A generic secret containing your NGC API key, referenced by existingSecret
- An image pull secret, referenced by existingImagePullSecret

Download the Parent Helm Chart#

Add the NeMo Microservices Helm Chart helm repo.

helm repo add nmp https://helm.ngc.nvidia.com/nvidia/nemo-microservices \
   --username='$oauthtoken' \
   --password=$NGC_API_KEY

helm repo update

Prepare Your Values File#

Create a values-evaluator-only.yaml file to enable only the Evaluator and its required dependencies. For more advanced scenarios (for example, enabling Milvus, Argo, Ingress), see the advanced example below.

Minimal (Evaluator Only)

tags:
  platform: false
  evaluator: true

Advanced (Milvus, Argo, Ingress)

tags:
  platform: false
  evaluator: true

evaluator:
  milvus:
    enabled: true
    standalone:
      persistence:
        enabled: true
        persistentVolumeClaim:
          size: 50Gi
          storageClass: "standard"
ingress:
  enabled: true

If you want to use external services for any dependency, set its enabled value to false and provide the appropriate external endpoint in your values file.

Install with Helm#

Run the following command to install the platform with your custom values file:

helm --namespace nemo-evaluator install nemo-evaluator \
  nmp/nemo-microservices-helm-chart \
  -f values-evaluator-only.yaml

Post-Installation Steps#

Port-forward the Evaluator service:

kubectl -n <NAMESPACE> \
  port-forward \
  service/<release-name>-nemo-evaluator \
  7331:7331

Replace <NAMESPACE> and <release-name> as appropriate.
Verify the deployment:
- Launch an evaluation job using the Evaluator API.
- Check logs and service status as needed.

Troubleshooting#

Secret Ownership/Label Errors#

If you encounter an error like:

Error: INSTALLATION FAILED: Unable to continue with install: Secret "nvcrimagepullsecret" in namespace "default" exists and cannot be imported into the current release: invalid ownership metadata; label validation error: missing key "app.kubernetes.io/managed-by": must be set to "Helm"; annotation validation error: missing key "meta.helm.sh/release-name": must be set to "nemo-platform"; annotation validation error: missing key "meta.helm.sh/release-namespace": must be set to "default"

Make sure you have set existingSecret and existingImagePullSecret in your values file as shown above.
Do not let Helm try to create or import secrets you created manually.

ImagePullBackOff or manifest unknown#

If pods fail to start and you encounter events like:

Failed to pull image "nvcr.io/nvidia/nemo-microservices/evaluator:${parent_helm_chart_latest_version}": Error response from daemon: manifest for nvcr.io/nvidia/nemo-microservices/evaluator:${parent_helm_chart_latest_version} not found: manifest unknown: manifest unknown

Check that the image tag exists in NGC.

Override the image tag in your values file if needed.

evaluator:
  image:
    tag: "${parent_helm_chart_latest_version}"  # Change to a valid tag if needed

Ensure your image pull secret is correct and referenced properly.