Advanced Configuration for NeMo Evaluator#

This guide provides advanced configuration options for the NeMo Evaluator microservice, such as connecting to external dependencies, customizing workflow orchestration, and enabling advanced monitoring.

Before You Start#

Complete the Helm Chart deployment before using these advanced options. This article assumes you have a working NeMo Evaluator deployment and want to customize or extend it for production or advanced use cases.

Externalizing Dependencies#

By default, the Helm chart deploys all required dependencies. For production or custom environments, you may wish to use your own Milvus, or PostgreSQL instances.

External Dependency Configuration#

Milvus

milvus:
  enabled: false
external:
  milvus:
    endpoint: "<url to the pre-installed milvus>"

PostgreSQL

By default, the NeMo Entity Store Helm chart uses the Bitnami PostgreSQL chart. To use an external PostgreSQL database, set the appropriate values and see the PostgreSQL section for details.

Data Store Integration#

To use a custom or external Data Store, specify the endpoint in your custom-values.yaml:

external:
  dataStore:
    endpoint: "http://<nemo-data-store-service>.<nemo-data-store-namespace>.svc.cluster.local:8000"

Milvus#

For Retriever and RAG evaluations with large datasets, you may want to tune Milvus storage or use your own instance. Example for default Milvus install:

milvus:
  enabled: true
  serviceName: milvus
  cluster:
    enabled: false
  etcd:
    enabled: false
  pulsar:
    enabled: false
  minio:
    enabled: false
    tls:
      enabled: false
  standalone:
    persistence:
      enabled: true
      persistentVolumeClaim:
        size: 100Gi
        storageClass: standard
    extraEnv:
      - name: LOG_LEVEL
        value: error
  extraConfigFiles:
    user.yaml: |+
      etcd:
        use:
          embed: true
        data:
          dir: /var/lib/milvus/etcd
      common:
        storageType: local

Tip

For large datasets, set storage size to at least 100Gi. For production, use an external Milvus as described above.

Monitoring & Telemetry#

NeMo Evaluator is instrumented with OpenTelemetry. To enable exporters and customize telemetry:

Basic OpenTelemetry

otelExporterEnabled: true
otelEnvVars:
  OTEL_EXPORTER_OTLP_ENDPOINT: "http://<otel-collector>:4317"
  OTEL_SERVICE_NAME: "nemo-evaluator"
  OTEL_TRACES_EXPORTER: otlp
  OTEL_METRICS_EXPORTER: otlp
  OTEL_LOGS_EXPORTER: otlp
  OTEL_PROPAGATORS: "tracecontext,baggage"
  OTEL_RESOURCE_ATTRIBUTES: "deployment.environment=$(NAMESPACE)"
  OTEL_PYTHON_EXCLUDED_URLS: "health"
  OTEL_PYTHON_LOGGING_AUTO_INSTRUMENTATION_ENABLED: "true"

With OpenTelemetry Collector

opentelemetry-collector:
  enabled: true

With Zipkin

zipkin:
  enabled: true

Evaluator Monitoring & Telemetry Configuration Options#
Name	Description	Value
`zipkin.enabled`	Deploy Zipkin for metrics.	`false`
`opentelemetry-collector.enabled`	Deploy OpenTelemetry Collector for metrics.	`false`
`opentelemetry-collector.config`	OpenTelemetry Collector configurations.	`{{}}`
`otelExporterEnabled`	Enable OpenTelemetry exporters for Evaluator.	`false`
`otelEnvVars`	Environment variables to configure OpenTelemetry for Evaluator.	`{{}}`
`logLevel`	Log level for both OTLP and console exporters.	`INFO`

Port Forwarding & Access#

To access the Evaluator microservice locally for testing or debugging:

kubectl -n <NAMESPACE> port-forward service/myrelease-nemo-evaluator 7331:7331

After port-forwarding, your data scientists can use the local host URL to access the NeMo Evaluator microservice APIs.