Deploy Harmonizer NIM with Helm | NVIDIA NIM Harmonizer

Deploy Harmonizer NIM on Kubernetes with the NVIDIA NIM for World Foundation Models (WFM) Helm chart. For supported GPUs and profiles, refer to the Support Matrix.

Prerequisites

Prepare a Kubernetes cluster with supported NVIDIA GPU nodes and the GPU Operator installed.
Create an NGC_API_KEY and authenticate to NGC as described in Quickstart for Harmonizer NIM.

Deploy the NIM

Complete these steps to install the chart and verify the deployment:

Download the nim-wfm Helm chart from NGC and inspect its defaults.

After you have an NGC API key, download version 1.3.0 or later of the nim-wfm Helm chart from NGC. This chart version includes the gRPC Service port and configurable inference test used by Harmonizer NIM. For available versions, refer to the NGC Catalog.
```
helm fetch https://helm.ngc.nvidia.com/nim/charts/nim-wfm-1.3.0.tgz \
   --username='$oauthtoken' \
   --password=$NGC_API_KEY
```
You can inspect the chart README and values before deploying:
```
helm show readme nim-wfm-1.3.0.tgz
helm show values nim-wfm-1.3.0.tgz
```

Create the Kubernetes registry and model-download secrets.

Create one secret for pulling the container image and one secret for model asset downloads:

kubectl create secret docker-registry ngc-secret \
   --docker-server=nvcr.io \
   --docker-username='$oauthtoken' \
   --docker-password=$NGC_API_KEY
kubectl create secret generic ngc-api \
   --from-literal=NGC_API_KEY=$NGC_API_KEY

Create custom-values.yaml with the public image and persistent model cache.

Create a custom-values.yaml file with a persistent model cache.

image:
  repository: nvcr.io/nim/nvidia/harmonizer
  tag: 1.0.0
imagePullSecrets:
  - name: ngc-secret
model:
  name: nvidia/harmonizer
  ngcAPISecret: ngc-api
  nimCache: /model-store
  inferenceProtocol: http
  apiPort: 8000
persistence:
  enabled: true
  size: 60Gi
resources:
  limits:
    nvidia.com/gpu: 1
startupProbe:
  failureThreshold: 360
tests:
  inference:
    enabled: true
    path: /v1/infer
    timeoutSeconds: 300
    expectedStatus: 200
    payload: |
      {
        "request_id": "helm-test",
        "images": [
          {
            "id": "frame-000000",
            "data": "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAIAAACQd1PeAAAADElEQVR4nGNoaGgAAAMEAYFL09IQAAAAAElFTkSuQmCC"
          }
        ],
        "parameters": {
          "output_format": "png",
          "context_length": 0,
          "return_policy": "all"
        }
      }

Do not set NGC_API_KEY or NIM_CACHE_PATH through the env list. Use model.ngcAPISecret for the NGC key and model.nimCache for the cache mount path. When persistence is enabled, the chart mounts the model-cache persistent volume claim (PVC) at model.nimCache and sets NIM_CACHE_PATH to the same value. Thus, the example relocates the container default of /opt/nim/.cache to the persistent /model-store volume. For more information, refer to Environment Variables and Configuration.

Optional: Create grpc-values.yaml to select gRPC inference.

The inference server runs either HTTP or gRPC. To use gRPC, create a grpc-values.yaml file that selects the gRPC entrypoint and exposes its port through the Kubernetes Service:
```
model:
  inferenceProtocol: grpc
  grpcPort: 50051
service:
  grpcPort: 50051
tests:
  inference:
    enabled: false
```
The HTTP management server remains available on model.apiPort for health, readiness, and metrics when gRPC inference is selected. The chart’s readiness test therefore works in both protocol modes. The generic inference test sends an HTTP POST request and is not rendered in gRPC mode. Test the full gRPC harmonizer.v1.Harmonizer/Process call with a client generated from the Harmonizer protocol definition, as described in the API Reference.
Install the chart, wait for readiness, and run its tests.

Choose one of the following install commands. For HTTP inference, use your custom values:
```
helm install harmonizer-nim nim-wfm-1.3.0.tgz \
   -f custom-values.yaml
```
For gRPC inference, apply the gRPC override after the common values:
```
helm install harmonizer-nim nim-wfm-1.3.0.tgz \
   -f custom-values.yaml \
   -f grpc-values.yaml
```
Wait for the pod to become ready:
```
kubectl get pods
kubectl logs -f statefulset/harmonizer-nim-nim-wfm
```
After the NIM becomes ready, press Ctrl+C to stop following the logs. This stops the log viewer and leaves the deployment running.

Run the chart tests:
```
helm test harmonizer-nim --logs
```
With the HTTP Values on this page, the tests check the HTTP readiness route and send one image to /v1/infer with the configured payload. With the gRPC override, the tests check the HTTP readiness route only.
Forward the service ports and run an inference request.

Port-forward the HTTP service:
```
kubectl port-forward service/harmonizer-nim-nim-wfm 8000:http-api
```
Keep port forwarding running in this terminal. In a second terminal on the same host, verify readiness:
```
curl -sS http://localhost:8000/v1/health/ready
```
For request examples, refer to Quickstart for Harmonizer NIM and the API Reference.

For a gRPC deployment, forward both the HTTP management port and the gRPC inference port:
```
kubectl port-forward service/harmonizer-nim-nim-wfm \
   8000:http-api 50051:grpc-api
```
Keep port forwarding running in this terminal. In a second terminal, connect the generated gRPC client to localhost:50051. Keep readiness probes and management requests on localhost:8000.

Troubleshooting

Recover from a Fatal CUDA Error

Harmonizer treats CUDA error 226 (cudaErrorContained) as unrecoverable. It marks the server unready, rejects or settles pending GPU work, and then exits with status 70 after a 250 ms grace period. The affected server process does not remain running indefinitely in an unready or gRPC NOT_SERVING state. If the error occurs during model loading or warmup, the process exits immediately because the server and its health endpoints have not started.

For a deployment created by this chart, the readiness endpoint immediately returns 503 during a serving-time transition. The 250 ms interval is shorter than the chart’s normal readiness-probe cadence, so the process exit normally makes the pod unready and removes it from Service endpoints before the probe records the transition. The exit, not the readiness probe, causes Kubernetes to restart the container according to the pod restart policy. Inspect the previous container instance and its restart count with commands such as the following:

$ kubectl logs <pod-name> --previous
$ kubectl get pod <pod-name> -o jsonpath='{.status.containerStatuses[0].restartCount}{"\n"}'

A container restart creates a fresh CUDA context but does not move the pod to a different GPU or node. If the error recurs, the pod can enter CrashLoopBackOff. Inspect GPU Xid, ECC, and NVLink health and remove unhealthy hardware from service according to your cluster operations policy.

For more information about the health-state transition, refer to Observability Metrics for Harmonizer NIM.

Pod Remains Pending

If the pod is stuck in Pending, check scheduling events:

kubectl describe pod <pod-name>

Common causes are missing GPU resources, node taints without matching tolerations, and storage configuration issues.

Startup Fails Before Readiness

If startup fails before readiness, inspect the container logs:

kubectl logs -f <pod-name>

Large first-run model downloads and startup compilation can take time. Use a persistent cache volume and increase startupProbe.failureThreshold if your cluster has slow model storage or network access.