> For clean Markdown content of this page, append .md to this URL. For the complete documentation index, see https://docs.nvidia.com/dynamo/llms.txt. For full content including API reference and SDK examples, see https://docs.nvidia.com/dynamo/llms-full.txt.

# Health Check Reference

Dynamo exposes two related health interfaces: HTTP status endpoints for external observers and
optional canary requests that actively exercise idle worker endpoints. For a local procedure, see
[Observe a Local Deployment](/dynamo/dev/cli/operations/observability#check-deployment-health).

## Endpoint Summary

| Process                     | Default port                           | Path      | Behavior                                                        |
| --------------------------- | -------------------------------------- | --------- | --------------------------------------------------------------- |
| Frontend                    | `DYN_HTTP_PORT`, default `8000`        | `/live`   | Reports whether the HTTP service is running or shutting down    |
| Frontend                    | `DYN_HTTP_PORT`, default `8000`        | `/health` | Reports frontend health plus discovered endpoints and instances |
| Worker or standalone router | `DYN_SYSTEM_PORT`, disabled by default | `/live`   | Reports system status                                           |
| Worker or standalone router | `DYN_SYSTEM_PORT`, disabled by default | `/health` | Reports system status                                           |

Set `DYN_SYSTEM_HEALTH_PATH` or `DYN_SYSTEM_LIVE_PATH` before process startup to replace the worker
or router paths. For the frontend, use `DYN_HTTP_SVC_HEALTH_PATH` or `DYN_HTTP_SVC_LIVE_PATH`.

## Frontend Responses

A live frontend returns `200 OK`:

```json
{
  "message": "Service is live",
  "status": "live"
}
```

During graceful shutdown, `/live` remains `200 OK` while the Frontend is draining admitted requests.
After the drain finishes or times out and the service enters the stopping state, `/live` returns
`503 Service Unavailable` with status `shutting_down`.

The frontend `/health` endpoint returns `200 OK` while ready and lists the endpoints and instances currently
visible through discovery:

```json
{
  "status": "healthy",
  "endpoints": [
    "dyn://dynamo.backend.generate"
  ],
  "instances": [
    {
      "namespace": "dynamo",
      "component": "backend",
      "endpoint": "generate",
      "instance_id": 7587888160958628000
    }
  ]
}
```

An empty `endpoints` or `instances` array means the frontend does not currently discover workers; it
does not change the frontend response status by itself. As soon as graceful draining starts, `/health`
returns `503 Service Unavailable` with status `not_ready` and a `stage` value that identifies the
current lifecycle stage. This removes the Frontend from readiness routing before admitted responses
finish.

## System-Status Responses

Workers and standalone routers expose health endpoints only when `DYN_SYSTEM_PORT` is zero or a
positive port. A negative value disables the system-status server.

Before the component is ready, `/health` and `/live` return `503 Service Unavailable`:

```json
{
  "endpoints": {
    "generate": "notready"
  },
  "status": "notready",
  "uptime": {
    "nanos": 313803539,
    "secs": 12
  }
}
```

The worker `/live` and `/health` paths currently report the same system-status payload and status
code. After startup, both paths return `200 OK` with status `ready`:

```json
{
  "endpoints": {
    "clear_kv_blocks": "ready",
    "generate": "ready",
    "load_metrics": "ready"
  },
  "status": "ready",
  "uptime": {
    "nanos": 356504530,
    "secs": 18
  }
}
```

`DYN_SYSTEM_STARTING_HEALTH_STATUS` controls the initial status. The legacy
`DYN_SYSTEM_USE_ENDPOINT_HEALTH_STATUS` variable is deprecated and no longer controls endpoint
selection in the current runtime.

## Active Canary Configuration

| Variable                           | Default         | Purpose                                                                             |
| ---------------------------------- | --------------- | ----------------------------------------------------------------------------------- |
| `DYN_HEALTH_CHECK_ENABLED`         | `false`         | Enables active checks for registered worker endpoints                               |
| `DYN_CANARY_WAIT_TIME`             | `10`            | Seconds without successful endpoint activity before sending a canary                |
| `DYN_HEALTH_CHECK_REQUEST_TIMEOUT` | `3`             | Seconds to wait for the canary response                                             |
| `DYN_HEALTH_CHECK_PAYLOAD`         | Backend default | Optional JSON object or `@/path/to/file.json` payload override for unified backends |

Each backend supplies a minimal inference payload. Dynamo marks the request as a health check and
uses the backend's normal endpoint path, so the check covers request handling and engine execution
rather than only process reachability. For the lifecycle and rationale, see
[Observability Architecture](/dynamo/dev/knowledge-base/concepts/observability-architecture#active-worker-health-checks).

## Related Reference

* [Environment Variables](/dynamo/dev/reference/observability/environment-variables#health-checks)
* [Local Observability Stack](/dynamo/dev/reference/observability/local-stack)
* [Configure Kubernetes Health Probes](/dynamo/dev/kubernetes/operations/observability#check-and-customize-health-probes)