Health Check Reference

Dynamo exposes two related health interfaces: HTTP status endpoints for external observers and optional canary requests that actively exercise idle worker endpoints. For a local procedure, see Observe a Local Deployment.

Endpoint Summary

Set DYN_SYSTEM_HEALTH_PATH or DYN_SYSTEM_LIVE_PATH before process startup to replace the worker or router paths. For the frontend, use DYN_HTTP_SVC_HEALTH_PATH or DYN_HTTP_SVC_LIVE_PATH.

Frontend Responses

A live frontend returns 200 OK:

1
{
2
  "message": "Service is live",
3
  "status": "live"
4
}

During graceful shutdown, /live remains 200 OK while the Frontend is draining admitted requests. After the drain finishes or times out and the service enters the stopping state, /live returns 503 Service Unavailable with status shutting_down.

The frontend /health endpoint returns 200 OK while ready and lists the endpoints and instances currently visible through discovery:

1
{
2
  "status": "healthy",
3
  "endpoints": [
4
    "dyn://dynamo.backend.generate"
5
  ],
6
  "instances": [
7
    {
8
      "namespace": "dynamo",
9
      "component": "backend",
10
      "endpoint": "generate",
11
      "instance_id": 7587888160958628000
12
    }
13
  ]
14
}

An empty endpoints or instances array means the frontend does not currently discover workers; it does not change the frontend response status by itself. As soon as graceful draining starts, /health returns 503 Service Unavailable with status not_ready and a stage value that identifies the current lifecycle stage. This removes the Frontend from readiness routing before admitted responses finish.

System-Status Responses

Workers and standalone routers expose health endpoints only when DYN_SYSTEM_PORT is zero or a positive port. A negative value disables the system-status server.

Before the component is ready, /health and /live return 503 Service Unavailable:

1
{
2
  "endpoints": {
3
    "generate": "notready"
4
  },
5
  "status": "notready",
6
  "uptime": {
7
    "nanos": 313803539,
8
    "secs": 12
9
  }
10
}

The worker /live and /health paths currently report the same system-status payload and status code. After startup, both paths return 200 OK with status ready:

1
{
2
  "endpoints": {
3
    "clear_kv_blocks": "ready",
4
    "generate": "ready",
5
    "load_metrics": "ready"
6
  },
7
  "status": "ready",
8
  "uptime": {
9
    "nanos": 356504530,
10
    "secs": 18
11
  }
12
}

DYN_SYSTEM_STARTING_HEALTH_STATUS controls the initial status. The legacy DYN_SYSTEM_USE_ENDPOINT_HEALTH_STATUS variable is deprecated and no longer controls endpoint selection in the current runtime.

Active Canary Configuration

Each backend supplies a minimal inference payload. Dynamo marks the request as a health check and uses the backend’s normal endpoint path, so the check covers request handling and engine execution rather than only process reachability. For the lifecycle and rationale, see Observability Architecture.

Related Reference

• Environment Variables
• Local Observability Stack
• Configure Kubernetes Health Probes