Is this page helpful?

Troubleshooting Dynamo Mode#

Use this documentation to troubleshoot failures specific to Dynamo mode. For cache permissions, GPU runtime access, and executable temporary mounts, refer to Container Startup Problems.

A Worker Serves Inference Without Dynamo#

Single-container and worker-only roles can fall back to direct vLLM serving when the image does not contain the Dynamo packages. A successful inference response therefore does not prove that the worker registered with Dynamo.

A genuine worker logs:

Starting Dynamo worker: python -m dynamo.vllm

The fallback path logs:

Launching vLLM: vllm serve

Use the NIM 3.0 vLLM image published for Dynamo mode and verify the image tag against the deployment recipe.

The KV Router Requires NATS#

The KV router has a legacy subscriber path that expects NATS when the event plane is not set explicitly:

KV router's EventSubscriber requires NATS.

Set DYN_EVENT_PLANE=zmq on the frontend and worker for a file-discovery deployment. Single-container mode sets this automatically.

A Port Is Already in Use#

The worker system server and frontend cannot listen on the same port:

ERROR: Failed to bind to address 0.0.0.0:8000: Address already in use

Leave NIM_HEALTH_PORT unset to use the worker default of 9090, or set it to another free port. Do not configure DYN_SYSTEM_PORT directly: the entrypoint derives it from NIM_HEALTH_PORT and overwrites a direct value.

Workers Do Not Register#

If /v1/models stays empty in a split deployment, check the following:

  • The frontend and workers use the same NIM_MODEL_PROFILE. A metadata checksum mismatch prevents registration.

  • Discovery, event-plane, and request-plane settings match across components.

  • Worker logs show model-load progress and the genuine Dynamo launch marker.

For profile-selection details, refer to Model Profiles and Selection.

Frontend Readiness Fails#

NIM 3.0 adds management endpoints to the Dynamo frontend. Probe /v1/health/ready on the frontend HTTP port. A published DGD recipe uses that route on port 8000. Use /v1/models separately to confirm that the expected model is registered.

Metrics Are Missing#

Scrape Dynamo frontend metrics from /v1/metrics on the frontend HTTP port. Scrape vLLM metrics from /metrics on each worker system port, which is configured with NIM_HEALTH_PORT and defaults to 9090. Refer to Logging and Observability for a two-target Prometheus configuration.