Metrics Catalog

Dynamo exposes metrics in Prometheus exposition format at the /metrics HTTP endpoint. All Dynamo-generated metrics use the dynamo_* prefix and carry the hierarchy labels documented in Metric Labels. This page is the field catalog; for setup and dashboards see the Metrics guide.

Metric types

The type on each field below is a Prometheus metric type:

• counter — a cumulative value that only increases (or resets to zero on restart), e.g. total requests.
• gauge — a single value that can go up or down, e.g. in-flight requests.
• histogram — samples observations into configurable buckets and exposes _bucket, _sum, and _count series, e.g. request duration. Labeled histograms/counters/gauges register a metric family, not one series.

Where each family is exposed

Backend workers expose dynamo_component_* only when DYN_SYSTEM_PORT is set (disabled by default; the Kubernetes operator typically sets 9090, local examples use 8081). See Environment Variables.

Engine pass-through metrics (vllm:*, sglang:*, trtllm_*) are emitted by the backend engines themselves and are cataloged separately — see Metrics Comparison. NIXL transfer metrics are exposed on their own port and owned by the upstream NIXL project.

Frontend metrics

Emitted by the HTTP frontend at /metrics on port 8000 by default. Most carry a model label.

Use this PromQL expression to calculate the five-minute token reuse ratio by model:

sum by (model) (rate(dynamo_frontend_tokenizer_cache_cached_tokens_total[5m]))
/
(
  sum by (model) (rate(dynamo_frontend_tokenizer_cache_cached_tokens_total[5m]))
  +
  sum by (model) (rate(dynamo_frontend_tokenizer_cache_uncached_tokens_total[5m]))
)

The ratio is defined only when the model has an active L1 cache and observed tokens. Partial hits increment both token counters. The counters remain zero when DYN_TOKENIZER_CACHE=0, encoding fails, or the tokenizer has no registered special-token boundaries.

Migration

Counters for request migration, the mechanism that continues in-flight requests on a healthy worker when the original worker fails. See Request Migration Architecture for the internals.

Cancellation and rejection

Counters for requests that end early: cancellations (the client or frontend aborts an in-flight request) and rejections (the Frontend sheds load with HTTP 529 by default when all workers are busy). The worker-side counterparts are under Component metrics.

Per-worker load and timing gauges

dynamo_frontend_worker_* gauges appear once workers register and begin serving. They live on the frontend’s local registry (not component-scoped) and do not carry dynamo_namespace or dynamo_component labels. Frontend-only — not available on the standalone router. Labeled by worker_id, dp_rank, and worker_type.

Model configuration metrics

dynamo_frontend_model_* gauges are populated from worker backend registration and all carry a model label. When multiple workers register the same model name, only the first instance’s configuration is recorded.

Component metrics

Emitted by backend workers (python -m dynamo.vllm, python -m dynamo.sglang, etc.) at /metrics on DYN_SYSTEM_PORT. All carry the hierarchy labels (dynamo_namespace, dynamo_component, dynamo_endpoint) — see Runtime-injected labels.

The following worker-side gauges and counter appear only when a hard concurrency cap is set with --engine-request-limit; see Worker-Side Request Admission.

Specialized components expose additional families under their own prefix, e.g. dynamo_preprocessor_* for preprocessor components.

Router metrics

The router exposes metrics for routing decisions and overhead. Not every metric appears in every deployment — see Availability by configuration.

Router request metrics

dynamo_component_router_* histograms and counters for aggregate request-level statistics. On the frontend, exposed at /metrics on the HTTP port; on the standalone router, exposed on DYN_SYSTEM_PORT. Populated per-request when --router-mode kv is active; registered with zero values in non-KV modes.

Per-request routing overhead

dynamo_router_overhead_* histograms (milliseconds) track time spent in each phase of the routing decision. Registered on the frontend port with a router_id label (the frontend’s discovery instance ID). Created only when the frontend has DRT discovery enabled (--router-mode kv); absent in non-KV modes and on the standalone router.

Router queue metrics

The frontend registers these metrics when queueing is enabled by --router-queue-threshold or a threshold in --router-policy-config. They carry model, worker_type, and policy_class. With policy-family or cache-bucket configuration, policy_class is the resolved physical queue.

KV indexer metrics

The standalone indexer exposes the device-tier-only counter dynamo_kvrouter_kv_cache_events_applied; see Standalone KV Indexer.

Availability by configuration

Not all router metrics appear in every deployment. This matrix shows which groups are registered and populated in each configuration.

Key:

• Registered and populated — the metric appears at /metrics with real values.
• Registered, always zero — the metric appears at /metrics but is never incremented (useful for dashboards that expect it to exist).
• Not registered / Not created — the metric does not appear at /metrics at all.

Related

• Metric Labels — the dimensions attached to these metrics.
• Environment Variables — variables that enable and configure metric emission.
• Operator Metrics — Kubernetes operator metrics catalog.
• Metrics Comparison — engine pass-through metrics (vllm:*, sglang:*, trtllm_*).