Router

The Dynamo KV router selects an eligible worker using reusable KV cache state and projected active load. Use it when prefix reuse should influence placement; use a non-KV routing mode when your deployment needs only load balancing or an externally selected worker.

Start With Your Request Path

Choose the page that matches where requests enter Dynamo:

For a topology comparison and the configuration boundary between the Frontend and workers, see the Router Guide.

Follow a Request Through the Router

1. The request host tokenizes and normalizes the request.
2. The router filters workers that cannot serve it.
3. The router scores eligible workers using cache overlap and projected load.
4. Aggregated deployments run on the selected worker, while disaggregated deployments select separate prefill and decode workers. Workers publish KV lifecycle events when available; set --no-router-kv-events to predict cache state when the router cannot consume them.

Read KV-Aware Routing for the architecture-level flow. Read Routing Concepts for the cost model, Router Filtering for eligibility, and Deficit Round Robin Queue Scheduling for policy-class arbitration.

Continue by Task

The Frontend Configuration Reference is the canonical source for embedded-router flags, environment variables, defaults, and boolean forms.