> For clean Markdown content of this page, append .md to this URL. For the complete documentation index, see https://docs.nvidia.com/dynamo/llms.txt. For full content including API reference and SDK examples, see https://docs.nvidia.com/dynamo/llms-full.txt.

# Disaggregated Serving

Dynamo supports disaggregated serving where prefill (prompt processing) and decode (token generation) are handled by separate worker pools. The frontend activates an internal prefill router when it discovers compatible typed prefill and decode services.

For the high-level deployment matrix, see [Router Guide](/dynamo/dev/knowledge-base/modular-components/router/router-guide). For the router flags used in this setup, see [Configuration and Tuning](/dynamo/dev/knowledge-base/modular-components/router/configuration-and-tuning).

If prefill and decode workers span topology domains such as zones or racks, use [Topology-Aware KV Transfer](/dynamo/dev/knowledge-base/modular-components/router/topology-aware-kv-transfer) to constrain or bias decode routing toward workers in the selected prefill worker's transfer domain.

## Automatic Prefill Router Activation

The prefill router is automatically created when:
1. A decode worker is registered with `WorkerType.Decode`, for example via `register_model()` with `ModelType.Chat | ModelType.Completions`.
2. A prefill service in the same Runtime namespace is registered with the same model name and `WorkerType.Prefill`.

Key characteristics of the prefill router:
- **Always disables active block tracking** (`track_active_blocks=false`) since prefill workers do not perform decode.
- **Runs between preprocessing and decode routing** and returns handoff metadata for the selected decode worker.
- **Uses decode-only routing when no prefill router is active.** Once a request has been dispatched to prefill, a prefill or handoff failure is returned to the request; it does not retry through an aggregated decode-only path.

Key characteristics of the decode routing stage in disaggregated mode:
- **Disables overlap scoring** (`overlap_score_credit=0`) because decode routing should not chase prefix reuse.
- **Disables KV reuse assumption** (`assume_kv_reuse=false`) unless the backend can truly deduplicate transferred blocks.
- **Disables prefill-token tracking** (`track_prefill_tokens=false`) so decode-side load reflects decode work rather than already-completed prompt work.

<Note>
The Rust router contains an experimental conditional bypass path for programmatic or embedded configurations. It can keep a cache-hot request on a selected decode worker instead of performing remote prefill. The standard `dynamo.frontend` and standalone Python router CLIs do not expose that configuration, so ordinary CLI deployments follow the prefill-handoff-decode flow documented below.
</Note>

## Setup Example

When both workers are registered, requests are automatically routed.

```python
# Decode worker registration (in your decode worker)
decode_endpoint = runtime.endpoint("dynamo.decode.generate")

await register_model(
    model_input=ModelInput.Tokens,
    model_type=ModelType.Chat | ModelType.Completions,
    endpoint=decode_endpoint,
    model_name="meta-llama/Llama-2-7b-hf",
    worker_type=WorkerType.Decode,
    needs=[[WorkerType.Prefill]],
    # ... other parameters
)

await decode_endpoint.serve_endpoint(decode_handler.generate)

# Prefill worker registration (in your prefill worker)
prefill_endpoint = runtime.endpoint("dynamo.prefill.generate")

await register_model(
    model_input=ModelInput.Tokens,
    model_type=ModelType.Empty,  # prefill workers expose no OpenAI surface
    endpoint=prefill_endpoint,
    model_name="meta-llama/Llama-2-7b-hf",
    worker_type=WorkerType.Prefill,
    needs=[[WorkerType.Decode]],
    # ... other parameters
)

await prefill_endpoint.serve_endpoint(prefill_handler.generate)
```

<Note>
The automatic disaggregated routing setup described here is currently supported by the integrated `dynamo.frontend` path. It is not provided as a single turnkey mode by the standalone Python router (`python -m dynamo.router`). If you build this topology with standalone routers, you must launch and connect the prefill and decode routing stages yourself and handle request handoff, including the `disaggregated_params` returned by prefill. For an advanced reference, see the [Global Router](https://github.com/ai-dynamo/dynamo/tree/main/components/src/dynamo/global_router), which composes local prefill and decode router pools explicitly.
</Note>

## Request Flow

The following diagram shows an overview of the major components in disaggregated serving:

```mermaid
graph TD
    HTTP[HTTP]
    ROUTER[Router]
    PREFILL[Prefill Worker]
    DECODE[Decode Worker]

    classDef worker_style fill:#f3e5f5,stroke:#333,stroke-width:2px,color:#333;
    classDef router_style fill:#2e8b57,stroke:#333,stroke-width:2px,color:#fff;

    class PREFILL,DECODE worker_style
    class ROUTER router_style

    HTTP <--> |"request/response"| ROUTER
    ROUTER --> |"1. send to prefill"| PREFILL
    PREFILL --> |"2. return NIXL metadata"| ROUTER
    ROUTER --> |"3. send with metadata"| DECODE
    DECODE --> |"4. stream response"| ROUTER

    PREFILL -.-> |"publish kv events"| ROUTER

    linkStyle 0,1,2,3,4 stroke:#8b4513,stroke-width:2px
    linkStyle 5 stroke:#2196f3,stroke-width:2px
```

When topology-aware KV transfer is enabled, the prefill router also derives decode `RoutingConstraints` from the selected prefill worker's runtime topology metadata before the request enters the decode router.