> For clean Markdown content of this page, append .md to this URL. For the complete documentation index, see https://docs.nvidia.com/dynamo/llms.txt. For full content including API reference and SDK examples, see https://docs.nvidia.com/dynamo/llms-full.txt.

# Disaggregated Serving

> Split prefill and decode across independent workers

Disaggregated serving separates the prefill and decode phases of inference into independent workers, each scalable on its own and connected via [NIXL](/dynamo/dev/knowledge-base/design-documents/disaggregated-serving) for KV cache transfer. This is the local equivalent of the disaggregated pattern in the [Kubernetes Disaggregated Serving guide](/dynamo/dev/kubernetes/model-deployment/disaggregated-serving) — the same architecture, driven by CLI flags instead of a DynamoGraphDeployment.

For the architecture and when to use it, see [Disaggregated Serving](/dynamo/dev/kubernetes/model-deployment/disaggregated-serving).

## How it works locally

A disaggregated deployment is still a frontend plus workers (see the [Introduction](/dynamo/dev/cli/model-deployment/introduction)), except instead of one worker doing both phases, you run:

* A **prefill worker** — computes the prompt's KV cache.
* A **decode worker** — receives the KV cache over NIXL and generates tokens.

The `launch/disagg.sh` presets start the frontend and both workers, pinned to separate GPUs with the KV-transfer config wired up. Requires 2 GPUs.

## Disaggregated serving

#### vLLM

```bash
cd $DYNAMO_HOME/examples/backends/vllm
bash launch/disagg.sh
```

Each worker needs a unique `VLLM_NIXL_SIDE_CHANNEL_PORT`; the preset sets these for you.

#### SGLang

```bash
cd $DYNAMO_HOME/examples/backends/sglang
./launch/disagg.sh
```

For how SGLang disaggregation works with Dynamo — the bootstrap mechanism and RDMA transfer flow — see [SGLang Disaggregation](/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/disaggregation).

#### TensorRT-LLM

```bash
cd $DYNAMO_HOME/examples/backends/trtllm
./launch/disagg.sh
```

## Adding KV-aware routing

To scale each pool and route requests by cache overlap, use the `disagg_router.sh` presets (2 prefill + 2 decode workers, 4 GPUs). See [KV-Aware Routing](/dynamo/dev/cli/model-deployment/kv-aware-routing#disaggregated-serving-with-kv-routing).

With a disaggregated deployment running, try adding another prefill worker — the frontend discovers and uses it automatically.

## Troubleshooting

**Workers fail to start with NIXL errors.** Ensure NIXL is installed and side-channel ports don't conflict. Each worker in a multi-worker setup needs a unique side-channel port.

## See also

* **[Disaggregated Serving (architecture)](/dynamo/dev/kubernetes/model-deployment/disaggregated-serving)** — design and rationale
* **[KV-Aware Routing](/dynamo/dev/cli/model-deployment/kv-aware-routing)** — route across prefill/decode pools
* **[Tuning Disaggregated Performance](/dynamo/dev/kubernetes/operations/performance-tuning)** — P/D tuning guide
* **[vLLM local deployment examples](/dynamo/dev/recipes/cli-templates/v-llm)** — copyable commands for each launch script