> For clean Markdown content of this page, append .md to this URL. For the complete documentation index, see https://docs.nvidia.com/dynamo/llms.txt. For full content including API reference and SDK examples, see https://docs.nvidia.com/dynamo/llms-full.txt. # Encoder Disaggregation ## Overview Encoder disaggregation separates the vision encoder from the prefill/decode pipeline into its own worker. Instead of running image encoding inline, a dedicated encode worker handles media processing and transfers the resulting embeddings to downstream workers via NIXL (RDMA). This enables: - Independent scaling of encode workers based on vision workload - Reduced GPU memory pressure on prefill/decode workers - Better GPU utilization by matching worker counts to actual bottlenecks ## When to Use Use encoder disaggregation when: - Vision encoding is a bottleneck and you need to scale encoders independently of LLM workers - You want to run the vision encoder on different hardware (e.g., smaller GPUs for encoding, larger GPUs for LLM inference) - Your deployment handles high volumes of multimodal requests and encoding throughput is limiting For simple deployments or development/testing, the aggregated (EPD) pattern is easier to set up. ## Deployment Patterns **Separate encoder, combined prefill and decode** Frontend → Processor → Encode Worker → PD Worker → Response The encode worker runs the vision model and transfers embeddings over NIXL to the combined prefill and decode worker. **Separate encoder, prefill, and decode** Frontend → Processor → Encode Worker → Prefill Worker → Decode Worker → Response The encode worker transfers embeddings over NIXL to the prefill worker. The prefill worker then transfers the KV cache to the decode worker. ## Launching ```bash cd $DYNAMO_HOME/examples/backends/vllm # E/PD bash launch/disagg_multimodal_e_pd.sh --model "Qwen/Qwen3-VL-30B-A3B-Instruct-FP8" # E/P/D bash launch/disagg_multimodal_epd.sh --model "Qwen/Qwen3-VL-30B-A3B-Instruct-FP8" ``` ```bash cd $DYNAMO_HOME/examples/backends/sglang # E/PD ./launch/multimodal_epd.sh # E/P/D ./launch/multimodal_disagg.sh ``` ```bash cd $DYNAMO_HOME/examples/backends/trtllm # E/PD bash launch/disagg_e_pd.sh # E/P/D ./launch/epd_multimodal_image_and_embeddings.sh ``` See the backend-specific documentation ([vLLM](/dynamo/dev/knowledge-base/modular-components/backends/v-llm/vllm-multimodal), [TRT-LLM](/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/tensorrt-llm-multimodal), [SGLang](/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/sglang-multimodal)) for full configuration details and component flags. ## Support Matrix | Backend | E/PD | E/P/D | Notes | |---------|------|-------|-------| | **vLLM** | Yes | Yes | Separate encode worker currently handles `image_url` inputs; `video_url` inputs stay on the prefill/PD path | | **TensorRT-LLM** | — | Yes | Supports image URLs (via `MultimodalEncoder`) and pre-computed embeddings (via NIXL) | | **SGLang** | Yes | Yes | NIXL for embeddings; bootstrap mechanism for P/D KV transfer |