> For clean Markdown content of this page, append .md to this URL. For the complete documentation index, see https://docs.nvidia.com/dynamo/llms.txt. For full content including API reference and SDK examples, see https://docs.nvidia.com/dynamo/llms-full.txt.

# Multimodal Model Serving

Dynamo supports multimodal inference across multiple LLM backends, enabling models to process images, video, and audio alongside text.

## Which Feature to Use

Dynamo provides support for improving latency and throughput for multimodal workloads, with image and video inputs, through the following features. Use them together or separately, depending on your workload characteristics:

| Workload | Feature | Benefit |
|----------|---------|---------|
| Workload where significant time is spent preprocessing. | Frontend media decoding | Move base64 decoding and pixel conversion off the worker's critical path, and use all CPU cores to preprocess in parallel. |
| Workload includes repeated multimodal content across requests. | [Embedding cache](/dynamo/dev/multimodal/embedding-cache) | Skip re-encoding repeated multimodal content. |
| Workload includes repeated multimodal content across requests, and multiple backend workers serve multimodal requests. | [Multimodal KV routing](/dynamo/dev/multimodal/multimodal-kv-routing) | Maximize KV cache hit rates for multimodal content. |
| Workload where media encoding is a bottleneck. | [EPD disaggregation](/dynamo/dev/multimodal/encoder-disaggregation) | Scale encoders independently of LLM workers. |


<Info>
These features currently support image and video inputs only. Support for audio modalities will be added in upcoming releases.
</Info>

## Multimodal Performance Optimization Features

<CardGroup cols={2}>
  <Card title="Frontend Media Decoding" icon="regular image">
    Move base64 decoding and pixel conversion off the worker's critical path. Documentation coming soon.
  </Card>
  <Card title="Embedding Cache" icon="regular database" href="/dynamo/dev/multimodal/embedding-cache">
    Cache vision encoder embeddings to skip re-encoding repeated multimodal content
  </Card>
  <Card title="Multimodal KV Routing" icon="regular arrows-split-up-and-left" href="/dynamo/dev/multimodal/multimodal-kv-routing">
    Route multimodal requests to workers with the best KV cache overlap
  </Card>
  <Card title="EPD Disaggregation" icon="regular microchip" href="/dynamo/dev/multimodal/encoder-disaggregation">
    Separate vision encoding into a dedicated worker for independent scaling
  </Card>
</CardGroup>

## Example Workflows

Reference implementations for deploying multimodal models for each backend:

- [SGLang Multimodal](/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/sglang-multimodal)
- [TensorRT-LLM Multimodal](/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/tensorrt-llm-multimodal)
- [vLLM Multimodal](/dynamo/dev/knowledge-base/modular-components/backends/v-llm/vllm-multimodal)

To use an author-provided custom vision tower or projector, see [Custom Vision Encoders](/dynamo/dev/advanced-customizations/custom-vision-encoders).