> For clean Markdown content of this page, append .md to this URL. For the complete documentation index, see https://docs.nvidia.com/dynamo/llms.txt. For full content including API reference and SDK examples, see https://docs.nvidia.com/dynamo/llms-full.txt.

# VLM Performance Optimization

Once you can serve a VLM in Dynamo (see [Image Understanding](/dynamo/dev/image-understanding/overview)),
these features cut the time a vision language model (VLM) spends on multiple
stages in the pipeline: fetching media, finding the right worker, running the
vision encoder, and prefill. Depending on your workload characteristics, you may
benefit from using all of these features or only some of them.

## When Each Feature Helps

![Where is the bottleneck in your pipeline? A VLM request has six steps: the frontend receives it, the router picks a worker, the worker loads the image, runs the vision encoder, prefills, and decodes. Multimodal KV routing helps at the router. Frontend media decoding and the shared image cache help when loading the image. The embedding cache and encoder disaggregation help at the encoder.](/dynamo/_fern-files/dynamo.docs.buildwithfern.com/85d8beeb5b8887c1bf7127c9eff17d373e28ee88bf7420fa176d7a4fb2bbac33/assets/img/vlm-request-path.svg)

| Feature | Use it when | Supported modalities |
| --- | --- | --- |
| [Multimodal KV routing](/dynamo/dev/image-understanding/vlm-performance-optimization/multimodal-kv-routing) sends repeat requests to the worker that already holds their KV cache | The same images or clips repeat across requests and you run several workers | Image and video |
| [Frontend media decoding](/dynamo/dev/image-understanding/vlm-performance-optimization/frontend-media-decoding) moves media decoding to the Rust frontend | Fetching and decoding media takes a large share of request latency or worker CPU time | Image and VP8/VP9 video |
| [Shared image cache](/dynamo/dev/image-understanding/vlm-performance-optimization/shared-image-cache) reuses downloaded image bytes across workers | Several workers download the same image URLs | Image URLs |
| [Embedding cache](/dynamo/dev/image-understanding/vlm-performance-optimization/embedding-cache) skips re-encoding repeated images and clips | The same images or clips repeat and the vision encoder is busy | Image and video |
| [Encoder disaggregation](/dynamo/dev/image-understanding/vlm-performance-optimization/encoder-disaggregation) scales vision encoders separately from LLM workers | Vision encoding is the bottleneck, or encoders need different hardware than the LLM | Image (vLLM video stays on the PD path) |

If your workload is all unique media, the caches and routing give little benefit.
Start with frontend media decoding or encoder disaggregation instead.

## Support Matrix

| Feature | vLLM | SGLang | TensorRT-LLM |
| --- | --- | --- | --- |
| [Multimodal KV routing](/dynamo/dev/image-understanding/vlm-performance-optimization/multimodal-kv-routing) | Image and video | Image, and video on aggregated CUDA workers | Image only |
| [Frontend media decoding](/dynamo/dev/image-understanding/vlm-performance-optimization/frontend-media-decoding) | Image and VP8/VP9 video, aggregated | Image, and VP8/VP9 video on CUDA, aggregated | Image, aggregated |
| [Shared image cache](/dynamo/dev/image-understanding/vlm-performance-optimization/shared-image-cache) | Image URLs loaded through Dynamo | Not supported for standard URL loading | Image URLs loaded through Dynamo |
| [Embedding cache](/dynamo/dev/image-understanding/vlm-performance-optimization/embedding-cache) | Aggregated and E/PD | E/PD | E/PD |
| [Encoder disaggregation](/dynamo/dev/image-understanding/vlm-performance-optimization/encoder-disaggregation) | E/PD and E/P/D (images only; video stays on the PD path) | E/PD and E/P/D | E/PD and E/P/D (images and pre-computed embeddings) |

Each feature page lists the supported models, flags, and limitations for its
backends.

> Speed up image and video requests with routing, decoding, caching, and encoder disaggregation

## Docs

- [Multimodal KV Routing](https://docs.nvidia.com/dynamo/dev/image-understanding/vlm-performance-optimization/multimodal-kv-routing.md): Route repeated images and videos to the worker that already holds their KV cache
- [Frontend Media Decoding](https://docs.nvidia.com/dynamo/dev/image-understanding/vlm-performance-optimization/frontend-media-decoding.md): Decode images and video in the Rust frontend instead of the backend worker
- [Shared Image Cache](https://docs.nvidia.com/dynamo/dev/image-understanding/vlm-performance-optimization/shared-image-cache.md): Reuse downloaded image bytes across workers
- [Embedding Cache](https://docs.nvidia.com/dynamo/dev/image-understanding/vlm-performance-optimization/embedding-cache.md): Skip the vision encoder for images and clips it has already encoded
- [Encoder Disaggregation](https://docs.nvidia.com/dynamo/dev/image-understanding/vlm-performance-optimization/encoder-disaggregation.md): Scale vision encoders separately from LLM workers