Multimodal Model Serving | NVIDIA Dynamo Documentation

Dynamo supports multimodal inference across multiple LLM backends, enabling models to process images, video, and audio alongside text.

Which Feature to Use

Dynamo provides support for improving latency and throughput for multimodal workloads, with image and video inputs, through the following features. Use them together or separately, depending on your workload characteristics:

Workload	Feature	Benefit
Workload where significant time is spent preprocessing.	Parallel media decoding	Move image fetching and decompression off the worker’s critical path.
Workload includes repeated multimodal content across requests.	Embedding cache	Skip re-encoding repeated multimodal content.
Workload includes repeated multimodal content across requests, and multiple backend workers serve multimodal requests.	Multimodal KV routing	Maximize KV cache hit rates for multimodal content.
Workload where media encoding is a bottleneck.	Encoder disaggregation	Scale encoders independently of LLM workers.

These features currently support image and video inputs only. Support for audio modalities will be added in upcoming releases.

Multimodal Performance Optimization Features

Parallel Media Decoding

Decode image inputs concurrently in the Rust frontend and transfer pixels through NIXL

Embedding Cache

Cache vision encoder embeddings to skip re-encoding repeated multimodal content

Multimodal KV Routing

Include multimodal identity in cache-aware, load-balanced worker selection

Encoder Disaggregation

Separate vision encoding into a dedicated worker for independent scaling

Example Workflows

Reference implementations for deploying multimodal models for each backend:

To use an author-provided custom vision tower or projector, see Custom Vision Encoders.