Multimodal Model Serving | NVIDIA Dynamo Documentation

Dynamo supports multimodal inference across multiple LLM backends, enabling models to process images, video, and audio alongside text.

Which Feature to Use

Dynamo provides support for improving latency and throughput for multimodal workloads, with image and video inputs, through the following features. Use them together or separately, depending on your workload characteristics:

Workload	Feature	Benefit
Workload where significant time is spent preprocessing.	Frontend media decoding	Move base64 decoding and pixel conversion off the worker’s critical path, and use all CPU cores to preprocess in parallel.
Workload includes repeated multimodal content across requests.	Embedding cache	Skip re-encoding repeated multimodal content.
Workload includes repeated multimodal content across requests, and multiple backend workers serve multimodal requests.	Multimodal KV routing	Maximize KV cache hit rates for multimodal content.
Workload where media encoding is a bottleneck.	EPD disaggregation	Scale encoders independently of LLM workers.

These features currently support image and video inputs only. Support for audio modalities will be added in upcoming releases.

Multimodal Performance Optimization Features

Frontend Media Decoding

Move base64 decoding and pixel conversion off the worker’s critical path. Documentation coming soon.

Embedding Cache

Cache vision encoder embeddings to skip re-encoding repeated multimodal content

Multimodal KV Routing

Route multimodal requests to workers with the best KV cache overlap

EPD Disaggregation

Separate vision encoding into a dedicated worker for independent scaling

Example Workflows

Reference implementations for deploying multimodal models for each backend:

To use an author-provided custom vision tower or projector, see Custom Vision Encoders.