Diffusion
Overview
Dynamo serves diffusion models for text-to-image, text-to-video, image-to-video, text-to-audio, and text-to-text generation. These workloads use the same frontend, routing, scaling, and observability infrastructure as LLM inference.
The built-in backends expose OpenAI-compatible endpoints for images (/v1/images/generations), videos (/v1/videos), audio (/v1/audio/speech), and text generation.
Built-in diffusion backends currently use Dynamo CLI launch scripts and python -m dynamo.* entrypoints. Dynamo does not yet ship prebuilt Kubernetes recipes for these backends. FastVideo is the exception and includes a Kubernetes deployment path.
Choose a Workflow
Start with the type of content you want to generate.
Generate images from text prompts.
Backends: vLLM-Omni, SGLang, and TensorRT-LLM
Generate video clips from text prompts.
Animate a source image from a text prompt.
Backends: vLLM-Omni and SGLang
Generate speech from text.
Backend: vLLM-Omni
Generate text through iterative diffusion rather than autoregressive decoding.
Backend: SGLang
Choose a Backend
Compare backend coverage and deployment constraints.
Best for: Mixed media workloads, text-to-audio, or disaggregated multi-stage serving.
Supports: Text-to-image, text-to-video, image-to-video, and text-to-audio.
Main limitation: Each worker serves one output modality at a time. Audio streaming and voice cloning are not yet supported.
Best for: Diffusion alongside an existing SGLang deployment or text-to-text diffusion models.
Supports: Text-to-image, text-to-video, image-to-video, and text-to-text.
Main limitation: Text-to-audio is not supported.
Best for: NVIDIA-optimized image and video generation when experimental status is acceptable.
Supports: Text-to-image and text-to-video.
Main limitation: Not recommended for production. Video output requires an NVENC-capable NVIDIA GPU.
Best for: Fast, production-oriented text-to-video generation on Kubernetes.
Supports: Text-to-video with a distilled LTX-2 model and five-step inference.
Main limitation: Uses a purpose-built runtime image that can take 20–40 minutes or longer to build initially.
Install the Backend
Choose the tab for the backend you plan to use.
vLLM-Omni
SGLang
TensorRT-LLM
FastVideo
vLLM-Omni provides image, video, and audio generation through python -m dynamo.vllm.omni.
Prerequisites
- A working vLLM backend setup.
- An
amd64host. Dynamo container builds do not install vLLM-Omni onarm64.
Dynamo container images include vLLM-Omni. A PyPI installation of ai-dynamo[vllm] does not include it automatically. For a source installation, pin the vLLM-Omni release that matches your vLLM version:
Local launchers, media storage, and stage configuration
The agg_omni_*.sh and disagg_omni_glm_image.sh scripts launch the frontend and workers directly on one host. They are intended for local development and testing and do not create a DynamoGraphDeployment.
Generated media uses fsspec storage. The default is file:///tmp/dynamo_media. Set --media-output-fs-url for S3, GCS, or Azure Blob storage, and optionally set --media-output-http-url to rewrite response URLs.
vLLM-Omni includes built-in YAML stage configurations for supported models. Set --stage-configs-path only to override them.
See the vLLM-Omni Configuration reference for the complete flag surface.