vLLM-Omni Configuration

CLI flag reference for the vLLM-Omni multimodal backend (python -m dynamo.vllm.omni).

View as Markdown

The vLLM-Omni backend uses a dedicated entrypoint: python -m dynamo.vllm.omni. This page documents its CLI flags. For deployment guides and per-modality examples, see the Diffusion Overview.

These are the vLLM-Omni orchestration flags. The omni worker also accepts the same underlying vLLM engine arguments and the cross-cutting Dynamo Runtime flags (--namespace, --endpoint, and others).

General Flags

--omni
flag

Enable the vLLM-Omni orchestrator (required for all omni workloads).

--output-modalities
text | image | video | audio

Output modality the worker registers endpoints for.

--stage-configs-path
path

Path to stage config YAML (optional; vLLM-Omni uses model defaults if omitted).

--boundary-ratio
floatDefaults to 0.875

MoE expert switching boundary.

--flow-shift
float

Scheduler flow_shift (5.0 for 720p, 12.0 for 480p).

--default-video-fps
intDefaults to 16

Default frames per second for generated videos.

Memory & Performance

--vae-use-slicing
flag

Enable VAE slicing for memory optimization.

--vae-use-tiling
flag

Enable VAE tiling for memory optimization.

--enable-layerwise-offload
flag

Enable layerwise offloading on DiT modules to reduce GPU memory.

--layerwise-num-gpu-layers
intDefaults to 1

Number of ready layers to keep on GPU during generation.

--enforce-eager
flag

Disable torch.compile for diffusion models.

--enable-cpu-offload
flag

Enable CPU offloading for diffusion models.

Diffusion Cache

--cache-backend
cache_dit | tea_cache

Diffusion cache backend.

--cache-config
json

Cache configuration as a JSON string (overrides defaults).

--enable-cache-dit-summary
flag

Enable cache-dit summary logging after diffusion forward passes.

Parallelism

--ulysses-degree
intDefaults to 1

GPUs for Ulysses sequence parallelism in diffusion.

--ring-degree
intDefaults to 1

GPUs for ring sequence parallelism in diffusion.

--cfg-parallel-size
intDefaults to 1

GPUs for classifier-free guidance parallelism (1 or 2).

Disaggregated Mode

See vLLM-Omni Disaggregated Serving for usage.

--stage-id
int

Run as a single-stage worker for the given stage ID. Requires --stage-configs-path.

--omni-router
flag

Run as the stage router. Requires --stage-configs-path. Mutually exclusive with --stage-id.

Media Storage

--media-output-fs-url
urlDefaults to file:///tmp/dynamo_media

Filesystem URL for storing generated media.

--media-output-http-url
url

Base URL for rewriting media paths in responses (optional).