vLLM-Omni Configuration | NVIDIA Dynamo Documentation

The vLLM-Omni backend uses a dedicated entrypoint: python -m dynamo.vllm.omni. This page documents its CLI flags. For deployment guides and per-modality examples, see the Diffusion Overview.

These are the vLLM-Omni orchestration flags. The omni worker also accepts the same underlying vLLM engine arguments and the cross-cutting Dynamo Runtime flags (--namespace, --endpoint, and others).

General Flags

--omni

flag

Enable the vLLM-Omni orchestrator (required for all omni workloads).

--output-modalities

text | image | video | audio

Output modality the worker registers endpoints for.

--stage-configs-path

path

Path to stage config YAML (optional; vLLM-Omni uses model defaults if omitted).

--boundary-ratio

float

MoE expert switching boundary. If omitted, the vLLM-Omni model pipeline selects its default.

--flow-shift

float

Scheduler flow_shift (5.0 for 720p, 12.0 for 480p).

--default-video-fps

intDefaults to 16

Default frames per second for generated videos.

--task-type

string

Model-defined startup task or checkpoint partition.

--lora-path

path ...

Diffusion checkpoint adapter path(s) fused into the model at startup. This is separate from request-time LoRA loading.

--diffusion-attention-backend

string

Default diffusion attention backend, such as TRTLLM_ATTN, FLASH_ATTN, or FASTVIDEO_VSA.

--fastvideo-vsa-topk

int

Key/value blocks retained per query block by FASTVIDEO_VSA. The value must be positive; FastH3 VSA starts with 64.

Memory & Performance

--vae-use-slicing

flag

Enable VAE slicing for memory optimization.

--vae-use-tiling

flag

Enable VAE tiling for memory optimization.

--enable-layerwise-offload

flag

Enable layerwise offloading on DiT modules to reduce GPU memory.

--layerwise-num-gpu-layers

intDefaults to 1

Number of ready layers to keep on GPU during generation.

--enforce-eager

flag

Disable torch.compile for diffusion models.

--enable-cpu-offload

flag

Enable CPU offloading for diffusion models.

Diffusion Cache

--cache-backend

cache_dit | tea_cache

Diffusion cache backend.

--cache-config

json

Cache configuration as a JSON string (overrides defaults).

--enable-cache-dit-summary

flag

Enable cache-dit summary logging after diffusion forward passes.

Parallelism

--ulysses-degree

intDefaults to 1

GPUs for Ulysses sequence parallelism in diffusion.

--ring-degree

intDefaults to 1

GPUs for ring sequence parallelism in diffusion.

--cfg-parallel-size

intDefaults to 1

GPUs for classifier-free guidance parallelism (1 or 2).

--vae-patch-parallel-size

intDefaults to 1

Ranks used for VAE patch or tile parallelism.

--text-encoder-tp-size

intDefaults to 1

Tensor-parallel ranks used by the diffusion text encoder.

Disaggregated Mode

See vLLM-Omni Disaggregated Serving for usage.

--stage-id

int

Run as a single-stage worker for the given stage ID. Requires --stage-configs-path.

--omni-router

flag

Run as the stage router. Requires --stage-configs-path. Mutually exclusive with --stage-id.

Media Storage

--media-output-fs-url

urlDefaults to file:///tmp/dynamo_media

Filesystem URL for storing generated media.

--media-output-http-url

url

Base URL for rewriting media paths in responses (optional).