vLLM-Omni Configuration | NVIDIA Dynamo Documentation

The vLLM-Omni backend uses a dedicated entrypoint: python -m dynamo.vllm.omni. This page documents its CLI flags. For deployment guides and per-modality examples, see the Diffusion Overview.

These are the vLLM-Omni orchestration flags. The omni worker also accepts the same underlying vLLM engine arguments and the cross-cutting Dynamo Runtime flags (--namespace, --endpoint, and others).

General Flags

--omni

flag

Enable the vLLM-Omni orchestrator (required for all omni workloads).

--output-modalities

text | image | video | audio

Output modality the worker registers endpoints for.

--stage-configs-path

path

Path to stage config YAML (optional; vLLM-Omni uses model defaults if omitted).

--boundary-ratio

float

MoE expert switching boundary. If omitted, the vLLM-Omni model pipeline selects its default.

--flow-shift

float

Scheduler flow_shift (5.0 for 720p, 12.0 for 480p).

--default-video-fps

intDefaults to 16

Default frames per second for generated videos.

--task-type

string

Model-defined startup task or checkpoint partition.

--no-guardrails

flagDefaults to false

Disable native model guardrails, such as Cosmos3 guardrails, for an aggregated Omni worker. The environment equivalent is DYN_OMNI_NO_GUARDRAILS=true. Omitting the option preserves the model’s guardrail configuration and its dependency requirements. Standalone stages, the stage router, and realtime workers reject this option.

--lora-path

path ...

Diffusion checkpoint adapter path(s) fused into the model at startup. This is separate from request-time LoRA loading.

--diffusion-attention-backend

string

Default diffusion attention backend, such as TRTLLM_ATTN, FLASH_ATTN, or FASTVIDEO_VSA.

--fastvideo-vsa-topk

int

Key/value blocks retained per query block by FASTVIDEO_VSA. The value must be positive; FastH3 VSA starts with 64.

Image Edit Capabilities

Experimental. The maximum number of input images depends on the loaded model, not the model name clients send. Models with unknown metadata accept at most one reference; the model must still support image conditioning.

Use an aggregated Omni worker for image edits. Stage-router deployments reject edit references and continue to support text-to-image generation.

These limits come from vLLM-Omni 0.31.0rc1 registry metadata and may change with the installed version:

Model pipeline	Maximum input images
`Flux2KleinPipeline`	4
`QwenImageEditPlusPipeline`	4
`HunyuanImage3Pipeline`	3
`BooguImagePipeline`	1

Only FLUX.2 Klein 4B has been GPU-qualified for this integration; the other limits come from registry metadata, not GPU testing.

See the Image Edits API for request formats, input budgets, configuration placement, errors, and rolling upgrades.

Memory & Performance

--image-png-compression-level

intDefaults to 6

Lossless PNG compression level for generated images, from 0 to 9. 0 disables compression for faster CPU encoding and larger responses; 1 uses fast compression, and 9 uses the strongest compression. The default 6 preserves Pillow’s existing encoding behavior. All levels preserve pixels.

Set DYN_OMNI_IMAGE_PNG_COMPRESSION_LEVEL as an environment alternative; an explicit CLI value takes precedence. Invalid values fail startup. Set this on the aggregated worker, or on the --omni-router process in disaggregated mode. It applies to PNG images returned as base64, stored for URL responses, or embedded in chat responses. It does not change model inference or video encoding.

Measure both encoding time and end-to-end latency when tuning this setting: larger payloads can offset CPU savings on slower networks. Compare backends at matched PNG compression levels, and report first-request and warm latency separately from model-loading time.

--vae-use-slicing

flag

Enable VAE slicing for memory optimization.

--vae-use-tiling

flag

Enable VAE tiling for memory optimization.

--enable-layerwise-offload

flag

Enable layerwise offloading on DiT modules to reduce GPU memory.

--layerwise-num-gpu-layers

int

This legacy option is no longer supported and causes a configuration error when set. Remove the flag or unset DYN_OMNI_LAYERWISE_NUM_GPU_LAYERS. Use --enable-layerwise-offload without a layer-count override to let vLLM-Omni manage layerwise offloading.

--enforce-eager

flag

Disable torch.compile for diffusion models.

--enable-cpu-offload

flag

Enable CPU offloading for diffusion models.

Diffusion Cache

--cache-backend

cache_dit | tea_cache

Diffusion cache backend.

--cache-config

json

Cache configuration as a JSON string (overrides defaults).

--enable-cache-dit-summary

flag

Enable cache-dit summary logging after diffusion forward passes.

Parallelism

--ulysses-degree

intDefaults to 1

GPUs for Ulysses sequence parallelism in diffusion.

--ring-degree

intDefaults to 1

GPUs for ring sequence parallelism in diffusion.

--cfg-parallel-size

intDefaults to 1

GPUs for classifier-free guidance parallelism (1 or 2).

--vae-patch-parallel-size

intDefaults to 1

Ranks used for VAE patch or tile parallelism.

--text-encoder-tp-size

intDefaults to 1

Tensor-parallel ranks used by the diffusion text encoder.

Disaggregated Mode

See vLLM-Omni Disaggregated Serving for usage.

--stage-id

int

Run as a single-stage worker for the given stage ID. Requires --stage-configs-path.

--omni-router

flag

Run as the stage router. Requires --stage-configs-path. Mutually exclusive with --stage-id.

Media Storage

--media-output-fs-url

urlDefaults to file:///tmp/dynamo_media

Filesystem URL for storing generated media.

--media-output-http-url

url

Base URL for rewriting media paths in responses (optional).