DiffusionWorkflows

Text-to-Video

Generate videos from text prompts with vLLM-Omni, SGLang, TensorRT-LLM, or FastVideo

View as Markdown

Choose a backend for text-to-video generation. See the Diffusion Overview for installation and shared configuration.

Text-to-video generation runs a vLLM-Omni worker with --output-modalities video.

Tested Models

ModelNotes
Wan-AI/Wan2.1-T2V-1.3B-DiffusersDefault model (1 GPU)
Wan-AI/Wan2.2-T2V-A14B-Diffusers

To run a non-default model, pass --model to the launch script:

$bash examples/backends/vllm/launch/agg_omni_video.sh --model Wan-AI/Wan2.2-T2V-A14B-Diffusers

Launch

Launch using the provided script with Wan-AI/Wan2.1-T2V-1.3B-Diffusers:

$bash examples/backends/vllm/launch/agg_omni_video.sh

Generate a Video

Generate a video via /v1/videos:

$curl -s http://localhost:8000/v1/videos \
>  -H "Content-Type: application/json" \
>  -d '{
>    "model": "Wan-AI/Wan2.1-T2V-1.3B-Diffusers",
>    "prompt": "A drone flyover of a mountain landscape",
>    "seconds": 2,
>    "size": "832x480",
>    "response_format": "url"
>  }'

The response returns a video URL or base64 data depending on response_format (e.g. {"object": "video", "status": "completed", "data": [{"url": "file:///tmp/dynamo_media/videos/req-abc123.mp4"}]}).

Request Parameters (nvext)

The /v1/videos endpoint also accepts NVIDIA extensions via the nvext field for fine-grained control:

nvext.fps
intDefaults to 24

Frames per second.

nvext.num_frames
int

Number of frames (overrides fps * seconds).

nvext.negative_prompt
string

Negative prompt for guidance.

nvext.num_inference_steps
intDefaults to 50

Number of denoising steps.

nvext.guidance_scale
floatDefaults to 5.0

CFG guidance scale.

nvext.seed
int

Random seed for reproducibility.

The nvext.boundary_ratio and nvext.guidance_scale_2 fields apply to the dual-expert MoE schedule used in image-to-video. See Image-to-Video with vLLM-Omni.

See Also