Skip to navigation
DiffusionWorkflows

Image-to-Video

Animate source images with vLLM-Omni or SGLang

View as Markdown

Choose a backend for image-to-video generation. See the Diffusion Overview for installation and shared configuration.

Image-to-video (I2V) uses the same /v1/videos endpoint as text-to-video, with an additional input_reference field that provides the source image. Run a vLLM-Omni worker with --output-modalities video.

Tested Models

ModelNotes
Wan-AI/Wan2.2-TI2V-5B-DiffusersDefault model
Wan-AI/Wan2.2-I2V-A14B-DiffusersDual-expert MoE (model card)

For experimental Cosmos3 Nano reference conditioning, use the Cosmos3 example, which specifies its model revision, runtime requirements, and guardrail settings.

To run a non-default model, pass --model to the launch script:

bash examples/backends/vllm/launch/agg_omni_i2v.sh --model Wan-AI/Wan2.2-I2V-A14B-Diffusers

Launch

Launch with the provided script using Wan-AI/Wan2.2-TI2V-5B-Diffusers:

bash examples/backends/vllm/launch/agg_omni_i2v.sh

Generate a Video from an Image

curl -s http://localhost:8000/v1/videos \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Wan-AI/Wan2.2-TI2V-5B-Diffusers",
    "prompt": "A bear playing with yarn, smooth motion",
    "input_reference": "https://example.com/bear.png",
    "size": "832x480",
    "response_format": "url",
    "nvext": {
      "num_inference_steps": 40,
      "num_frames": 33,
      "guidance_scale": 1.0,
      "boundary_ratio": 0.875,
      "guidance_scale_2": 1.0,
      "seed": 42
    }
  }'

The input_reference field accepts:

  • HTTP/HTTPS URL: "https://example.com/image.png"
  • Base64 data URI: "data:image/png;base64,iVBORw0KGgo..."

Worker-local paths and file:// URLs are rejected. To use a client-local image, encode it as a data URI before sending the request, as the Cosmos3 example does.

Request Parameters (nvext)

I2V accepts all text-to-video nvext fields, plus two fields that control the dual-expert MoE denoising schedule in Wan2.x models:

nvext.boundary_ratio
floatDefaults to 0.875

MoE expert switching boundary (I2V).

nvext.guidance_scale_2
floatDefaults to 1.0

CFG scale for the low-noise expert (I2V).

See Also