Model CoverageDiffusion

LTX-2.3

View as Markdown

LTX-2.3 is a text-to-video diffusion model from Lightricks. Its dual-stream transformer denoises video and audio latents together so generated clips can include synchronized sound.

TaskText-to-Video with Audio
ArchitectureDual-stream DiT (Flow Matching)
HF OrgLightricks

Available Models

LTX-2.3 supports full-parameter and LoRA fine-tuning with synchronized video and audio.

Example Recipes

RecipeDescription
ltx2_3_t2v_flow.yamlFull fine-tuning with flow matching
ltx2_3_t2v_flow_lora.yamlParameter-efficient LoRA fine-tuning
generate_ltx2.yamlGenerate video and mux the model’s audio into the output

Prepare Video and Audio Data

Install the diffusion and media dependencies, then preprocess clips with a fixed frame count. LTX-2.3 samples video at 24 FPS and trims or pads each audio track to the matching duration.

$uv sync --extra diffusion --extra diffusion-media
$uv run python -m tools.diffusion.preprocessing_multiprocess video \
>  --video_dir /path/to/videos \
>  --output_dir /path/to/ltx2-cache \
>  --processor ltx2 \
>  --num_frames 121 \
>  --output_format pt \
>  --resolution_preset 512p \
>  --caption_format sidecar

LTX-2.3 requires an explicit frame count of the form 8n+1, such as 9, 89, or 121. Clips without an audio stream use silence while preserving the video and audio alignment contract.

Fine-Tune LTX-2.3

Set data.dataloader.cache_dir and checkpoint.checkpoint_dir in the selected recipe, then launch one process per GPU.

$uv run torchrun --nproc-per-node=8 \
>  examples/diffusion/finetune/finetune.py \
>  -c examples/diffusion/finetune/ltx2_3_t2v_flow.yaml

For LoRA, use the LoRA recipe instead:

$uv run torchrun --nproc-per-node=8 \
>  examples/diffusion/finetune/finetune.py \
>  -c examples/diffusion/finetune/ltx2_3_t2v_flow_lora.yaml

Both recipes use flow_matching.adapter_type: ltx2. The adapter applies the same sampled noise level to the video and audio streams and combines their flow-matching losses.

Generate Video with Audio

Point model.checkpoint at a consolidated full-fine-tuning checkpoint, or set model.lora_weights for a LoRA checkpoint.

$uv run python examples/diffusion/generate/generate.py \
>  -c examples/diffusion/generate/configs/generate_ltx2.yaml \
>  --model.checkpoint /path/to/checkpoint

The generation command writes an MP4 and muxes the generated waveform into it. See the Diffusion Fine-Tuning Guide and Diffusion Dataset Preparation guide for shared training and dataset options.

Hugging Face Model Cards