Skip to navigation
Model CoverageOmniOmni Models

Omni Models

View as Markdown

Omni models go beyond image-text understanding to support additional modalities such as audio, video, or a combination of all — text, image, audio, and video in a single unified model.

Run Omni Models with NeMo AutoModel

To run omni models with NeMo AutoModel, use NeMo container version 26.10.00 or later. If the model you want to fine-tune requires newer source than the container provides, use a current checkout so the package and its example recipes stay on one revision:

git clone https://github.com/NVIDIA-NeMo/Automodel.git
cd Automodel
uv sync --locked --all-groups --all-extras --extra vlm-media

For other installation options, see our NeMo AutoModel Installation Guide.

Omni models such as Qwen3-Omni need the vlm-media extra included in the source-sync command above for video/vision preprocessing; it is not installed in the Docker container by default. See Media Extras.

Automatic speech recognition: Qwen2.5-Omni and Qwen3-Omni support supervised ASR fine-tuning on Hugging Face audio datasets. See the Qwen3-Omni ASR fine-tuning guide and the ASR recipes on the model cards below.

Supported Models

This table combines recipe-backed checkpoints with documented model families. Dates show when the current checkpoint first appeared in a recipe, or when a documentation-only family page was added. See the combined model support log for recipe-backed checkpoints of every model type.

Fine-Tune Omni Models

All supported omni models can be fine-tuned using full SFT or PEFT (LoRA) approaches. See the VLM Fine-Tuning Guide for general setup instructions.