Omni Models

Omni models go beyond image-text understanding to support additional modalities such as audio, video, or a combination of all — text, image, audio, and video in a single unified model.

Run Omni Models with NeMo AutoModel

To run omni models with NeMo AutoModel, use NeMo container version 26.06.00 or later. If the model you want to fine-tune requires newer source than the container provides, use a current checkout so the package and its example recipes stay on one revision:

$ git clone https://github.com/NVIDIA-NeMo/Automodel.git
$ cd Automodel
$ uv sync --locked --all-groups --all-extras --extra vlm-media

For other installation options, see our NeMo AutoModel Installation Guide.

Omni models such as Qwen3-Omni need the vlm-media extra included in the source-sync command above for video/vision preprocessing; it is not installed in the Docker container by default. See Media Extras.

Automatic speech recognition: Qwen2.5-Omni and Qwen3-Omni support supervised ASR fine-tuning on Hugging Face audio datasets. See the Qwen3-Omni ASR fine-tuning guide and the ASR recipes on the model cards below.

Supported Models

This table combines recipe-backed checkpoints with documented model families. Dates show when the current checkpoint first appeared in a recipe, or when a documentation-only family page was added. See the combined model support log for recipe-backed checkpoints of every model type.

Date	Model	Architectures	Recipe
2026-05-29	Qwen2.5-Omni-7B	`Qwen2_5OmniForConditionalGeneration`	recipe
2026-05-29	Qwen2.5-Omni-3B	`Qwen2_5OmniForConditionalGeneration`	recipe
2026-04-27	Nemotron-3-Nano-Omni (documentation)	`NemotronH_Nano_Omni_Reasoning_V3`
2025-11-10	Qwen3-Omni-30B-A3B-Instruct	`Qwen3OmniForConditionalGeneration`	recipe
2025-07-25	Phi-4-multimodal-instruct	`Phi4MultimodalForCausalLM`	recipe

Fine-Tune Omni Models

All supported omni models can be fine-tuned using full SFT or PEFT (LoRA) approaches. See the VLM Fine-Tuning Guide for general setup instructions.