> For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.nvidia.com/nemo/automodel/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/automodel/_mcp/server. # Vision Language Models (VLMs) ## Introduction Vision Language Models (VLMs) integrate vision and language processing capabilities, enabling models to understand images and generate text descriptions, answer visual questions, and perform multimodal reasoning. NeMo AutoModel LLM APIs can be easily extended to support VLM tasks. While most of the training setup is the same as for LLMs, some additional steps are required to prepare the data and model for VLM training. ## Run VLMs with NeMo AutoModel To run VLMs with NeMo AutoModel, use NeMo container version [`26.06.00`](https://catalog.ngc.nvidia.com/orgs/nvidia/containers/nemo-automodel?version=26.06.00) or later. If the model you want to fine-tune requires newer source than the container provides, use a current checkout so the package and its example recipes stay on one revision: ```bash git clone https://github.com/NVIDIA-NeMo/Automodel.git cd Automodel uv sync --locked --all-groups --all-extras --extra vlm-media ``` For other installation options, see our [Installation Guide](/get-started/installation). VLMs that decode video or use Qwen / Mistral vision preprocessing also need the `vlm-media` extra included in the source-sync command above; it is not installed in the Docker container by default. See [Media Extras](/get-started/installation#media-extras-video--image-decode). ## Supported Models NeMo AutoModel supports [AutoModelForImageTextToText](https://huggingface.co/docs/transformers/main/model_doc/auto#transformers.AutoModelForImageTextToText) in the [Image-Text-to-Text](https://huggingface.co/models?pipeline_tag=image-text-to-text\&sort=trending) category. This table combines recipe-backed checkpoints with documented model families. Dates show when the current checkpoint first appeared in a recipe, or when a documentation-only family page was added. See the [combined model support log](/model-coverage/release-log) for recipe-backed checkpoints of every model type. | Date | Model | Architectures | Recipe | | :--------- | :----------------------------------------------------------------------------------------------------------------------- | :----------------------------------------------------------------------------------------- | :--------------------------------------------------------------------------------------------------------------------------------- | | 2026-08-10 | [Muse-Glimmer-30B](/model-coverage/vision-language-models/muse/muse-glimmer) | `MuseGlimmerForConditionalGeneration` | [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/muse_glimmer/muse_glimmer_30b_medpix.yaml) | | 2026-07-30 | [Inkling-Small](/model-coverage/vision-language-models/thinkingmachines/inkling) | `InklingForConditionalGeneration` | [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/inkling/Inkling_small_medpix_ep64.yaml) | | 2026-07-29 | [Qwen3.5-122B-A10B](https://huggingface.co/Qwen/Qwen3.5-122B-A10B) | | [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/qwen3_5_moe/qwen3_5_122b_128k_ep8cp32.yaml) | | 2026-07-29 | [Kimi-K3](/model-coverage/vision-language-models/moonshotai/kimi-k3) (documentation) | | | | 2026-07-17 | [Inkling](/model-coverage/vision-language-models/thinkingmachines/inkling) | `InklingForConditionalGeneration` | [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/inkling/inkling_medpix.yaml) | | 2026-06-12 | [MiniMax-M3](/model-coverage/vision-language-models/minimax/minimax-m3) | `MiniMaxM3SparseForConditionalGeneration` — 428B total / 22B active MoE VLM | [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/minimax_m3/minimax_m3_vl_lora_pp4ep8_8node.yaml) | | 2026-05-29 | [Step-3.7-Flash](/model-coverage/vision-language-models/stepfun-ai/step-3-7-flash) | `Step3p7ForConditionalGeneration` — 198B total / 13B active MoE VLM | [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/stepfun/step3p7_medpix_200b_ep32pp4.yaml) | | 2026-04-29 | [Mistral-Medium-3.5-128B](https://huggingface.co/mistralai/Mistral-Medium-3.5-128B) | | [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/mistral3p5/mistral3p5_128b_medpix.yaml) | | 2026-04-29 | [Mistral Medium 3.5](/model-coverage/vision-language-models/mistralai/mistral-medium-3-5) (documentation) | `Mistral3ForConditionalGeneration` (Pixtral vision tower + dense Ministral-3 text decoder) | | | 2026-04-27 | [Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16](https://huggingface.co/nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16) | | [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/nemotron_omni/nemotron_omni_cord_v2.yaml) | | 2026-04-22 | [Qwen3.6-27B](/model-coverage/vision-language-models/qwen/qwen3-6-vl) | `Qwen3_5ForConditionalGeneration`, `Qwen3_5MoeForConditionalGeneration` | [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/qwen3_5/qwen3_6_27b.yaml) | | 2026-04-21 | [Qwen3.5-27B](https://huggingface.co/Qwen/Qwen3.5-27B) | | [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/qwen3_5/qwen3_5_27b_tp4pp4.yaml) | | 2026-04-17 | [LLaVA-OneVision-1.5-8B-Instruct](/model-coverage/vision-language-models/lmms-lab/llava-onevision) | `LlavaOneVisionForConditionalGeneration` | [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/llava_onevision/llava_ov_1_5_8b_lora.yaml) | | 2026-04-17 | [LLaVA-OneVision-1.5-4B-Instruct](/model-coverage/vision-language-models/lmms-lab/llava-onevision) | `LlavaOneVisionForConditionalGeneration` | [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/llava_onevision/llava_ov_1_5_4b_finetune.yaml) | | 2026-04-16 | [Qwen3.6-35B-A3B](/model-coverage/vision-language-models/qwen/qwen3-6-vl) | `Qwen3_5ForConditionalGeneration`, `Qwen3_5MoeForConditionalGeneration` | [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/qwen3_5_moe/qwen3_6_35b.yaml) | | 2026-04-16 | [Qwen3.5-397B-A17B](https://huggingface.co/Qwen/Qwen3.5-397B-A17B) | | [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/qwen3_5_moe/qwen3_5_moe_medpix.yaml) | | 2026-04-05 | [SmolVLM](/model-coverage/vision-language-models/huggingface/smolvlm) (documentation) | `SmolVLMForConditionalGeneration` | | | 2026-04-05 | [Qwen3.5](/model-coverage/vision-language-models/qwen/qwen3-5-vl) (documentation) | `Qwen3_5ForConditionalGeneration`, `Qwen3_5MoeForConditionalGeneration` | | | 2026-04-05 | [Ministral3 VL](/model-coverage/vision-language-models/mistralai/ministral3-vl) (documentation) | `Mistral3ForConditionalGeneration` | | | 2026-04-05 | [LLaVA](/model-coverage/vision-language-models/llava-hf/llava) (documentation) | `LlavaForConditionalGeneration` / `LlavaNextForConditionalGeneration` | | | 2026-04-05 | [Llama 4](/model-coverage/vision-language-models/meta/llama-4) (documentation) | `Llama4ForConditionalGeneration` | | | 2026-04-05 | [InternVL](/model-coverage/vision-language-models/internlm/internvl) (documentation) | `InternVLForConditionalGeneration` | | | 2026-04-03 | [Gemma-4-E4B-it](/model-coverage/vision-language-models/google/gemma-4) | `Gemma4UnifiedForConditionalGeneration`, `Gemma4ForConditionalGeneration` | [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/gemma4/gemma4_4b.yaml) | | 2026-04-03 | [Gemma-4-E2B-it](/model-coverage/vision-language-models/google/gemma-4) | `Gemma4UnifiedForConditionalGeneration`, `Gemma4ForConditionalGeneration` | [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/gemma4/gemma4_2b.yaml) | | 2026-04-03 | [Gemma-4-31B-it](/model-coverage/vision-language-models/google/gemma-4) | `Gemma4UnifiedForConditionalGeneration`, `Gemma4ForConditionalGeneration` | [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/gemma4/gemma4_31b.yaml) | | 2026-04-03 | [Gemma-4-26B-A4B-it](/model-coverage/vision-language-models/google/gemma-4) | `Gemma4UnifiedForConditionalGeneration`, `Gemma4ForConditionalGeneration` | [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/gemma4/gemma4_26b_a4b_moe.yaml) | | 2026-03-16 | [Mistral-Small-4-119B-2603](/model-coverage/vision-language-models/mistralai/mistral-small-4) | `MistralForConditionalGeneration` | [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/mistral4/mistral4_medpix.yaml) | | 2026-03-02 | [Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B) | | [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/qwen3_5/qwen3_5_9b.yaml) | | 2026-03-02 | [Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B) | | [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/qwen3_5/qwen3_5_4b.yaml) | | 2026-02-24 | [Qwen3.5-35B-A3B](https://huggingface.co/Qwen/Qwen3.5-35B-A3B) | | [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/qwen3_5_moe/qwen3_5_35b.yaml) | | 2026-02-19 | [Kimi-K2.5](https://huggingface.co/moonshotai/Kimi-K2.5) | | [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/kimi/kimi25vl_medpix.yaml) | | 2026-02-06 | [Qwen3-VL-235B-A22B-Instruct](https://huggingface.co/Qwen/Qwen3-VL-235B-A22B-Instruct) | | [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/qwen3/qwen3_vl_moe_235b.yaml) | | 2026-01-30 | [Kimi-VL-A3B-Instruct](/model-coverage/vision-language-models/moonshotai/kimi-vl) | `KimiVLForConditionalGeneration` | [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/kimi/kimi2vl_cordv2.yaml) | | 2026-01-12 | [NVIDIA-Nemotron-Parse-v1.1](/model-coverage/vision-language-models/nvidia/nemotron-parse) | `NemotronParseForConditionalGeneration` | [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/nemotron/nemotron_parse_v1_1.yaml) | | 2025-12-05 | [Ministral-3-8B-Reasoning-2512](https://huggingface.co/mistralai/Ministral-3-8B-Reasoning-2512) | | [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/mistral/ministral3_8b_medpix.yaml) | | 2025-12-05 | [Ministral-3-3B-Reasoning-2512](https://huggingface.co/mistralai/Ministral-3-3B-Reasoning-2512) | | [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/mistral/ministral3_3b_medpix.yaml) | | 2025-12-05 | [Ministral-3-14B-Reasoning-2512](https://huggingface.co/mistralai/Ministral-3-14B-Reasoning-2512) | | [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/mistral/ministral3_14b_medpix.yaml) | | 2025-11-19 | [InternVL3\_5-4B-hf](https://huggingface.co/OpenGVLab/InternVL3_5-4B-hf) | | [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/internvl/internvl_3_5_4b.yaml) | | 2025-11-17 | [Qwen3-VL-30B-A3B-Instruct](https://huggingface.co/Qwen/Qwen3-VL-30B-A3B-Instruct) | | [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/qwen3/qwen3_vl_moe_30b_te_deepep.yaml) | | 2025-10-23 | [Qwen3-VL-8B-Instruct](/model-coverage/vision-language-models/qwen/qwen3-vl-qwen3-vl-moe) | `Qwen3VLForConditionalGeneration` | [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/qwen3/qwen3_vl_8b_instruct_rdr.yaml) | | 2025-10-23 | [Qwen3-VL-4B-Thinking](https://huggingface.co/Qwen/Qwen3-VL-4B-Thinking) | | [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/qwen3/qwen3_vl_4b_instruct_rdr.yaml) | | 2025-07-15 | [Qwen2.5-VL-3B-Instruct](/model-coverage/vision-language-models/qwen/qwen2-5-vl) | `Qwen2_5VLForConditionalGeneration` | [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/qwen2_5/qwen2_5_vl_3b_rdr.yaml) | | 2025-07-15 | [Gemma-3n-e4b-it](https://huggingface.co/google/gemma-3n-e4b-it) | | [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/gemma3n/gemma3n_vl_4b_medpix.yaml) | | 2025-07-15 | [Gemma-3-4b-it](/model-coverage/vision-language-models/google/gemma-3-vl-gemma-3n) | `Gemma3ForCausalLM`, `Gemma3ForConditionalGeneration` | [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/gemma3/gemma3_vl_4b_cord_v2.yaml) | ## Fine-Tuning All supported models can be fine-tuned using either full SFT or PEFT (LoRA) approaches. See the [Gemma 3 Fine-Tuning Guide](/recipes-e2e-examples/gemma-3-3n) for a complete walkthrough covering dataset preparation, configuration, and multi-GPU training. In these guides, we use the `quintend/rdr-items` and `naver-clova-ix/cord-v2` datasets for demonstration purposes. Update the recipe YAML `dataset` section to use your own data. See [VLM datasets](/datasets/multi-modal-dataset) and [dataset overview](/datasets/overview).