Model CoverageVision Language Models

Qwen3.5

View as Markdown

Qwen3.5 is Alibaba Cloud’s unified vision-language model series, including dense and MoE variants for image and multimodal understanding tasks.

TaskImage-Text-to-Text
ArchitectureQwen3_5ForConditionalGeneration, Qwen3_5MoeForConditionalGeneration
Parameters4B – 397B total
HF OrgQwen

Available Models

  • Qwen3.5-4B, 9B, and 27B: dense models
  • Qwen3.5-35B-A3B, 122B-A10B, and 397B-A17B: MoE models

Architectures

  • Qwen3_5ForConditionalGeneration — dense models
  • Qwen3_5MoeForConditionalGeneration — MoE models

Example Recipes

RecipeDatasetDescription
qwen3_5_4b.yamlMedPix-VQASFT — Qwen3.5-VL 4B on MedPix
qwen3_5_9b.yamlMedPix-VQASFT — Qwen3.5-VL 9B on MedPix
qwen3_5_moe_medpix.yamlMedPix-VQASFT — Qwen3.5-MoE on MedPix
qwen3_5_35b.yamlMedPix-VQASFT — Qwen3.5 35B on MedPix
qwen3_5_122b_128k_ep8cp32.yamlMedPix-VQASFT — Qwen3.5-MoE 122B with EP8, CP32, 128K packing, and a trainable vision tower

Try with NeMo AutoModel

1. Clone and install from source (full instructions):

$git clone https://github.com/NVIDIA-NeMo/Automodel.git
$cd Automodel
$uv sync --locked --all-groups --all-extras --extra vlm-media

2. Run the recipe from inside the repo:

$uv run automodel --nproc-per-node=8 examples/vlm_finetune/qwen3_5/qwen3_5_4b.yaml

1. Pull the container and mount a checkpoint directory:

$docker run --gpus all -it --rm \
>  --shm-size=8g \
>  -v $(pwd)/checkpoints:/opt/Automodel/checkpoints \
>  nvcr.io/nvidia/nemo-automodel:26.06.00

2. Navigate to the AutoModel directory (where the recipes are):

$cd /opt/Automodel

3. Install media dependencies (not installed in the container by default):

$uv pip install ".[vlm-media]"

4. Run the recipe:

$automodel --nproc-per-node=8 examples/vlm_finetune/qwen3_5/qwen3_5_4b.yaml

See the Installation Guide and VLM Fine-Tuning Guide.

Fine-Tuning

See the VLM Fine-Tuning Guide.

Validated Large-Model, Long-Context Scale

The Qwen3.5-MoE VLM training path has been validated at both 397-billion-parameter model scale and 128K context length. Qwen3.5-397B-A17B completed a 10-step end-to-end training run on 256 H100 GPUs with FSDP2, CP64, EP64, packed sequences, full activation checkpointing, and a trainable vision tower. The workload mixed text, image, and video data and included genuine examples of approximately 120K tokens.

The run maintained finite, decreasing loss, validating the training mechanics for this combined large-model and long-context regime. It is not a model convergence result. The public 397B recipe provides a starting configuration for that checkpoint; use the 122B EP8/CP32 recipe as the published 128K long-context reference.

Dense Qwen3.5 and Qwen3.5-MoE support context-parallel vision frame sharding. The Qwen3.5-MoE path composes expert and context parallelism with packed sequences; see the Context-Parallel Vision Frame Sharding guide.

Hugging Face Model Cards