> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/automodel/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/automodel/_mcp/server.

# Qwen3.5

> Supported Qwen3.5 dense and MoE checkpoints and NeMo AutoModel recipes.

Qwen3.5 is Alibaba Cloud's unified vision-language model series, including dense and MoE variants for image and multimodal understanding tasks.

|                  |                                                                         |
| ---------------- | ----------------------------------------------------------------------- |
| **Task**         | Image-Text-to-Text                                                      |
| **Architecture** | `Qwen3_5ForConditionalGeneration`, `Qwen3_5MoeForConditionalGeneration` |
| **Parameters**   | 4B – 397B total                                                         |
| **HF Org**       | [Qwen](https://huggingface.co/Qwen)                                     |

## Available Models

* **Qwen3.5-4B, 9B, and 27B**: dense models
* **Qwen3.5-35B-A3B, 122B-A10B, and 397B-A17B**: MoE models

## Architectures

* `Qwen3_5ForConditionalGeneration` — dense models
* `Qwen3_5MoeForConditionalGeneration` — MoE models

## Example Recipes

| Recipe                                                                                                                                                    | Dataset    | Description                                                                       |
| --------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------- | --------------------------------------------------------------------------------- |
| [qwen3\_5\_4b.yaml](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/qwen3_5/qwen3_5_4b.yaml)                                     | MedPix-VQA | SFT — Qwen3.5-VL 4B on MedPix                                                     |
| [qwen3\_5\_9b.yaml](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/qwen3_5/qwen3_5_9b.yaml)                                     | MedPix-VQA | SFT — Qwen3.5-VL 9B on MedPix                                                     |
| [qwen3\_5\_moe\_medpix.yaml](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/qwen3_5_moe/qwen3_5_moe_medpix.yaml)                | MedPix-VQA | SFT — Qwen3.5-MoE on MedPix                                                       |
| [qwen3\_5\_35b.yaml](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/qwen3_5_moe/qwen3_5_35b.yaml)                               | MedPix-VQA | SFT — Qwen3.5 35B on MedPix                                                       |
| [qwen3\_5\_122b\_128k\_ep8cp32.yaml](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/qwen3_5_moe/qwen3_5_122b_128k_ep8cp32.yaml) | MedPix-VQA | SFT — Qwen3.5-MoE 122B with EP8, CP32, 128K packing, and a trainable vision tower |

## Try with NeMo AutoModel

**1. Clone and install from source** ([full instructions](/get-started/installation)):

```bash
git clone https://github.com/NVIDIA-NeMo/Automodel.git
cd Automodel
uv sync --locked --all-groups --all-extras --extra vlm-media
```

**2. Run the recipe** from inside the repo:

```bash
uv run automodel --nproc-per-node=8 examples/vlm_finetune/qwen3_5/qwen3_5_4b.yaml
```

#### Run with Docker

**1. Pull the container** and mount a checkpoint directory:

```bash
docker run --gpus all -it --rm \
  --shm-size=8g \
  -v $(pwd)/checkpoints:/opt/Automodel/checkpoints \
  nvcr.io/nvidia/nemo-automodel:26.06.00
```

**2.** Navigate to the AutoModel directory (where the recipes are):

```bash
cd /opt/Automodel
```

**3. Install media dependencies** (not installed in the container by default):

```bash
uv pip install ".[vlm-media]"
```

**4. Run the recipe**:

```bash
automodel --nproc-per-node=8 examples/vlm_finetune/qwen3_5/qwen3_5_4b.yaml
```

See the [Installation Guide](/get-started/installation) and [VLM Fine-Tuning Guide](/recipes-e2e-examples/gemma-3-3n).

## Fine-Tuning

See the [VLM Fine-Tuning Guide](/recipes-e2e-examples/gemma-3-3n).

### Validated Large-Model, Long-Context Scale

The Qwen3.5-MoE VLM training path has been validated at both
**397-billion-parameter model scale** and **128K context length**.
Qwen3.5-397B-A17B completed a 10-step end-to-end training run on 256 H100 GPUs
with FSDP2, CP64, EP64, packed sequences, full activation checkpointing, and a
trainable vision tower. The workload mixed text, image, and video data and
included genuine examples of approximately 120K tokens.

The run maintained finite, decreasing loss, validating the training mechanics
for this combined large-model and long-context regime. It is not a model
convergence result. The public
[397B recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/qwen3_5_moe/qwen3_5_moe_medpix.yaml)
provides a starting configuration for that checkpoint; use the
[122B EP8/CP32 recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/qwen3_5_moe/qwen3_5_122b_128k_ep8cp32.yaml)
as the published 128K long-context reference.

Dense Qwen3.5 and Qwen3.5-MoE support context-parallel vision frame sharding. The
Qwen3.5-MoE path composes expert and context parallelism with packed sequences; see the
[Context-Parallel Vision Frame Sharding guide](/development/cp-vision-frame-sharding).

## Hugging Face Model Cards

* [Qwen](https://huggingface.co/Qwen)