bridge.recipes.qwen_vl.h100.qwen35_vl#

Qwen3.5-VL recipes with shared Qwen3.6-VL 35B-A3B support.

This module provides pretrain, SFT, and PEFT configurations for Qwen3.5-VL models:

  • Dense: 800M, 2B, 4B, 9B, 27B

  • MoE: 35B-A3B, 122B-A10B, 397B-A17B

Qwen3.6 35B-A3B uses the same architecture and shares the 35B recipe.

Module Contents#

Functions#

_apply_qwen35_vl_35b_a3b_16gpu_h100_execution_config

Apply the measured 16-H100 execution policy for Qwen3.5/Qwen3.6-VL 35B-A3B.

qwen35_vl_9b_pretrain_4gpu_h100_bf16_mock_config

Return a pre-training config for Qwen3.5-VL 9B (dense).

qwen35_vl_27b_pretrain_16gpu_h100_bf16_mock_config

Return a pre-training config for Qwen3.5-VL 27B (dense).

qwen35_vl_35b_a3b_pretrain_8gpu_h100_bf16_mock_config

Return the shared 8-GPU pre-training config for Qwen3.5/Qwen3.6-VL 35B-A3B.

qwen35_vl_35b_a3b_pretrain_16gpu_h100_bf16_functional_config

Return the full-pretraining config for Qwen3.5/Qwen3.6-VL 35B-A3B on 16 H100 GPUs.

qwen35_vl_122b_a10b_pretrain_128gpu_h100_bf16_mock_config

Return a pre-training config for Qwen3.5-VL 122B-A10B (MoE).

qwen35_vl_397b_a17b_pretrain_512gpu_h100_bf16_mock_config

Return a pre-training config for Qwen3.5-VL 397B-A17B (MoE).

qwen35_vl_800m_sft_1gpu_h100_bf16_config

Return a full SFT config for Qwen3.5-VL 800M (dense).

qwen35_vl_2b_sft_1gpu_h100_bf16_config

Return a full SFT config for Qwen3.5-VL 2B (dense).

qwen35_vl_4b_sft_2gpu_h100_bf16_config

Return a full SFT config for Qwen3.5-VL 4B (dense).

qwen35_vl_9b_sft_4gpu_h100_bf16_config

Return a full SFT config for Qwen3.5-VL 9B (dense).

qwen35_vl_27b_sft_16gpu_h100_bf16_config

Return a full SFT config for Qwen3.5-VL 27B (dense).

_qwen35_vl_35b_a3b_sft_base_config

Build the convergence-preserving SFT base for Qwen3.5/Qwen3.6-VL 35B-A3B.

qwen35_vl_35b_a3b_sft_16gpu_h100_bf16_config

Return the tuned full-SFT config for Qwen3.5/Qwen3.6-VL 35B-A3B on 16 H100 GPUs.

qwen35_vl_35b_a3b_sft_long_context_32gpu_h100_bf16_config

Return the shared long-context SFT config for Qwen3.5/Qwen3.6-VL 35B-A3B.

qwen35_vl_35b_a3b_sft_2gpu_h100_bf16_fsdp_config

Return a full SFT config for Qwen3.5/Qwen3.6-VL 35B-A3B (MoE) with Megatron FSDP.

qwen35_vl_122b_a10b_sft_48gpu_h100_bf16_config

Return a full SFT config for Qwen3.5-VL 122B-A10B (MoE).

qwen35_vl_397b_a17b_sft_128gpu_h100_bf16_config

Return a full SFT config for Qwen3.5-VL 397B-A17B (MoE).

qwen35_vl_800m_peft_1gpu_h100_bf16_config

Return a PEFT config for Qwen3.5-VL 800M (dense).

qwen35_vl_2b_peft_1gpu_h100_bf16_config

Return a PEFT config for Qwen3.5-VL 2B (dense).

qwen35_vl_4b_peft_1gpu_h100_bf16_config

Return a PEFT config for Qwen3.5-VL 4B (dense).

qwen35_vl_9b_peft_1gpu_h100_bf16_config

Return a PEFT config for Qwen3.5-VL 9B (dense).

qwen35_vl_27b_peft_2gpu_h100_bf16_config

Return a PEFT config for Qwen3.5-VL 27B (dense).

qwen35_vl_35b_a3b_peft_4gpu_h100_bf16_config

Return a PEFT config for Qwen3.5/Qwen3.6-VL 35B-A3B (MoE).

qwen35_vl_35b_a3b_peft_16gpu_h100_bf16_config

Return the tuned LoRA config for Qwen3.5/Qwen3.6-VL 35B-A3B on 16 H100 GPUs.

qwen35_vl_122b_a10b_peft_8gpu_h100_bf16_config

Return a PEFT config for Qwen3.5-VL 122B-A10B (MoE).

qwen35_vl_397b_a17b_peft_32gpu_h100_bf16_config

Return a PEFT config for Qwen3.5-VL 397B-A17B (MoE).

Data#

__all__

API#

bridge.recipes.qwen_vl.h100.qwen35_vl._apply_qwen35_vl_35b_a3b_16gpu_h100_execution_config(
cfg: megatron.bridge.training.config.ConfigContainer,
) → None#

Apply the measured 16-H100 execution policy for Qwen3.5/Qwen3.6-VL 35B-A3B.

bridge.recipes.qwen_vl.h100.qwen35_vl.qwen35_vl_9b_pretrain_4gpu_h100_bf16_mock_config() → megatron.bridge.training.config.ConfigContainer#

Return a pre-training config for Qwen3.5-VL 9B (dense).

bridge.recipes.qwen_vl.h100.qwen35_vl.qwen35_vl_27b_pretrain_16gpu_h100_bf16_mock_config() → megatron.bridge.training.config.ConfigContainer#

Return a pre-training config for Qwen3.5-VL 27B (dense).

bridge.recipes.qwen_vl.h100.qwen35_vl.qwen35_vl_35b_a3b_pretrain_8gpu_h100_bf16_mock_config() → megatron.bridge.training.config.ConfigContainer#

Return the shared 8-GPU pre-training config for Qwen3.5/Qwen3.6-VL 35B-A3B.

The recipe freezes the language and vision towers and trains the vision projection. Dataset-specific batch and execution topology overrides belong in the launcher command so this shared recipe preserves its convergence defaults.

bridge.recipes.qwen_vl.h100.qwen35_vl.qwen35_vl_35b_a3b_pretrain_16gpu_h100_bf16_functional_config() → megatron.bridge.training.config.ConfigContainer#

Return the full-pretraining config for Qwen3.5/Qwen3.6-VL 35B-A3B on 16 H100 GPUs.

The recipe defaults to the mock VLM dataset, while maintained launchers may select a real dataset without rebuilding the model execution policy.

bridge.recipes.qwen_vl.h100.qwen35_vl.qwen35_vl_122b_a10b_pretrain_128gpu_h100_bf16_mock_config() → megatron.bridge.training.config.ConfigContainer#

Return a pre-training config for Qwen3.5-VL 122B-A10B (MoE).

bridge.recipes.qwen_vl.h100.qwen35_vl.qwen35_vl_397b_a17b_pretrain_512gpu_h100_bf16_mock_config() → megatron.bridge.training.config.ConfigContainer#

Return a pre-training config for Qwen3.5-VL 397B-A17B (MoE).

bridge.recipes.qwen_vl.h100.qwen35_vl.qwen35_vl_800m_sft_1gpu_h100_bf16_config() → megatron.bridge.training.config.ConfigContainer#

Return a full SFT config for Qwen3.5-VL 800M (dense).

Default configuration: 1 GPU

  • TP=1, PP=1

  • LR=5e-6 (full SFT)

  • Sequence length: 4096

Note: num_kv_heads=2, so max TP=2.

bridge.recipes.qwen_vl.h100.qwen35_vl.qwen35_vl_2b_sft_1gpu_h100_bf16_config() → megatron.bridge.training.config.ConfigContainer#

Return a full SFT config for Qwen3.5-VL 2B (dense).

Default configuration: 1 GPU

  • TP=1, PP=1

  • LR=5e-6 (full SFT)

  • Sequence length: 4096

Note: num_kv_heads=2, so max TP=2.

bridge.recipes.qwen_vl.h100.qwen35_vl.qwen35_vl_4b_sft_2gpu_h100_bf16_config() → megatron.bridge.training.config.ConfigContainer#

Return a full SFT config for Qwen3.5-VL 4B (dense).

Default configuration: 2 GPUs

  • TP=2, PP=1

  • LR=5e-6 (full SFT)

  • Sequence length: 4096

Note: num_kv_heads=4, so max TP=4.

bridge.recipes.qwen_vl.h100.qwen35_vl.qwen35_vl_9b_sft_4gpu_h100_bf16_config() → megatron.bridge.training.config.ConfigContainer#

Return a full SFT config for Qwen3.5-VL 9B (dense).

Default configuration: 4 GPUs

  • TP=4, PP=1

  • LR=5e-6 (full SFT)

  • Sequence length: 4096

Note: num_kv_heads=4, so max TP=4.

bridge.recipes.qwen_vl.h100.qwen35_vl.qwen35_vl_27b_sft_16gpu_h100_bf16_config() → megatron.bridge.training.config.ConfigContainer#

Return a full SFT config for Qwen3.5-VL 27B (dense).

Default configuration: 16 GPUs

  • TP=4, PP=4

  • LR=5e-6 (full SFT)

  • Sequence length: 4096

bridge.recipes.qwen_vl.h100.qwen35_vl._qwen35_vl_35b_a3b_sft_base_config() → megatron.bridge.training.config.ConfigContainer#

Build the convergence-preserving SFT base for Qwen3.5/Qwen3.6-VL 35B-A3B.

Default configuration: 16 GPUs

  • TP=1, PP=2, EP=8

  • MBS=1, GBS=32

  • LR=2e-5 (full SFT)

  • Sequence length: 4096

bridge.recipes.qwen_vl.h100.qwen35_vl.qwen35_vl_35b_a3b_sft_16gpu_h100_bf16_config() → megatron.bridge.training.config.ConfigContainer#

Return the tuned full-SFT config for Qwen3.5/Qwen3.6-VL 35B-A3B on 16 H100 GPUs.

bridge.recipes.qwen_vl.h100.qwen35_vl.qwen35_vl_35b_a3b_sft_long_context_32gpu_h100_bf16_config() → megatron.bridge.training.config.ConfigContainer#

Return the shared long-context SFT config for Qwen3.5/Qwen3.6-VL 35B-A3B.

Default configuration: 32 GPUs

  • TP=1, PP=4, CP=2, EP=8

  • MBS=2, GBS=512 with deferred in-batch packing

  • Sequence length: 8192

bridge.recipes.qwen_vl.h100.qwen35_vl.qwen35_vl_35b_a3b_sft_2gpu_h100_bf16_fsdp_config() → megatron.bridge.training.config.ConfigContainer#

Return a full SFT config for Qwen3.5/Qwen3.6-VL 35B-A3B (MoE) with Megatron FSDP.

Uses Megatron FSDP for memory-efficient training with AG/RS overlap. Requires fsdp_dtensor checkpoint format (convert offline with checkpoint_inspector.py convert-torch-dist-to-fsdp-dtensor).

Default configuration: 2 GPUs

  • TP=1, PP=1, EP=2

  • Megatron FSDP with double buffering

  • NCCL UB disabled (heterogeneous FSDP units cause hangs)

  • LR=2e-5 (full SFT)

  • Sequence length: 4096

bridge.recipes.qwen_vl.h100.qwen35_vl.qwen35_vl_122b_a10b_sft_48gpu_h100_bf16_config() → megatron.bridge.training.config.ConfigContainer#

Return a full SFT config for Qwen3.5-VL 122B-A10B (MoE).

Default configuration: 48 GPUs

  • TP=2, PP=6, EP=8

  • LR=2e-5 (full SFT)

  • Sequence length: 4096

bridge.recipes.qwen_vl.h100.qwen35_vl.qwen35_vl_397b_a17b_sft_128gpu_h100_bf16_config() → megatron.bridge.training.config.ConfigContainer#

Return a full SFT config for Qwen3.5-VL 397B-A17B (MoE).

Default configuration: 128 GPUs

  • TP=2, PP=4, EP=32

  • LR=2e-5 (full SFT)

  • Sequence length: 4096

bridge.recipes.qwen_vl.h100.qwen35_vl.qwen35_vl_800m_peft_1gpu_h100_bf16_config() → megatron.bridge.training.config.ConfigContainer#

Return a PEFT config for Qwen3.5-VL 800M (dense).

Default configuration: 1 GPU

  • TP=1, PP=1

  • LR=1e-4 (PEFT)

  • Sequence length: 4096

bridge.recipes.qwen_vl.h100.qwen35_vl.qwen35_vl_2b_peft_1gpu_h100_bf16_config() → megatron.bridge.training.config.ConfigContainer#

Return a PEFT config for Qwen3.5-VL 2B (dense).

Default configuration: 1 GPU

  • TP=1, PP=1

  • LR=1e-4 (PEFT)

  • Sequence length: 4096

bridge.recipes.qwen_vl.h100.qwen35_vl.qwen35_vl_4b_peft_1gpu_h100_bf16_config() → megatron.bridge.training.config.ConfigContainer#

Return a PEFT config for Qwen3.5-VL 4B (dense).

Default configuration: 1 GPU

  • TP=1, PP=1

  • LR=1e-4 (PEFT)

  • Sequence length: 4096

bridge.recipes.qwen_vl.h100.qwen35_vl.qwen35_vl_9b_peft_1gpu_h100_bf16_config() → megatron.bridge.training.config.ConfigContainer#

Return a PEFT config for Qwen3.5-VL 9B (dense).

Default configuration: 1 GPU

  • TP=1, PP=1

  • LR=1e-4 (PEFT)

  • Sequence length: 4096

bridge.recipes.qwen_vl.h100.qwen35_vl.qwen35_vl_27b_peft_2gpu_h100_bf16_config() → megatron.bridge.training.config.ConfigContainer#

Return a PEFT config for Qwen3.5-VL 27B (dense).

Default configuration: 2 GPUs

  • TP=2, PP=1

  • LR=1e-4 (PEFT)

  • Sequence length: 4096

bridge.recipes.qwen_vl.h100.qwen35_vl.qwen35_vl_35b_a3b_peft_4gpu_h100_bf16_config() → megatron.bridge.training.config.ConfigContainer#

Return a PEFT config for Qwen3.5/Qwen3.6-VL 35B-A3B (MoE).

Default configuration: 4 GPUs

  • TP=2, PP=1, EP=4

  • LR=2e-4 (PEFT)

  • Sequence length: 4096

bridge.recipes.qwen_vl.h100.qwen35_vl.qwen35_vl_35b_a3b_peft_16gpu_h100_bf16_config() → megatron.bridge.training.config.ConfigContainer#

Return the tuned LoRA config for Qwen3.5/Qwen3.6-VL 35B-A3B on 16 H100 GPUs.

bridge.recipes.qwen_vl.h100.qwen35_vl.qwen35_vl_122b_a10b_peft_8gpu_h100_bf16_config() → megatron.bridge.training.config.ConfigContainer#

Return a PEFT config for Qwen3.5-VL 122B-A10B (MoE).

Default configuration: 8 GPUs

  • TP=2, PP=1, EP=8

  • LR=2e-4 (PEFT)

  • Sequence length: 4096

bridge.recipes.qwen_vl.h100.qwen35_vl.qwen35_vl_397b_a17b_peft_32gpu_h100_bf16_config() → megatron.bridge.training.config.ConfigContainer#

Return a PEFT config for Qwen3.5-VL 397B-A17B (MoE).

Default configuration: 32 GPUs

  • TP=2, PP=1, EP=32

  • LR=2e-4 (PEFT)

  • Sequence length: 4096

bridge.recipes.qwen_vl.h100.qwen35_vl.__all__#

[‘qwen35_vl_122b_a10b_peft_8gpu_h100_bf16_config’, ‘qwen35_vl_122b_a10b_pretrain_128gpu_h100_bf16_mo…