bridge.recipes.qwen_vl.h100.qwen35_vl#

Qwen3.5-VL recipes.

This module provides pretrain, SFT, and PEFT configurations for Qwen3.5-VL models:

  • Dense: 800M, 2B, 4B, 9B, 27B

  • MoE: 35B-A3B, 122B-A10B, 397B-A17B

Module Contents#

Functions#

qwen35_vl_9b_pretrain_4gpu_h100_bf16_mock_config

Return a pre-training config for Qwen3.5-VL 9B (dense).

qwen35_vl_27b_pretrain_16gpu_h100_bf16_mock_config

Return a pre-training config for Qwen3.5-VL 27B (dense).

qwen35_vl_35b_a3b_pretrain_8gpu_h100_bf16_mock_config

Return a pre-training config for Qwen3.5-VL 35B-A3B (MoE).

qwen35_vl_122b_a10b_pretrain_128gpu_h100_bf16_mock_config

Return a pre-training config for Qwen3.5-VL 122B-A10B (MoE).

qwen35_vl_397b_a17b_pretrain_512gpu_h100_bf16_mock_config

Return a pre-training config for Qwen3.5-VL 397B-A17B (MoE).

qwen35_vl_800m_sft_1gpu_h100_bf16_config

Return a full SFT config for Qwen3.5-VL 800M (dense).

qwen35_vl_2b_sft_1gpu_h100_bf16_config

Return a full SFT config for Qwen3.5-VL 2B (dense).

qwen35_vl_4b_sft_2gpu_h100_bf16_config

Return a full SFT config for Qwen3.5-VL 4B (dense).

qwen35_vl_9b_sft_4gpu_h100_bf16_config

Return a full SFT config for Qwen3.5-VL 9B (dense).

qwen35_vl_27b_sft_16gpu_h100_bf16_config

Return a full SFT config for Qwen3.5-VL 27B (dense).

qwen35_vl_35b_a3b_sft_16gpu_h100_bf16_config

Return a full SFT config for Qwen3.5-VL 35B-A3B (MoE).

qwen35_vl_35b_a3b_sft_2gpu_h100_bf16_fsdp_config

Return a full SFT config for Qwen3.5-VL 35B-A3B (MoE) with Megatron FSDP.

qwen35_vl_122b_a10b_sft_48gpu_h100_bf16_config

Return a full SFT config for Qwen3.5-VL 122B-A10B (MoE).

qwen35_vl_397b_a17b_sft_128gpu_h100_bf16_config

Return a full SFT config for Qwen3.5-VL 397B-A17B (MoE).

qwen35_vl_800m_peft_1gpu_h100_bf16_config

Return a PEFT config for Qwen3.5-VL 800M (dense).

qwen35_vl_2b_peft_1gpu_h100_bf16_config

Return a PEFT config for Qwen3.5-VL 2B (dense).

qwen35_vl_4b_peft_1gpu_h100_bf16_config

Return a PEFT config for Qwen3.5-VL 4B (dense).

qwen35_vl_9b_peft_1gpu_h100_bf16_config

Return a PEFT config for Qwen3.5-VL 9B (dense).

qwen35_vl_27b_peft_2gpu_h100_bf16_config

Return a PEFT config for Qwen3.5-VL 27B (dense).

qwen35_vl_35b_a3b_peft_4gpu_h100_bf16_config

Return a PEFT config for Qwen3.5-VL 35B-A3B (MoE).

qwen35_vl_122b_a10b_peft_8gpu_h100_bf16_config

Return a PEFT config for Qwen3.5-VL 122B-A10B (MoE).

qwen35_vl_397b_a17b_peft_32gpu_h100_bf16_config

Return a PEFT config for Qwen3.5-VL 397B-A17B (MoE).

Data#

__all__

API#

bridge.recipes.qwen_vl.h100.qwen35_vl.qwen35_vl_9b_pretrain_4gpu_h100_bf16_mock_config() → megatron.bridge.training.config.ConfigContainer#

Return a pre-training config for Qwen3.5-VL 9B (dense).

bridge.recipes.qwen_vl.h100.qwen35_vl.qwen35_vl_27b_pretrain_16gpu_h100_bf16_mock_config() → megatron.bridge.training.config.ConfigContainer#

Return a pre-training config for Qwen3.5-VL 27B (dense).

bridge.recipes.qwen_vl.h100.qwen35_vl.qwen35_vl_35b_a3b_pretrain_8gpu_h100_bf16_mock_config() → megatron.bridge.training.config.ConfigContainer#

Return a pre-training config for Qwen3.5-VL 35B-A3B (MoE).

bridge.recipes.qwen_vl.h100.qwen35_vl.qwen35_vl_122b_a10b_pretrain_128gpu_h100_bf16_mock_config() → megatron.bridge.training.config.ConfigContainer#

Return a pre-training config for Qwen3.5-VL 122B-A10B (MoE).

bridge.recipes.qwen_vl.h100.qwen35_vl.qwen35_vl_397b_a17b_pretrain_512gpu_h100_bf16_mock_config() → megatron.bridge.training.config.ConfigContainer#

Return a pre-training config for Qwen3.5-VL 397B-A17B (MoE).

bridge.recipes.qwen_vl.h100.qwen35_vl.qwen35_vl_800m_sft_1gpu_h100_bf16_config() → megatron.bridge.training.config.ConfigContainer#

Return a full SFT config for Qwen3.5-VL 800M (dense).

Default configuration: 1 GPU

  • TP=1, PP=1

  • LR=5e-6 (full SFT)

  • Sequence length: 4096

Note: num_kv_heads=2, so max TP=2.

bridge.recipes.qwen_vl.h100.qwen35_vl.qwen35_vl_2b_sft_1gpu_h100_bf16_config() → megatron.bridge.training.config.ConfigContainer#

Return a full SFT config for Qwen3.5-VL 2B (dense).

Default configuration: 1 GPU

  • TP=1, PP=1

  • LR=5e-6 (full SFT)

  • Sequence length: 4096

Note: num_kv_heads=2, so max TP=2.

bridge.recipes.qwen_vl.h100.qwen35_vl.qwen35_vl_4b_sft_2gpu_h100_bf16_config() → megatron.bridge.training.config.ConfigContainer#

Return a full SFT config for Qwen3.5-VL 4B (dense).

Default configuration: 2 GPUs

  • TP=2, PP=1

  • LR=5e-6 (full SFT)

  • Sequence length: 4096

Note: num_kv_heads=4, so max TP=4.

bridge.recipes.qwen_vl.h100.qwen35_vl.qwen35_vl_9b_sft_4gpu_h100_bf16_config() → megatron.bridge.training.config.ConfigContainer#

Return a full SFT config for Qwen3.5-VL 9B (dense).

Default configuration: 4 GPUs

  • TP=4, PP=1

  • LR=5e-6 (full SFT)

  • Sequence length: 4096

Note: num_kv_heads=4, so max TP=4.

bridge.recipes.qwen_vl.h100.qwen35_vl.qwen35_vl_27b_sft_16gpu_h100_bf16_config() → megatron.bridge.training.config.ConfigContainer#

Return a full SFT config for Qwen3.5-VL 27B (dense).

Default configuration: 16 GPUs

  • TP=4, PP=4

  • LR=5e-6 (full SFT)

  • Sequence length: 4096

bridge.recipes.qwen_vl.h100.qwen35_vl.qwen35_vl_35b_a3b_sft_16gpu_h100_bf16_config() → megatron.bridge.training.config.ConfigContainer#

Return a full SFT config for Qwen3.5-VL 35B-A3B (MoE).

Default configuration: 16 GPUs

  • TP=2, PP=1, EP=16

  • LR=2e-5 (full SFT)

  • Sequence length: 4096

bridge.recipes.qwen_vl.h100.qwen35_vl.qwen35_vl_35b_a3b_sft_2gpu_h100_bf16_fsdp_config() → megatron.bridge.training.config.ConfigContainer#

Return a full SFT config for Qwen3.5-VL 35B-A3B (MoE) with Megatron FSDP.

Uses Megatron FSDP for memory-efficient training with AG/RS overlap. Requires fsdp_dtensor checkpoint format (convert offline with checkpoint_inspector.py convert-torch-dist-to-fsdp-dtensor).

Default configuration: 2 GPUs

  • TP=1, PP=1, EP=2

  • Megatron FSDP with double buffering

  • NCCL UB disabled (heterogeneous FSDP units cause hangs)

  • LR=2e-5 (full SFT)

  • Sequence length: 4096

bridge.recipes.qwen_vl.h100.qwen35_vl.qwen35_vl_122b_a10b_sft_48gpu_h100_bf16_config() → megatron.bridge.training.config.ConfigContainer#

Return a full SFT config for Qwen3.5-VL 122B-A10B (MoE).

Default configuration: 48 GPUs

  • TP=2, PP=6, EP=8

  • LR=2e-5 (full SFT)

  • Sequence length: 4096

bridge.recipes.qwen_vl.h100.qwen35_vl.qwen35_vl_397b_a17b_sft_128gpu_h100_bf16_config() → megatron.bridge.training.config.ConfigContainer#

Return a full SFT config for Qwen3.5-VL 397B-A17B (MoE).

Default configuration: 128 GPUs

  • TP=2, PP=4, EP=32

  • LR=2e-5 (full SFT)

  • Sequence length: 4096

bridge.recipes.qwen_vl.h100.qwen35_vl.qwen35_vl_800m_peft_1gpu_h100_bf16_config() → megatron.bridge.training.config.ConfigContainer#

Return a PEFT config for Qwen3.5-VL 800M (dense).

Default configuration: 1 GPU

  • TP=1, PP=1

  • LR=1e-4 (PEFT)

  • Sequence length: 4096

bridge.recipes.qwen_vl.h100.qwen35_vl.qwen35_vl_2b_peft_1gpu_h100_bf16_config() → megatron.bridge.training.config.ConfigContainer#

Return a PEFT config for Qwen3.5-VL 2B (dense).

Default configuration: 1 GPU

  • TP=1, PP=1

  • LR=1e-4 (PEFT)

  • Sequence length: 4096

bridge.recipes.qwen_vl.h100.qwen35_vl.qwen35_vl_4b_peft_1gpu_h100_bf16_config() → megatron.bridge.training.config.ConfigContainer#

Return a PEFT config for Qwen3.5-VL 4B (dense).

Default configuration: 1 GPU

  • TP=1, PP=1

  • LR=1e-4 (PEFT)

  • Sequence length: 4096

bridge.recipes.qwen_vl.h100.qwen35_vl.qwen35_vl_9b_peft_1gpu_h100_bf16_config() → megatron.bridge.training.config.ConfigContainer#

Return a PEFT config for Qwen3.5-VL 9B (dense).

Default configuration: 1 GPU

  • TP=1, PP=1

  • LR=1e-4 (PEFT)

  • Sequence length: 4096

bridge.recipes.qwen_vl.h100.qwen35_vl.qwen35_vl_27b_peft_2gpu_h100_bf16_config() → megatron.bridge.training.config.ConfigContainer#

Return a PEFT config for Qwen3.5-VL 27B (dense).

Default configuration: 2 GPUs

  • TP=2, PP=1

  • LR=1e-4 (PEFT)

  • Sequence length: 4096

bridge.recipes.qwen_vl.h100.qwen35_vl.qwen35_vl_35b_a3b_peft_4gpu_h100_bf16_config() → megatron.bridge.training.config.ConfigContainer#

Return a PEFT config for Qwen3.5-VL 35B-A3B (MoE).

Default configuration: 4 GPUs

  • TP=2, PP=1, EP=4

  • LR=2e-4 (PEFT)

  • Sequence length: 4096

bridge.recipes.qwen_vl.h100.qwen35_vl.qwen35_vl_122b_a10b_peft_8gpu_h100_bf16_config() → megatron.bridge.training.config.ConfigContainer#

Return a PEFT config for Qwen3.5-VL 122B-A10B (MoE).

Default configuration: 8 GPUs

  • TP=2, PP=1, EP=8

  • LR=2e-4 (PEFT)

  • Sequence length: 4096

bridge.recipes.qwen_vl.h100.qwen35_vl.qwen35_vl_397b_a17b_peft_32gpu_h100_bf16_config() → megatron.bridge.training.config.ConfigContainer#

Return a PEFT config for Qwen3.5-VL 397B-A17B (MoE).

Default configuration: 32 GPUs

  • TP=2, PP=1, EP=32

  • LR=2e-4 (PEFT)

  • Sequence length: 4096

bridge.recipes.qwen_vl.h100.qwen35_vl.__all__#

[‘qwen35_vl_122b_a10b_peft_8gpu_h100_bf16_config’, ‘qwen35_vl_122b_a10b_pretrain_128gpu_h100_bf16_mo…