bridge.recipes.qwen_vl.gb300.qwen35_vl

GB300 recipes for Qwen3.5/Qwen3.6-VL.

Module Contents

Functions

_apply_qwen35_vl_397b_a17b_64gpu_gb300_execution_config	Apply the 64-GB300 execution policy for Qwen3.5/Qwen3.6-VL 397B-A17B.
qwen35_vl_397b_a17b_pretrain_config	Return the full-pretraining config for Qwen3.5/Qwen3.6-VL 397B-A17B on 64 GB300 GPUs.
_apply_qwen35_vl_35b_a3b_16gpu_gb300_execution_config	Apply the 16-GB300 execution policy for Qwen3.5/Qwen3.6-VL 35B-A3B.
qwen35_vl_35b_a3b_pretrain_16gpu_gb300_bf16_config	Return the full-pretraining config for Qwen3.5/Qwen3.6-VL 35B-A3B on 16 GB300 GPUs.
qwen35_vl_35b_a3b_pretrain_8gpu_gb300_fp8mx_library_config	Return the MXFP8 full-pretraining config for Qwen3.5/Qwen3.6-VL 35B-A3B on 8 GB300 GPUs.

Data

__all__

API

bridge.recipes.qwen_vl.gb300.qwen35_vl._apply_qwen35_vl_397b_a17b_64gpu_gb300_execution_config(

cfg: megatron.bridge.training.config.ConfigContainer,

) → None

Apply the 64-GB300 execution policy for Qwen3.5/Qwen3.6-VL 397B-A17B.

Mirrors qwen35_vl_397b_a17b_pretrain_64gpu_gb300_bf16_config in perf_recipes/qwen_vl/gb300 so real-data runs share the benchmark’s execution policy. Only the parallel layout, dispatcher, graph and env settings are copied; dataset/tokenizer/freeze policy stay real-data.

bridge.recipes.qwen_vl.gb300.qwen35_vl.qwen35_vl_397b_a17b_pretrain_config() → megatron.bridge.training.config.ConfigContainer

Return the full-pretraining config for Qwen3.5/Qwen3.6-VL 397B-A17B on 64 GB300 GPUs.

Counterpart of :func:qwen35_vl_35b_a3b_pretrain_config for the 397B-A17B model: the recipe defaults to the mock VLM dataset, while maintained launchers may select a real dataset without rebuilding the model execution policy. The execution policy is the shipped 64-GB300 benchmark layout.

bridge.recipes.qwen_vl.gb300.qwen35_vl._apply_qwen35_vl_35b_a3b_16gpu_gb300_execution_config(

cfg: megatron.bridge.training.config.ConfigContainer,

) → None

Apply the 16-GB300 execution policy for Qwen3.5/Qwen3.6-VL 35B-A3B.

Counterpart of :func:_apply_qwen35_vl_397b_a17b_64gpu_gb300_execution_config. Only the parallel layout, dispatcher, graph and env settings are set here; dataset/tokenizer/freeze policy stay real-data.

bridge.recipes.qwen_vl.gb300.qwen35_vl.qwen35_vl_35b_a3b_pretrain_16gpu_gb300_bf16_config() → megatron.bridge.training.config.ConfigContainer

Return the full-pretraining config for Qwen3.5/Qwen3.6-VL 35B-A3B on 16 GB300 GPUs.

GB300 counterpart of the shared

Func:

megatron.bridge.recipes.qwen_vl.h100.qwen35_vl.qwen35_vl_35b_a3b_pretrain_config, which keeps its H100 execution policy. The recipe defaults to the mock VLM dataset, while maintained launchers may select a real dataset without rebuilding the model execution policy.

bridge.recipes.qwen_vl.gb300.qwen35_vl.qwen35_vl_35b_a3b_pretrain_8gpu_gb300_fp8mx_library_config() → megatron.bridge.training.config.ConfigContainer

Return the MXFP8 full-pretraining config for Qwen3.5/Qwen3.6-VL 35B-A3B on 8 GB300 GPUs.

Starts from :func:qwen35_vl_35b_a3b_pretrain_16gpu_gb300_bf16_config and keeps its parallel layout (EP 4, TP/PP/CP 1), CUDA graphs, recompute, batch shape (micro batch 4, global batch 2048) and real routing. On top it sets MXFP8, Transformer Engine’s CuTe DSL fused grouped MLP, cuDNN GDN, and the HybridEP, dispatcher and normalization settings of the configuration it was tuned on. With TP, PP and CP at 1, data parallelism is 8 on 8 GPUs (expert data parallelism 2), so each rank runs 64 micro batches per step.

Like the 16-GPU recipe, it defaults to the mock VLM dataset. The measured runs used an Energon dataset (--dataset energon dataset.path=...) with dataset.task_encoder.max_pixels=200704, and the vision CUDA graph cap inherited from the 16-GPU recipe (3712 patches per micro batch) was measured with that image budget at micro batch 4. Keep both, or re-measure the cap: a larger image budget can exceed it and stop training with an error.

cuDNN GDN is selected only on a Megatron-Core that declares gdn_kernel_backend (NVIDIA/Megatron-LM#7583); otherwise the GatedDeltaNet layers keep running on FLA.

Transformer Engine reads the fused-MLP gate NVTE_CUTEDSL_FUSED_GROUPED_MLP once, when it is imported. scripts/training/run_recipe.py, which train.sh runs, sets it in time by re-executing itself; with any other entry point, export it before Python starts, or Transformer Engine silently runs the unfused grouped GEMM.

The name carries a _library suffix because qwen35_vl_35b_a3b_pretrain_8gpu_gb300_fp8mx_config is already a perf recipe (a mock-data benchmark with EP 8 and global batch 512), and bare recipe lookup keeps resolving that name to it.

Measured on GB300 with real DataComp data and real routing (steps 101-150, one run per arm, submitted together), a launch configuration close to this recipe reached 11.3% more TFLOP/s per GPU on 8 GPUs than on 16 at the same global batch size. The two runs differ in data-parallel size and in micro batches per rank (64 vs 32), and one run per arm does not separate these effects, so no cause is claimed.

bridge.recipes.qwen_vl.gb300.qwen35_vl.__all__

[‘qwen35_vl_35b_a3b_pretrain_8gpu_gb300_fp8mx_library_config’, ‘qwen35_vl_35b_a3b_pretrain_16gpu_gb3…