bridge.perf_recipes.qwen_vl.common

Common helpers for qwen_vl performance recipes.

Module Contents

Functions

_use_model_vocab_null_tokenizer	Use a model-sized synthetic tokenizer for Qwen-VL performance runs.
_qwen35_vl_common	Apply VLM benchmark settings shared by Qwen3.5/Qwen3.6-VL.
_qwen35_vl_post	VLM post-overrides that must run after _benchmark_common.
_enable_partial_cuda_graphs	Re-enable partial (per-layer) CUDA graphs on the language stack, with attn.
_select_gdn_kernel_backend	Select the kernel for the GatedDeltaNet (GDN) layers when Megatron-Core exposes the choice.
_qwen35_vl_post_clear_scope	Apply Qwen3.5/Qwen3.6-VL post-overrides and clear graph scope.
_finalize_qwen3_vl	Apply Qwen3-VL perf defaults that must override generic benchmark defaults.
_finalize_qwen3_vl_with_moe_a2a_overlap	Apply Qwen3-VL perf defaults with MoE A2A overlap enabled.

API

bridge.perf_recipes.qwen_vl.common._use_model_vocab_null_tokenizer(

cfg: megatron.bridge.training.config.ConfigContainer,

) → None

Use a model-sized synthetic tokenizer for Qwen-VL performance runs.

bridge.perf_recipes.qwen_vl.common._qwen35_vl_common(

cfg: megatron.bridge.training.config.ConfigContainer,

) → None

Apply VLM benchmark settings shared by Qwen3.5/Qwen3.6-VL.

Must be called before _benchmark_common and after setting precision.

bridge.perf_recipes.qwen_vl.common._qwen35_vl_post(

cfg: megatron.bridge.training.config.ConfigContainer,

) → None

VLM post-overrides that must run after _benchmark_common.

Qwen3.5/Qwen3.6-VL disable RoPE fusion and CUDA graphs for variable-length VLM inputs; these override the perf defaults that _benchmark_common sets.

bridge.perf_recipes.qwen_vl.common._enable_partial_cuda_graphs(

cfg: megatron.bridge.training.config.ConfigContainer,

) → None

Re-enable partial (per-layer) CUDA graphs on the language stack, with attn.

MUST be called AFTER :func:_qwen35_vl_post, which sets cuda_graph_impl="none" and clears the module list for the variable-shape real-data path. These benchmarks run on MOCK data with a fixed seq_length (4096) and moe_router_force_load_balancing=True, so the variable-length concern that motivates disabling graphs does not apply here.

attn is the module that matters: on 397B/64x GB300 under forced load balancing, adding the attention graph moved the step from 390.5 to 580.2 TFLOP/s/GPU with GPU kernel time unchanged (+0.5%) and identical per-kernel instance counts – the entire gain is per-launch CPU overhead that graph replay removes.

set_cuda_graph_modules writes cuda_graph_modules and nulls cuda_graph_scope, so the two never end up set at once (MCore asserts).

bridge.perf_recipes.qwen_vl.common._select_gdn_kernel_backend(

cfg: megatron.bridge.training.config.ConfigContainer,

backend: Literal[transformer_engine, fla],

) → None

Select the kernel for the GatedDeltaNet (GDN) layers when Megatron-Core exposes the choice.

"transformer_engine" runs GDN through Transformer Engine’s GatedDeltaNetAttention on cuDNN frontend kernels; "fla" is Megatron-Core’s default FLA Triton kernel. Qwen3.5-VL runs 3 of every 4 language layers as GDN (30 of 40 at 35B-A3B, 45 of 60 at 397B-A17B), so with 16 microbatches per step the GB300 35B and 397B recipes run 480 and 720 GDN forward+backward passes per rank per step.

On GB300 at these recipes’ shapes (35B: MBS 4 x 4096 tokens x 32 heads x 128; 397B: MBS 1 x 4096 x 64 x 128), cuDNN frontend 1.29 takes 54% (35B) and 39% (397B) less time than FLA for the GDN layer forward+backward. End to end with MXFP8, one run per arm on launch configurations close to (not exactly) these recipes, throughput rises by 9.6% at 397B-A17B on 64 GPUs (EP 32, forced load balancing) and by 12.0% to 19.4% at 35B-A3B on 16 GPUs (EP 4, real routing, two images).

The cuDNN path needs a Megatron-Core with TransformerConfig.gdn_kernel_backend (NVIDIA/Megatron-LM#6645, re-landing as NVIDIA/Megatron-LM#7583); transformer-engine >= 2.19; nvidia-cudnn-frontend >= 1.29.0 (older GDN kernels can return NaN); and nvidia-cutlass-dsl >= 4.7.0. With an older CuTe DSL, cuDNN silently runs GDN on its cuTile engine, which takes 86% longer than FLA at the 35B shape. Transformer Engine 2.19 also raises under the FP8 autocast that the FP8-CS and MXFP8 recipes enable unless Megatron-Core turns it off around the GDN call, which #7583 does not; 2.20.2 and later ignore FP8 autocast in GDN. ConfigContainer.validate warns about missing or too-old packages, and about Transformer Engine 2.19 with FP8 or FP4, instead of this helper switching backends, so a recipe builds the same config in every environment. Compare against FLA with model.gdn_kernel_backend=fla.

Assigning an unknown field on the model config does not raise: on a Megatron-Core without the field it would create an unused attribute and leave the recipe looking enabled while running FLA, so the assignment is guarded, as in _enable_gdn_conv_fusion (recipes/qwen_vl/h100/qwen35_vl.py).

bridge.perf_recipes.qwen_vl.common._qwen35_vl_post_clear_scope(

cfg: megatron.bridge.training.config.ConfigContainer,

) → None

Apply Qwen3.5/Qwen3.6-VL post-overrides and clear graph scope.

bridge.perf_recipes.qwen_vl.common._finalize_qwen3_vl(

cfg: megatron.bridge.training.config.ConfigContainer,

) → None

Apply Qwen3-VL perf defaults that must override generic benchmark defaults.

bridge.perf_recipes.qwen_vl.common._finalize_qwen3_vl_with_moe_a2a_overlap(

cfg: megatron.bridge.training.config.ConfigContainer,

) → None

Apply Qwen3-VL perf defaults with MoE A2A overlap enabled.