bridge.recipes.qwen.h100.qwen3_moe

Module Contents

Functions

qwen3_30b_a3b_pretrain_8gpu_h100_bf16_config	Return a pre-training config for Qwen3-30B-A3B MoE.
qwen3_30b_a3b_pretrain_16gpu_h100_bf16_config	Return a pre-training config for Qwen3-30B-A3B on 16 H100 GPUs.
qwen3_235b_a22b_pretrain_256gpu_h100_bf16_config	Return a pre-training config for Qwen3-235B-A22B MoE.
qwen3_30b_a3b_sft_8gpu_h100_bf16_config	Return a full SFT config for Qwen3-30B-A3B MoE.
qwen3_30b_a3b_sft_16gpu_h100_bf16_config	Return a full SFT config for Qwen3-30B-A3B MoE on 16 H100 GPUs.
qwen3_235b_a22b_sft_64gpu_h100_bf16_config	Return a full SFT config for Qwen3-235B-A22B MoE.
qwen3_30b_a3b_peft_4gpu_h100_bf16_config	Return a PEFT config for Qwen3-30B-A3B MoE.
qwen3_235b_a22b_peft_16gpu_h100_bf16_config	Return a PEFT config for Qwen3-235B-A22B MoE.

Data

_QWEN3_30B_A3B_MODEL_ID
_QWEN3_30B_A3B_MODEL_REVISION
__all__

API

bridge.recipes.qwen.h100.qwen3_moe._QWEN3_30B_A3B_MODEL_ID

‘Qwen/Qwen3-30B-A3B’

bridge.recipes.qwen.h100.qwen3_moe._QWEN3_30B_A3B_MODEL_REVISION

‘ad44e777bcd18fa416d9da3bd8f70d33ebb85d39’

bridge.recipes.qwen.h100.qwen3_moe.qwen3_30b_a3b_pretrain_8gpu_h100_bf16_config() → megatron.bridge.training.config.ConfigContainer

Return a pre-training config for Qwen3-30B-A3B MoE.

Recommended parallelism: TP=4, PP=2, EP=4.

bridge.recipes.qwen.h100.qwen3_moe.qwen3_30b_a3b_pretrain_16gpu_h100_bf16_config() → megatron.bridge.training.config.ConfigContainer

Return a pre-training config for Qwen3-30B-A3B on 16 H100 GPUs.

Recommended parallelism: TP=1, PP=1, EP=16 with HybridEP.

bridge.recipes.qwen.h100.qwen3_moe.qwen3_235b_a22b_pretrain_256gpu_h100_bf16_config() → megatron.bridge.training.config.ConfigContainer

Return a pre-training config for Qwen3-235B-A22B MoE.

Recommended parallelism: TP=4, PP=16, CP=2, EP=8. Note: Uses account_for_embedding_in_pipeline_split and account_for_loss_in_pipeline_split for proper layer distribution in pipeline parallelism.

bridge.recipes.qwen.h100.qwen3_moe.qwen3_30b_a3b_sft_8gpu_h100_bf16_config() → megatron.bridge.training.config.ConfigContainer

Return a full SFT config for Qwen3-30B-A3B MoE.

Recommended parallelism: TP=4, PP=2, EP=4 (1 node, 8 GPUs with SP=True)

bridge.recipes.qwen.h100.qwen3_moe.qwen3_30b_a3b_sft_16gpu_h100_bf16_config() → megatron.bridge.training.config.ConfigContainer

Return a full SFT config for Qwen3-30B-A3B MoE on 16 H100 GPUs.

Recommended parallelism: TP=1, PP=1, EP=16 (2 nodes, 16 GPUs). This topology keeps the global batch size at 32 while using DP=16 and two gradient-accumulation steps for the default micro batch size of 1.

bridge.recipes.qwen.h100.qwen3_moe.qwen3_235b_a22b_sft_64gpu_h100_bf16_config() → megatron.bridge.training.config.ConfigContainer

Return a full SFT config for Qwen3-235B-A22B MoE.

Recommended parallelism: TP=4, PP=16, EP=4 (8 nodes, 64 GPUs with SP=True) Uses account_for_embedding_in_pipeline_split and account_for_loss_in_pipeline_split.

bridge.recipes.qwen.h100.qwen3_moe.qwen3_30b_a3b_peft_4gpu_h100_bf16_config(

peft_scheme: str | megatron.bridge.peft.base.PEFT = 'lora',

) → megatron.bridge.training.config.ConfigContainer

Return a PEFT config for Qwen3-30B-A3B MoE.

Parameters:

peft_scheme – PEFT scheme - ‘lora’, ‘dora’, or a PEFT instance. Default: ‘lora’

Recommended parallelism: TP=4, PP=1, EP=4 (1 node, 4 GPUs with SP=True) LoRA/DoRA uses dim=8, alpha=16, target_modules=[‘linear_qkv’, ‘linear_proj’]

bridge.recipes.qwen.h100.qwen3_moe.qwen3_235b_a22b_peft_16gpu_h100_bf16_config(

peft_scheme: str | megatron.bridge.peft.base.PEFT = 'lora',

) → megatron.bridge.training.config.ConfigContainer

Return a PEFT config for Qwen3-235B-A22B MoE.

Parameters:

peft_scheme – PEFT scheme - ‘lora’, ‘dora’, or a PEFT instance. Default: ‘lora’

Recommended parallelism: TP=4, PP=4, EP=4 (8 nodes, 64 GPUs with SP=True) LoRA/DoRA uses dim=8, alpha=16, target_modules=[‘linear_qkv’, ‘linear_proj’] Uses account_for_embedding_in_pipeline_split and account_for_loss_in_pipeline_split.

bridge.recipes.qwen.h100.qwen3_moe.__all__

[‘qwen3_235b_a22b_peft_16gpu_h100_bf16_config’, ‘qwen3_235b_a22b_pretrain_256gpu_h100_bf16_config’, …