bridge.recipes.beam.h200.beam

Beam recipes for H200 nodes (8 GPUs, 141 GB each).

Beam is a 501B-parameter MoE (52 layers, 256 experts, top-8) built on HybridModel. About 98% of its parameters are experts, so every recipe shards them with expert parallelism; cp_comm_type="a2a" is set because Beam’s sliding-window and NoPE layers need all-to-all context parallelism whenever CP > 1.

The SFT and PEFT recipes own the qwen3_30b_a3b_convergence_v2 contract (8K packed Tulu 3, GBS 8, FP32 optimizer state). The pretrain recipe keeps Beam’s own bounded settings.

Module Contents

Functions

_beam_model_config	Build the Beam model config from the pinned HF checkpoint, or from hf_path when given.
_set_beam_tokenizer	Point the tokenizer at the same checkpoint as the model.
_set_beam_parallelism	Apply the parallel layout and the Beam execution settings shared by every recipe.
_set_convergence_v2_optimizer	Apply the qwen3_30b_a3b_convergence_v2 optimizer, schedule and precision fingerprint.
_set_convergence_v2_dataset	Use the pinned 8K offline-packed Tulu 3 selection of qwen3_30b_a3b_convergence_v2.
beam_pretrain_128gpu_h200_bf16_config	Return a pre-training config for Beam on 16 H200 nodes.
beam_sft_128gpu_h200_bf16_config	Return a full SFT config for Beam on 16 H200 nodes under qwen3_30b_a3b_convergence_v2.
beam_peft_16gpu_h200_bf16_config	Return a PEFT config for Beam on 2 H200 nodes under qwen3_30b_a3b_convergence_v2.

Data

BEAM_HF_MODEL_ID
BEAM_HF_MODEL_REVISION
TULU3_SPLIT
TULU3_REVISION
CONVERGENCE_V2_SEQ_LENGTH
CONVERGENCE_V2_GLOBAL_BATCH_SIZE

API

bridge.recipes.beam.h200.beam.BEAM_HF_MODEL_ID

‘reflection/Beam-1.0-501B-A23B’

bridge.recipes.beam.h200.beam.BEAM_HF_MODEL_REVISION

‘3ea13c81833d9a523ed4bdc4152a33edd16bbd30’

bridge.recipes.beam.h200.beam.TULU3_SPLIT

‘train[:10000]’

bridge.recipes.beam.h200.beam.TULU3_REVISION

‘b14afda60f1bbebe55d5d2fa1e4df5042f97f8be’

bridge.recipes.beam.h200.beam.CONVERGENCE_V2_SEQ_LENGTH

8192

bridge.recipes.beam.h200.beam.CONVERGENCE_V2_GLOBAL_BATCH_SIZE

8

bridge.recipes.beam.h200.beam._beam_model_config(

hf_path: str | None,

) → megatron.bridge.models.common.ModelConfig

Build the Beam model config from the pinned HF checkpoint, or from hf_path when given.

bridge.recipes.beam.h200.beam._set_beam_tokenizer(

cfg: megatron.bridge.training.config.ConfigContainer,

hf_path: str | None,

) → None

Point the tokenizer at the same checkpoint as the model.

bridge.recipes.beam.h200.beam._set_beam_parallelism(

cfg: megatron.bridge.training.config.ConfigContainer,

*,

tensor_parallel: int,

pipeline_parallel: int,

expert_parallel: int,

seq_length: int,

) → None

Apply the parallel layout and the Beam execution settings shared by every recipe.

bridge.recipes.beam.h200.beam._set_convergence_v2_optimizer(

cfg: megatron.bridge.training.config.ConfigContainer,

*,

lr: float,

) → None

Apply the qwen3_30b_a3b_convergence_v2 optimizer, schedule and precision fingerprint.

bridge.recipes.beam.h200.beam._set_convergence_v2_dataset(

cfg: megatron.bridge.training.config.ConfigContainer,

*,

pad_seq_to_mult: int,

) → None

Use the pinned 8K offline-packed Tulu 3 selection of qwen3_30b_a3b_convergence_v2.

bridge.recipes.beam.h200.beam.beam_pretrain_128gpu_h200_bf16_config(

hf_path: str | None = None,

) → megatron.bridge.training.config.ConfigContainer

Return a pre-training config for Beam on 16 H200 nodes.

Parallelism: TP=2, PP=1, EP=128, SP (DP=64). GBS 256 at 4K tokens is four micro-batches per data-parallel rank. The precision-aware optimizer keeps BF16 gradients and Adam moments, which is what fits 501B parameters on 128 H200 GPUs; the recipe never selects a pretrained checkpoint.

Parameters:

hf_path – Optional local HF checkpoint used instead of the pinned Hub revision.

Returns:

ConfigContainer for Beam pre-training.

bridge.recipes.beam.h200.beam.beam_sft_128gpu_h200_bf16_config(

hf_path: str | None = None,

) → megatron.bridge.training.config.ConfigContainer

Return a full SFT config for Beam on 16 H200 nodes under qwen3_30b_a3b_convergence_v2.

Parallelism: TP=8, PP=2, EP=64, SP (DP=8), so each data-parallel rank takes one 8K pack of the eight per step. The contract’s GBS 8 caps DP at 8; reaching the 16 model-parallel ranks with TP and PP rather than CP keeps the contract’s loss normalization (CP > 1 forces per-token loss for finetuning).

Parameters:

hf_path – Optional local HF checkpoint used instead of the pinned Hub revision.

Returns:

ConfigContainer for Beam full-parameter SFT.

bridge.recipes.beam.h200.beam.beam_peft_16gpu_h200_bf16_config(

peft_scheme: str | megatron.bridge.peft.base.PEFT = 'lora',

hf_path: str | None = None,

) → megatron.bridge.training.config.ConfigContainer

Return a PEFT config for Beam on 2 H200 nodes under qwen3_30b_a3b_convergence_v2.

Parallelism: TP=2, PP=1, EP=16, SP (DP=8). LoRA/DoRA use rank 8, alpha 16 and no dropout on the attention projections linear_qkv (whose output also carries Beam’s attention output gate) and linear_proj; the base model stays frozen.

Parameters:

• peft_scheme – PEFT scheme - ‘lora’, ‘dora’, or a PEFT instance. Default: ‘lora’.

• hf_path – Optional local HF checkpoint used instead of the pinned Hub revision.

Returns:

ConfigContainer for Beam parameter-efficient fine-tuning.