bridge.recipes.glm.h100.glm5#

H100 recipes for GLM-5.2.

These recipes use the HybridEP MoE dispatcher and the TileLang DSA kernels. Both compile their kernels in every process during the first training step, and the pipeline stages reach that point one after another, so the first step takes far longer than later steps. TileLang loads the compiled kernels from TMPDIR, which must be on a filesystem that allows executing files. Its DSA backward kernels accumulate with floating-point atomics, so runs with deterministic mode enabled are not bit-for-bit reproducible.

Module Contents#

Functions#

glm52_pretrain_416gpu_h100_bf16_config

GLM-5.2 bounded pretraining on 416 H100 GPUs.

glm52_sft_416gpu_h100_bf16_config

GLM-5.2 bounded full SFT on 416 H100 GPUs.

glm52_sft_608gpu_h100_bf16_200k_config

GLM-5.2 200K packed SFT with context parallelism on 608 H100 GPUs.

glm52_peft_208gpu_h100_bf16_config

GLM-5.2 bounded LoRA SFT on 208 H100 GPUs.

Data#

API#

bridge.recipes.glm.h100.glm5._GLM52_MODEL_ID#

‘zai-org/GLM-5.2’

bridge.recipes.glm.h100.glm5._GLM52_MODEL_REVISION#

‘4d67f66cc64d3219133b767c253b2ad1425c6c88’

bridge.recipes.glm.h100.glm5._TULU3_REVISION#

‘b14afda60f1bbebe55d5d2fa1e4df5042f97f8be’

bridge.recipes.glm.h100.glm5._H100_DSA_KERNEL_BACKEND#

‘tilelang’

bridge.recipes.glm.h100.glm5._H100_DISTRIBUTED_TIMEOUT_MINUTES#

240

bridge.recipes.glm.h100.glm5.glm52_pretrain_416gpu_h100_bf16_config() → megatron.bridge.training.config.ConfigContainer#

GLM-5.2 bounded pretraining on 416 H100 GPUs.

bridge.recipes.glm.h100.glm5.glm52_sft_416gpu_h100_bf16_config() → megatron.bridge.training.config.ConfigContainer#

GLM-5.2 bounded full SFT on 416 H100 GPUs.

bridge.recipes.glm.h100.glm5.glm52_sft_608gpu_h100_bf16_200k_config() → megatron.bridge.training.config.ConfigContainer#

GLM-5.2 200K packed SFT with context parallelism on 608 H100 GPUs.

bridge.recipes.glm.h100.glm5.glm52_peft_208gpu_h100_bf16_config(
peft_scheme: str | megatron.bridge.peft.base.PEFT = 'lora',
) → megatron.bridge.training.config.ConfigContainer#

GLM-5.2 bounded LoRA SFT on 208 H100 GPUs.

bridge.recipes.glm.h100.glm5.glm52_h100_sft_config#

None

bridge.recipes.glm.h100.glm5.__all__#

[‘glm52_h100_sft_config’, ‘glm52_peft_208gpu_h100_bf16_config’, ‘glm52_pretrain_416gpu_h100_bf16_con…