bridge.recipes.deepseek.gb200.deepseek_v4#
GB200 recipes for DeepSeek V4 Flash.
Validated on OCI GB200 NVL72 (64 GPUs, PP=8, EP=8).
Module Contents#
Functions#
DeepSeek-V4-Flash BF16 pre-training on 64-GPU GB200 (PP=8, EP=8). |
|
DeepSeek-V4-Flash MXFP8 pre-training on 64-GPU GB200 (PP=8, EP=8). |
|
DeepSeek-V4-Flash BF16 Muon pre-training on 64-GPU GB200 (PP=8, EP=8). |
API#
- bridge.recipes.deepseek.gb200.deepseek_v4.deepseek_v4_flash_pretrain_64gpu_gb200_bf16_config() megatron.bridge.training.config.ConfigContainer#
DeepSeek-V4-Flash BF16 pre-training on 64-GPU GB200 (PP=8, EP=8).
All features validated in 1k-step convergence runs on OCI GB200 NVL72: fused DSA indexer, DSA indexer loss, HybridEP dispatcher, GroupedGEMM, permute fusion, selective recompute with mla_up_proj.
Note: set checkpoint.save_optim=False when using HybridEP.
- bridge.recipes.deepseek.gb200.deepseek_v4.deepseek_v4_flash_pretrain_64gpu_gb200_fp8mx_config() megatron.bridge.training.config.ConfigContainer#
DeepSeek-V4-Flash MXFP8 pre-training on 64-GPU GB200 (PP=8, EP=8).
Note: set checkpoint.save_optim=False when using HybridEP.
- bridge.recipes.deepseek.gb200.deepseek_v4.deepseek_v4_flash_pretrain_64gpu_gb200_bf16_muon_config() megatron.bridge.training.config.ConfigContainer#
DeepSeek-V4-Flash BF16 Muon pre-training on 64-GPU GB200 (PP=8, EP=8).