bridge.recipes.deepseek.gb200.deepseek_v4#
GB200 pretraining, SFT, and PEFT recipes for DeepSeek V4 Flash.
Module Contents#
Functions#
DeepSeek-V4-Flash BF16 pre-training on 64-GPU GB200 (PP=8, EP=8). |
|
DeepSeek-V4-Flash MXFP8 pre-training on 64-GPU GB200 (PP=8, EP=8). |
|
Return the scalable real-training DeepSeek V4 Flash GB200 config. |
|
DeepSeek-V4-Flash BF16 Muon pre-training on 64-GPU GB200 (PP=8, EP=8). |
|
Return the GB200-optimized offline-packed OpenMath SFT config. |
|
|
Return the GB200-optimized packed OpenMath LoRA config. |
API#
- bridge.recipes.deepseek.gb200.deepseek_v4.deepseek_v4_flash_pretrain_64gpu_gb200_bf16_config() megatron.bridge.training.config.ConfigContainer#
DeepSeek-V4-Flash BF16 pre-training on 64-GPU GB200 (PP=8, EP=8).
The core 64-GPU topology completed 1k-step convergence runs with fused DSA indexer and loss, HybridEP, grouped GEMM, permute fusion, and selective recompute. The current router fusion was also exercised by the derived library recipe’s 64-GPU 100-step run.
Note: set checkpoint.save_optim=False when using HybridEP.
- bridge.recipes.deepseek.gb200.deepseek_v4.deepseek_v4_flash_pretrain_64gpu_gb200_fp8mx_config() megatron.bridge.training.config.ConfigContainer#
DeepSeek-V4-Flash MXFP8 pre-training on 64-GPU GB200 (PP=8, EP=8).
Note: set checkpoint.save_optim=False when using HybridEP.
- bridge.recipes.deepseek.gb200.deepseek_v4.deepseek_v4_flash_pretrain_64gpu_gb200_fp8mx_library_config() megatron.bridge.training.config.ConfigContainer#
Return the scalable real-training DeepSeek V4 Flash GB200 config.
This 64-GPU variant uses PP4/VPP4, EP16, and selective recompute without activation offload. Expert capacity, paged stash, CUDA graphs, and forced load balancing remain disabled to preserve natural-routing training semantics.
- bridge.recipes.deepseek.gb200.deepseek_v4.deepseek_v4_flash_pretrain_64gpu_gb200_bf16_muon_config() megatron.bridge.training.config.ConfigContainer#
DeepSeek-V4-Flash BF16 Muon pre-training on 64-GPU GB200 (PP=8, EP=8).
- bridge.recipes.deepseek.gb200.deepseek_v4.deepseek_v4_flash_sft_openmath_thinking_packed_gb200_config() megatron.bridge.training.config.ConfigContainer#
Return the GB200-optimized offline-packed OpenMath SFT config.
This variant adds HybridEP dispatch, safe uneven-input padding, fused DSA execution, static packed-sequence shapes, and grouped expert GEMMs to the hardware-agnostic packed SFT recipe.
- bridge.recipes.deepseek.gb200.deepseek_v4.deepseek_v4_flash_peft_openmath_thinking_packed_gb200_config() megatron.bridge.training.config.ConfigContainer#
Return the GB200-optimized packed OpenMath LoRA config.