bridge.perf_recipes.deepseek.vrnvl72.deepseek_v3#
VRNVL72 performance recipes for DeepSeek V3.
Module Contents#
Functions#
DeepSeek V3 pretrain: 128× VRNVL72, BF16. |
|
DeepSeek V3 pretrain: 128× VRNVL72, FP8 current-scaling. |
|
DeepSeek V3 pretrain: 128× VRNVL72, MXFP8. |
|
DeepSeek V3 pretrain: 128× VRNVL72, NVFP4. |
|
DeepSeek V3 pretrain: 256× VRNVL72, BF16 (alias of GB300). |
|
DeepSeek V3 pretrain: 256× VRNVL72, FP8-CS (alias of GB300). |
|
DeepSeek V3 pretrain: 256× VRNVL72, FP8-MX (alias of GB300). |
|
DeepSeek V3 MXFP8 debugging proxy: 13 decoder layers on 64 VRNVL72 GPUs. |
|
DeepSeek V3 pretrain: 256× VRNVL72, NVFP4 with full-iteration CUDA graph. |
|
DeepSeek V3 NVFP4 debugging proxy: 13 decoder layers on 64 VRNVL72 GPUs. |
API#
- bridge.perf_recipes.deepseek.vrnvl72.deepseek_v3.deepseek_v3_pretrain_128gpu_vrnvl72_bf16_config() megatron.bridge.perf_recipes.deepseek.common.ConfigContainer#
DeepSeek V3 pretrain: 128× VRNVL72, BF16.
- bridge.perf_recipes.deepseek.vrnvl72.deepseek_v3.deepseek_v3_pretrain_128gpu_vrnvl72_fp8cs_config() megatron.bridge.perf_recipes.deepseek.common.ConfigContainer#
DeepSeek V3 pretrain: 128× VRNVL72, FP8 current-scaling.
- bridge.perf_recipes.deepseek.vrnvl72.deepseek_v3.deepseek_v3_pretrain_128gpu_vrnvl72_fp8mx_config() megatron.bridge.perf_recipes.deepseek.common.ConfigContainer#
DeepSeek V3 pretrain: 128× VRNVL72, MXFP8.
- bridge.perf_recipes.deepseek.vrnvl72.deepseek_v3.deepseek_v3_pretrain_128gpu_vrnvl72_nvfp4_config() megatron.bridge.perf_recipes.deepseek.common.ConfigContainer#
DeepSeek V3 pretrain: 128× VRNVL72, NVFP4.
- bridge.perf_recipes.deepseek.vrnvl72.deepseek_v3.deepseek_v3_pretrain_256gpu_vrnvl72_bf16_config() megatron.bridge.perf_recipes.deepseek.common.ConfigContainer#
DeepSeek V3 pretrain: 256× VRNVL72, BF16 (alias of GB300).
- bridge.perf_recipes.deepseek.vrnvl72.deepseek_v3.deepseek_v3_pretrain_256gpu_vrnvl72_fp8cs_config() megatron.bridge.perf_recipes.deepseek.common.ConfigContainer#
DeepSeek V3 pretrain: 256× VRNVL72, FP8-CS (alias of GB300).
- bridge.perf_recipes.deepseek.vrnvl72.deepseek_v3.deepseek_v3_pretrain_256gpu_vrnvl72_fp8mx_config() megatron.bridge.perf_recipes.deepseek.common.ConfigContainer#
DeepSeek V3 pretrain: 256× VRNVL72, FP8-MX (alias of GB300).
- bridge.perf_recipes.deepseek.vrnvl72.deepseek_v3.deepseek_v3_pretrain_64gpu_vrnvl72_fp8mx_proxy_config() megatron.bridge.perf_recipes.deepseek.common.ConfigContainer#
DeepSeek V3 MXFP8 debugging proxy: 13 decoder layers on 64 VRNVL72 GPUs.
Keep GBS=4096, per-layer shapes, EP=32 and one MTP layer. PP=1 without virtual pipelining gives dense DP=64, expert DP=2 and 64 microbatches. Retain full-iteration graphs and the non-pipelined combined-1F1B MoE overlap path. Allow nondeterministic kernels for the fused SwiGLU scale gradient, matching the GB300 proxy. This reduced-depth model does not reproduce at-scale pipeline traffic, convergence or memory use; GPU validation is required.
- bridge.perf_recipes.deepseek.vrnvl72.deepseek_v3.deepseek_v3_pretrain_256gpu_vrnvl72_nvfp4_config() megatron.bridge.perf_recipes.deepseek.common.ConfigContainer#
DeepSeek V3 pretrain: 256× VRNVL72, NVFP4 with full-iteration CUDA graph.
- bridge.perf_recipes.deepseek.vrnvl72.deepseek_v3.deepseek_v3_pretrain_64gpu_vrnvl72_nvfp4_proxy_config() megatron.bridge.perf_recipes.deepseek.common.ConfigContainer#
DeepSeek V3 NVFP4 debugging proxy: 13 decoder layers on 64 VRNVL72 GPUs.
Keep GBS=4096, per-layer shapes, EP=32 and one MTP layer. PP=1 without virtual pipelining gives dense DP=64, expert DP=2 and 64 microbatches. Retain full-iteration graphs and the non-pipelined combined-1F1B MoE overlap path. Use this reduced-depth model for feature and kernel performance experiments, not convergence or at-scale acceptance.