bridge.perf_recipes.deepseek.vrnvl72.deepseek_v3#
VRNVL72 performance recipes for DeepSeek V3.
Module Contents#
Functions#
DeepSeek V3 pretrain: 128× VRNVL72, BF16. |
|
DeepSeek V3 pretrain: 128× VRNVL72, FP8 current-scaling. |
|
DeepSeek V3 pretrain: 128× VRNVL72, MXFP8. |
|
DeepSeek V3 pretrain: 128× VRNVL72, NVFP4. |
|
DeepSeek V3 pretrain: 256× VRNVL72, BF16 (alias of GB300). |
|
DeepSeek V3 pretrain: 256× VRNVL72, FP8-CS (alias of GB300). |
|
DeepSeek V3 pretrain: 256× VRNVL72, FP8-MX (alias of GB300). |
|
DeepSeek V3 MXFP8 debugging proxy: 13 decoder layers on 64 VRNVL72 GPUs. |
|
DeepSeek V3 pretrain: 256× VRNVL72, NVFP4 with full-iteration CUDA graph. |
|
DeepSeek V3 NVFP4 debugging proxy: 13 decoder layers on 64 VRNVL72 GPUs. |
|
Shrink a 256-GPU VRNVL72 DeepSeek V3 recipe onto one 4-GPU VRNVL72 node. |
|
DeepSeek V3 MXFP8 profiling proxy: 13 decoder layers and 32 experts on one 4-GPU VRNVL72 node. |
|
DeepSeek V3 NVFP4 profiling proxy: 13 decoder layers and 32 experts on one 4-GPU VRNVL72 node. |
API#
- bridge.perf_recipes.deepseek.vrnvl72.deepseek_v3.deepseek_v3_pretrain_128gpu_vrnvl72_bf16_config() megatron.bridge.perf_recipes.deepseek.common.ConfigContainer#
DeepSeek V3 pretrain: 128× VRNVL72, BF16.
- bridge.perf_recipes.deepseek.vrnvl72.deepseek_v3.deepseek_v3_pretrain_128gpu_vrnvl72_fp8cs_config() megatron.bridge.perf_recipes.deepseek.common.ConfigContainer#
DeepSeek V3 pretrain: 128× VRNVL72, FP8 current-scaling.
- bridge.perf_recipes.deepseek.vrnvl72.deepseek_v3.deepseek_v3_pretrain_128gpu_vrnvl72_fp8mx_config() megatron.bridge.perf_recipes.deepseek.common.ConfigContainer#
DeepSeek V3 pretrain: 128× VRNVL72, MXFP8.
- bridge.perf_recipes.deepseek.vrnvl72.deepseek_v3.deepseek_v3_pretrain_128gpu_vrnvl72_nvfp4_config() megatron.bridge.perf_recipes.deepseek.common.ConfigContainer#
DeepSeek V3 pretrain: 128× VRNVL72, NVFP4.
- bridge.perf_recipes.deepseek.vrnvl72.deepseek_v3.deepseek_v3_pretrain_256gpu_vrnvl72_bf16_config() megatron.bridge.perf_recipes.deepseek.common.ConfigContainer#
DeepSeek V3 pretrain: 256× VRNVL72, BF16 (alias of GB300).
- bridge.perf_recipes.deepseek.vrnvl72.deepseek_v3.deepseek_v3_pretrain_256gpu_vrnvl72_fp8cs_config() megatron.bridge.perf_recipes.deepseek.common.ConfigContainer#
DeepSeek V3 pretrain: 256× VRNVL72, FP8-CS (alias of GB300).
- bridge.perf_recipes.deepseek.vrnvl72.deepseek_v3.deepseek_v3_pretrain_256gpu_vrnvl72_fp8mx_config() megatron.bridge.perf_recipes.deepseek.common.ConfigContainer#
DeepSeek V3 pretrain: 256× VRNVL72, FP8-MX (alias of GB300).
- bridge.perf_recipes.deepseek.vrnvl72.deepseek_v3.deepseek_v3_pretrain_64gpu_vrnvl72_fp8mx_proxy_config() megatron.bridge.perf_recipes.deepseek.common.ConfigContainer#
DeepSeek V3 MXFP8 debugging proxy: 13 decoder layers on 64 VRNVL72 GPUs.
Keep GBS=4096, per-layer shapes, EP=32 and one MTP layer. PP=1 without virtual pipelining gives dense DP=64, expert DP=2 and 64 microbatches. Retain full-iteration graphs and the non-pipelined combined-1F1B MoE overlap path. This reduced-depth model does not reproduce at-scale pipeline traffic, convergence or memory use; GPU validation is required.
- bridge.perf_recipes.deepseek.vrnvl72.deepseek_v3.deepseek_v3_pretrain_256gpu_vrnvl72_nvfp4_config() megatron.bridge.perf_recipes.deepseek.common.ConfigContainer#
DeepSeek V3 pretrain: 256× VRNVL72, NVFP4 with full-iteration CUDA graph.
- bridge.perf_recipes.deepseek.vrnvl72.deepseek_v3.deepseek_v3_pretrain_64gpu_vrnvl72_nvfp4_proxy_config() megatron.bridge.perf_recipes.deepseek.common.ConfigContainer#
DeepSeek V3 NVFP4 debugging proxy: 13 decoder layers on 64 VRNVL72 GPUs.
Keep GBS=4096, per-layer shapes, EP=32 and one MTP layer. PP=1 without virtual pipelining gives dense DP=64, expert DP=2 and 64 microbatches. Retain full-iteration graphs and the non-pipelined combined-1F1B MoE overlap path. Use this reduced-depth model for feature and kernel performance experiments, not convergence or at-scale acceptance.
- bridge.perf_recipes.deepseek.vrnvl72.deepseek_v3._apply_deepseek_v3_4gpu_vrnvl72_proxy(
- cfg: megatron.bridge.perf_recipes.deepseek.common.ConfigContainer,
Shrink a 256-GPU VRNVL72 DeepSeek V3 recipe onto one 4-GPU VRNVL72 node.
Both 4-GPU proxies go through this helper so that every model, parallelism and batch setting matches and only the quantization recipe differs between their profiles.
- bridge.perf_recipes.deepseek.vrnvl72.deepseek_v3.deepseek_v3_pretrain_4gpu_vrnvl72_fp8mx_proxy_config() megatron.bridge.perf_recipes.deepseek.common.ConfigContainer#
DeepSeek V3 MXFP8 profiling proxy: 13 decoder layers and 32 experts on one 4-GPU VRNVL72 node.
Matches
deepseek_v3_pretrain_4gpu_vrnvl72_nvfp4_proxy_configin everything except the quantization recipe, so the two profiles can be compared directly. Per-layer shapes and per-rank expert work follow the 256-GPU parent: TP=1, PP=1, EP=4, MBS=1, GBS=128, one MTP layer, full-iteration CUDA graphs, HybridEP and combined-1F1B MoE overlap. Attention and all GEMMs run in MXFP8; the LM head stays in BF16. This proxy does not reproduce at-scale communication, convergence or memory use.
- bridge.perf_recipes.deepseek.vrnvl72.deepseek_v3.deepseek_v3_pretrain_4gpu_vrnvl72_nvfp4_proxy_config() megatron.bridge.perf_recipes.deepseek.common.ConfigContainer#
DeepSeek V3 NVFP4 profiling proxy: 13 decoder layers and 32 experts on one 4-GPU VRNVL72 node.
Matches
deepseek_v3_pretrain_4gpu_vrnvl72_fp8mx_proxy_configin everything except the quantization recipe, so the two profiles can be compared directly. Per-layer shapes and per-rank expert work follow the 256-GPU parent: TP=1, PP=1, EP=4, MBS=1, GBS=128, one MTP layer, full-iteration CUDA graphs, HybridEP and combined-1F1B MoE overlap. GEMMs run in NVFP4, attention in MXFP8 E4M3 as in the MXFP8 proxy, and the LM head in BF16. This proxy does not reproduce at-scale communication, convergence or memory use.