bridge.perf_recipes.deepseek.vrnvl72.deepseek_v3#

VRNVL72 performance recipes for DeepSeek V3.

Module Contents#

Functions#

deepseek_v3_pretrain_128gpu_vrnvl72_bf16_config

DeepSeek V3 pretrain: 128× VRNVL72, BF16.

deepseek_v3_pretrain_128gpu_vrnvl72_fp8cs_config

DeepSeek V3 pretrain: 128× VRNVL72, FP8 current-scaling.

deepseek_v3_pretrain_128gpu_vrnvl72_fp8mx_config

DeepSeek V3 pretrain: 128× VRNVL72, MXFP8.

deepseek_v3_pretrain_128gpu_vrnvl72_nvfp4_config

DeepSeek V3 pretrain: 128× VRNVL72, NVFP4.

deepseek_v3_pretrain_256gpu_vrnvl72_bf16_config

DeepSeek V3 pretrain: 256× VRNVL72, BF16 (alias of GB300).

deepseek_v3_pretrain_256gpu_vrnvl72_fp8cs_config

DeepSeek V3 pretrain: 256× VRNVL72, FP8-CS (alias of GB300).

deepseek_v3_pretrain_256gpu_vrnvl72_fp8mx_config

DeepSeek V3 pretrain: 256× VRNVL72, FP8-MX (alias of GB300).

deepseek_v3_pretrain_64gpu_vrnvl72_fp8mx_proxy_config

DeepSeek V3 MXFP8 debugging proxy: 13 decoder layers on 64 VRNVL72 GPUs.

deepseek_v3_pretrain_256gpu_vrnvl72_nvfp4_config

DeepSeek V3 pretrain: 256× VRNVL72, NVFP4 with full-iteration CUDA graph.

deepseek_v3_pretrain_64gpu_vrnvl72_nvfp4_proxy_config

DeepSeek V3 NVFP4 debugging proxy: 13 decoder layers on 64 VRNVL72 GPUs.

API#

bridge.perf_recipes.deepseek.vrnvl72.deepseek_v3.deepseek_v3_pretrain_128gpu_vrnvl72_bf16_config() → megatron.bridge.perf_recipes.deepseek.common.ConfigContainer#

DeepSeek V3 pretrain: 128× VRNVL72, BF16.

bridge.perf_recipes.deepseek.vrnvl72.deepseek_v3.deepseek_v3_pretrain_128gpu_vrnvl72_fp8cs_config() → megatron.bridge.perf_recipes.deepseek.common.ConfigContainer#

DeepSeek V3 pretrain: 128× VRNVL72, FP8 current-scaling.

bridge.perf_recipes.deepseek.vrnvl72.deepseek_v3.deepseek_v3_pretrain_128gpu_vrnvl72_fp8mx_config() → megatron.bridge.perf_recipes.deepseek.common.ConfigContainer#

DeepSeek V3 pretrain: 128× VRNVL72, MXFP8.

bridge.perf_recipes.deepseek.vrnvl72.deepseek_v3.deepseek_v3_pretrain_128gpu_vrnvl72_nvfp4_config() → megatron.bridge.perf_recipes.deepseek.common.ConfigContainer#

DeepSeek V3 pretrain: 128× VRNVL72, NVFP4.

bridge.perf_recipes.deepseek.vrnvl72.deepseek_v3.deepseek_v3_pretrain_256gpu_vrnvl72_bf16_config() → megatron.bridge.perf_recipes.deepseek.common.ConfigContainer#

DeepSeek V3 pretrain: 256× VRNVL72, BF16 (alias of GB300).

bridge.perf_recipes.deepseek.vrnvl72.deepseek_v3.deepseek_v3_pretrain_256gpu_vrnvl72_fp8cs_config() → megatron.bridge.perf_recipes.deepseek.common.ConfigContainer#

DeepSeek V3 pretrain: 256× VRNVL72, FP8-CS (alias of GB300).

bridge.perf_recipes.deepseek.vrnvl72.deepseek_v3.deepseek_v3_pretrain_256gpu_vrnvl72_fp8mx_config() → megatron.bridge.perf_recipes.deepseek.common.ConfigContainer#

DeepSeek V3 pretrain: 256× VRNVL72, FP8-MX (alias of GB300).

bridge.perf_recipes.deepseek.vrnvl72.deepseek_v3.deepseek_v3_pretrain_64gpu_vrnvl72_fp8mx_proxy_config() → megatron.bridge.perf_recipes.deepseek.common.ConfigContainer#

DeepSeek V3 MXFP8 debugging proxy: 13 decoder layers on 64 VRNVL72 GPUs.

Keep GBS=4096, per-layer shapes, EP=32 and one MTP layer. PP=1 without virtual pipelining gives dense DP=64, expert DP=2 and 64 microbatches. Retain full-iteration graphs and the non-pipelined combined-1F1B MoE overlap path. Allow nondeterministic kernels for the fused SwiGLU scale gradient, matching the GB300 proxy. This reduced-depth model does not reproduce at-scale pipeline traffic, convergence or memory use; GPU validation is required.

bridge.perf_recipes.deepseek.vrnvl72.deepseek_v3.deepseek_v3_pretrain_256gpu_vrnvl72_nvfp4_config() → megatron.bridge.perf_recipes.deepseek.common.ConfigContainer#

DeepSeek V3 pretrain: 256× VRNVL72, NVFP4 with full-iteration CUDA graph.

bridge.perf_recipes.deepseek.vrnvl72.deepseek_v3.deepseek_v3_pretrain_64gpu_vrnvl72_nvfp4_proxy_config() → megatron.bridge.perf_recipes.deepseek.common.ConfigContainer#

DeepSeek V3 NVFP4 debugging proxy: 13 decoder layers on 64 VRNVL72 GPUs.

Keep GBS=4096, per-layer shapes, EP=32 and one MTP layer. PP=1 without virtual pipelining gives dense DP=64, expert DP=2 and 64 microbatches. Retain full-iteration graphs and the non-pipelined combined-1F1B MoE overlap path. Use this reduced-depth model for feature and kernel performance experiments, not convergence or at-scale acceptance.