bridge.perf_recipes.nemotronh.gb200.nemotronh#
GB200 performance recipes for NemotronH and Nemotron 3.
Module Contents#
Functions#
NemotronH 56B pretrain: 64× GB200, FP8 current-scaling. |
|
Nemotron 3 Super pretrain: 64× GB200, BF16. |
|
Nemotron 3 Super pretrain: 64× GB200, MXFP8. |
|
Nemotron 3 Super pretrain: 64× GB200, NVFP4. |
|
Nemotron 3 Ultra (550B-A55B LatentMoE) pretrain: 256× GB200, MXFP8, Megatron-FSDP (HSDP). |
|
Nemotron 3 Nano pretrain: 8× GB200, BF16. |
|
Nemotron 3 Nano pretrain: 8× GB200, MXFP8. |
|
Nemotron 3 Nano pretrain: 8× GB200, NVFP4. |
|
Nemotron 3.5 Lightning pretrain: 8× GB200, BF16. |
|
Nemotron 3.5 Lightning pretrain: 8× GB200, MXFP8. |
|
|
Nemotron 3.5 Lightning pretrain: 8× GB200, MXFP8, Megatron FSDP. |
Nemotron 3.5 Lightning pretrain: 8× GB200, NVFP4. |
Data#
API#
- bridge.perf_recipes.nemotronh.gb200.nemotronh._NEMOTRON_3_5_LIGHTNING_MODEL_ID#
‘nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16’
- bridge.perf_recipes.nemotronh.gb200.nemotronh._NEMOTRON_3_5_LIGHTNING_MODEL_REVISION#
‘b3caaabed0263651a17dc1f2d4ce97e794f76c44’
- bridge.perf_recipes.nemotronh.gb200.nemotronh.nemotronh_56b_pretrain_64gpu_gb200_fp8cs_config() megatron.bridge.perf_recipes.nemotronh.common.ConfigContainer#
NemotronH 56B pretrain: 64× GB200, FP8 current-scaling.
- bridge.perf_recipes.nemotronh.gb200.nemotronh.nemotron_3_super_pretrain_64gpu_gb200_bf16_config() megatron.bridge.perf_recipes.nemotronh.common.ConfigContainer#
Nemotron 3 Super pretrain: 64× GB200, BF16.
- bridge.perf_recipes.nemotronh.gb200.nemotronh.nemotron_3_super_pretrain_64gpu_gb200_fp8mx_config() megatron.bridge.perf_recipes.nemotronh.common.ConfigContainer#
Nemotron 3 Super pretrain: 64× GB200, MXFP8.
- bridge.perf_recipes.nemotronh.gb200.nemotronh.nemotron_3_super_pretrain_64gpu_gb200_nvfp4_config() megatron.bridge.perf_recipes.nemotronh.common.ConfigContainer#
Nemotron 3 Super pretrain: 64× GB200, NVFP4.
- bridge.perf_recipes.nemotronh.gb200.nemotronh.nemotron_3_ultra_pretrain_256gpu_gb200_fp8mx_config() megatron.bridge.perf_recipes.nemotronh.common.ConfigContainer#
Nemotron 3 Ultra (550B-A55B LatentMoE) pretrain: 256× GB200, MXFP8, Megatron-FSDP (HSDP).
TP2 + SP (due to smaller GB200 HBM) / PP1 / CP1 / EP64 / ETP1, GBS 256 / MBS 1, seq 8192, BF16 + MXFP8 mixed precision, HybridEP flex dispatcher, CuteDSL fused grouped MLP, selective recompute + fine-grained activation offload of the expert MLP, MTP=2.
- bridge.perf_recipes.nemotronh.gb200.nemotronh.nemotron_3_nano_pretrain_8gpu_gb200_bf16_config() megatron.bridge.perf_recipes.nemotronh.common.ConfigContainer#
Nemotron 3 Nano pretrain: 8× GB200, BF16.
- bridge.perf_recipes.nemotronh.gb200.nemotronh.nemotron_3_nano_pretrain_8gpu_gb200_fp8mx_config() megatron.bridge.perf_recipes.nemotronh.common.ConfigContainer#
Nemotron 3 Nano pretrain: 8× GB200, MXFP8.
- bridge.perf_recipes.nemotronh.gb200.nemotronh.nemotron_3_nano_pretrain_8gpu_gb200_nvfp4_config() megatron.bridge.perf_recipes.nemotronh.common.ConfigContainer#
Nemotron 3 Nano pretrain: 8× GB200, NVFP4.
- bridge.perf_recipes.nemotronh.gb200.nemotronh.nemotron_3_5_lightning_pretrain_8gpu_gb200_bf16_config() megatron.bridge.perf_recipes.nemotronh.common.ConfigContainer#
Nemotron 3.5 Lightning pretrain: 8× GB200, BF16.
- bridge.perf_recipes.nemotronh.gb200.nemotronh.nemotron_3_5_lightning_pretrain_8gpu_gb200_fp8mx_config() megatron.bridge.perf_recipes.nemotronh.common.ConfigContainer#
Nemotron 3.5 Lightning pretrain: 8× GB200, MXFP8.
- bridge.perf_recipes.nemotronh.gb200.nemotronh.nemotron_3_5_lightning_pretrain_8gpu_gb200_fp8mx_fsdp_config() megatron.bridge.perf_recipes.nemotronh.common.ConfigContainer#
Nemotron 3.5 Lightning pretrain: 8× GB200, MXFP8, Megatron FSDP.
- bridge.perf_recipes.nemotronh.gb200.nemotronh.nemotron_3_5_lightning_pretrain_8gpu_gb200_nvfp4_config() megatron.bridge.perf_recipes.nemotronh.common.ConfigContainer#
Nemotron 3.5 Lightning pretrain: 8× GB200, NVFP4.