bridge.perf_recipes.nemotronh.gb300.nemotronh

GB300 performance recipes for Nemotron 3 and 3.5.

Module Contents

Functions

_build_nemotron_3_super_gb300_bf16	Shared HybridEP BF16 base for the Nemotron 3 Super GB300 recipe and its VRNVL72 alias.
nemotron_3_super_pretrain_64gpu_gb300_bf16_config	Nemotron 3 Super pretrain: 64× GB300, BF16, NCCL EP.
_build_nemotron_3_super_gb300_mxfp8
nemotron_3_super_pretrain_64gpu_gb300_fp8mx_config	Nemotron 3 Super pretrain: 64× GB300, MXFP8 with full-iteration CUDA graph.
_build_nemotron_3_super_gb300_nvfp4	Shared HybridEP NVFP4 base for the Nemotron 3 Super GB300 recipe and its VRNVL72 alias.
nemotron_3_super_pretrain_64gpu_gb300_nvfp4_config	Nemotron 3 Super pretrain: 64× GB300, NVFP4, NCCL EP.
_nemotron_3_ultra_gb300_fp8mx_config	Shared builder for Nemotron 3 Ultra GB300 MXFP8 GTP perf recipes.
nemotron_3_ultra_pretrain_256gpu_gb300_fp8mx_config	Nemotron 3 Ultra (550B-A55B LatentMoE) pretrain: 256× GB300, MXFP8, GTP.
nemotron_3_ultra_pretrain_256gpu_gb300_nvfp4_config	Nemotron 3 Ultra (550B-A55B LatentMoE) pretrain: 256× GB300, NVFP4, GTP.
_build_nemotron_3_nano_gb300_bf16	Shared HybridEP BF16 base for the Nemotron 3 Nano GB300 recipe and its VRNVL72 alias.
nemotron_3_nano_pretrain_8gpu_gb300_bf16_config	Nemotron 3 Nano pretrain: 8× GB300, BF16, NCCL EP.
_build_nemotron_3_nano_gb300_mxfp8
nemotron_3_nano_pretrain_8gpu_gb300_fp8mx_config	Nemotron 3 Nano pretrain: 8× GB300, MXFP8.
nemotron_3_nano_pretrain_8gpu_gb300_nvfp4_config	Nemotron 3 Nano pretrain: 8× GB300, NVFP4.
_build_nemotron_3_5_lightning_gb300_bf16	Shared HybridEP BF16 base for the Nemotron 3.5 Lightning GB300 recipe and its VRNVL72 alias.
nemotron_3_5_lightning_pretrain_8gpu_gb300_bf16_config	Nemotron 3.5 Lightning pretrain: 8× GB300, BF16, NCCL EP.
_build_nemotron_3_5_lightning_gb300_mxfp8
nemotron_3_5_lightning_pretrain_8gpu_gb300_fp8mx_config	Nemotron 3.5 Lightning pretrain: 8× GB300, MXFP8 with full-iteration CUDA graph.
nemotron_3_5_lightning_pretrain_8gpu_gb300_fp8mx_fsdp_config	Nemotron 3.5 Lightning pretrain: 8× GB300, MXFP8, Megatron FSDP.
nemotron_3_5_lightning_pretrain_8gpu_gb300_nvfp4_config	Nemotron 3.5 Lightning pretrain: 8× GB300, NVFP4.

Data

_NEMOTRON_3_5_LIGHTNING_MODEL_ID
_NEMOTRON_3_5_LIGHTNING_MODEL_REVISION

API

bridge.perf_recipes.nemotronh.gb300.nemotronh._NEMOTRON_3_5_LIGHTNING_MODEL_ID

‘nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16’

bridge.perf_recipes.nemotronh.gb300.nemotronh._NEMOTRON_3_5_LIGHTNING_MODEL_REVISION

‘b3caaabed0263651a17dc1f2d4ce97e794f76c44’

bridge.perf_recipes.nemotronh.gb300.nemotronh._build_nemotron_3_super_gb300_bf16() → megatron.bridge.perf_recipes.nemotronh.common.ConfigContainer

Shared HybridEP BF16 base for the Nemotron 3 Super GB300 recipe and its VRNVL72 alias.

bridge.perf_recipes.nemotronh.gb300.nemotronh.nemotron_3_super_pretrain_64gpu_gb300_bf16_config() → megatron.bridge.perf_recipes.nemotronh.common.ConfigContainer

Nemotron 3 Super pretrain: 64× GB300, BF16, NCCL EP.

bridge.perf_recipes.nemotronh.gb300.nemotronh._build_nemotron_3_super_gb300_mxfp8() → megatron.bridge.perf_recipes.nemotronh.common.ConfigContainer

bridge.perf_recipes.nemotronh.gb300.nemotronh.nemotron_3_super_pretrain_64gpu_gb300_fp8mx_config() → megatron.bridge.perf_recipes.nemotronh.common.ConfigContainer

Nemotron 3 Super pretrain: 64× GB300, MXFP8 with full-iteration CUDA graph.

bridge.perf_recipes.nemotronh.gb300.nemotronh._build_nemotron_3_super_gb300_nvfp4() → megatron.bridge.perf_recipes.nemotronh.common.ConfigContainer

Shared HybridEP NVFP4 base for the Nemotron 3 Super GB300 recipe and its VRNVL72 alias.

bridge.perf_recipes.nemotronh.gb300.nemotronh.nemotron_3_super_pretrain_64gpu_gb300_nvfp4_config() → megatron.bridge.perf_recipes.nemotronh.common.ConfigContainer

Nemotron 3 Super pretrain: 64× GB300, NVFP4, NCCL EP.

bridge.perf_recipes.nemotronh.gb300.nemotronh._nemotron_3_ultra_gb300_fp8mx_config(

*,

num_gpus: int,

expert_model_parallel_size: int,

global_batch_size: int,

) → megatron.bridge.perf_recipes.nemotronh.common.ConfigContainer

Shared builder for Nemotron 3 Ultra GB300 MXFP8 GTP perf recipes.

bridge.perf_recipes.nemotronh.gb300.nemotronh.nemotron_3_ultra_pretrain_256gpu_gb300_fp8mx_config(

*,

num_gpus: int = 256,

) → megatron.bridge.perf_recipes.nemotronh.common.ConfigContainer

Nemotron 3 Ultra (550B-A55B LatentMoE) pretrain: 256× GB300, MXFP8, GTP.

TP1 / PP1 / CP1 / EP64 / ETP1, GBS 256 / MBS 1, seq 8192, BF16 + MXFP8 mixed precision, dense GTP64, NCCLEP flex dispatcher, CuteDSL fused grouped MLP, selective recompute + fine-grained activation offload of the expert MLP, MTP=2.

Parameters:

num_gpus – Launch GPU count used to size HSDP optimizer instances. Defaults to the canonical 256-GPU benchmark allocation.

bridge.perf_recipes.nemotronh.gb300.nemotronh.nemotron_3_ultra_pretrain_256gpu_gb300_nvfp4_config(

*,

num_gpus: int = 256,

) → megatron.bridge.perf_recipes.nemotronh.common.ConfigContainer

Nemotron 3 Ultra (550B-A55B LatentMoE) pretrain: 256× GB300, NVFP4, GTP.

TP1 / PP1 / CP1 / EP64 / ETP1, GBS 256 / MBS 1, seq 8192, BF16 + NVFP4 mixed precision, dense GTP64, NCCLEP flex dispatcher with local attention/Mamba CUDA graphs, CuteDSL fused grouped MLP, selective recompute + fine-grained activation offload of the expert MLP, MTP=2.

Parameters:

num_gpus – Retained for compatibility; sharding is rack-local and the distributed-optimizer instance count is fixed at one.

bridge.perf_recipes.nemotronh.gb300.nemotronh._build_nemotron_3_nano_gb300_bf16() → megatron.bridge.perf_recipes.nemotronh.common.ConfigContainer

Shared HybridEP BF16 base for the Nemotron 3 Nano GB300 recipe and its VRNVL72 alias.

bridge.perf_recipes.nemotronh.gb300.nemotronh.nemotron_3_nano_pretrain_8gpu_gb300_bf16_config() → megatron.bridge.perf_recipes.nemotronh.common.ConfigContainer

Nemotron 3 Nano pretrain: 8× GB300, BF16, NCCL EP.

bridge.perf_recipes.nemotronh.gb300.nemotronh._build_nemotron_3_nano_gb300_mxfp8() → megatron.bridge.perf_recipes.nemotronh.common.ConfigContainer

bridge.perf_recipes.nemotronh.gb300.nemotronh.nemotron_3_nano_pretrain_8gpu_gb300_fp8mx_config() → megatron.bridge.perf_recipes.nemotronh.common.ConfigContainer

Nemotron 3 Nano pretrain: 8× GB300, MXFP8.

bridge.perf_recipes.nemotronh.gb300.nemotronh.nemotron_3_nano_pretrain_8gpu_gb300_nvfp4_config() → megatron.bridge.perf_recipes.nemotronh.common.ConfigContainer

Nemotron 3 Nano pretrain: 8× GB300, NVFP4.

bridge.perf_recipes.nemotronh.gb300.nemotronh._build_nemotron_3_5_lightning_gb300_bf16() → megatron.bridge.perf_recipes.nemotronh.common.ConfigContainer

Shared HybridEP BF16 base for the Nemotron 3.5 Lightning GB300 recipe and its VRNVL72 alias.

bridge.perf_recipes.nemotronh.gb300.nemotronh.nemotron_3_5_lightning_pretrain_8gpu_gb300_bf16_config() → megatron.bridge.perf_recipes.nemotronh.common.ConfigContainer

Nemotron 3.5 Lightning pretrain: 8× GB300, BF16, NCCL EP.

bridge.perf_recipes.nemotronh.gb300.nemotronh._build_nemotron_3_5_lightning_gb300_mxfp8() → megatron.bridge.perf_recipes.nemotronh.common.ConfigContainer

bridge.perf_recipes.nemotronh.gb300.nemotronh.nemotron_3_5_lightning_pretrain_8gpu_gb300_fp8mx_config() → megatron.bridge.perf_recipes.nemotronh.common.ConfigContainer

Nemotron 3.5 Lightning pretrain: 8× GB300, MXFP8 with full-iteration CUDA graph.

bridge.perf_recipes.nemotronh.gb300.nemotronh.nemotron_3_5_lightning_pretrain_8gpu_gb300_fp8mx_fsdp_config() → megatron.bridge.perf_recipes.nemotronh.common.ConfigContainer

Nemotron 3.5 Lightning pretrain: 8× GB300, MXFP8, Megatron FSDP.

bridge.perf_recipes.nemotronh.gb300.nemotronh.nemotron_3_5_lightning_pretrain_8gpu_gb300_nvfp4_config() → megatron.bridge.perf_recipes.nemotronh.common.ConfigContainer

Nemotron 3.5 Lightning pretrain: 8× GB300, NVFP4.