bridge.recipes.nemotronh.h100.nemotron_3_ultra#
Module Contents#
Functions#
Insert pipeline stage separators into a hybrid layer pattern. |
|
Return a pre-training config for Nemotron 3 Ultra. |
|
Apply large-scale execution settings without benchmark-only policy. |
|
Return a convergence-safe Nemotron 3 Ultra pretrain config for 256 H100 GPUs. |
|
|
Return a packed OpenMathInstruct-2 full SFT config for Nemotron 3 Ultra on 384 H100 GPUs. |
|
Return a packed OpenMathInstruct-2 PEFT config for Nemotron 3 Ultra. |
Data#
API#
- bridge.recipes.nemotronh.h100.nemotron_3_ultra.NEMOTRON_3_ULTRA_HF_MODEL_ID#
‘nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16’
- bridge.recipes.nemotronh.h100.nemotron_3_ultra.NEMOTRON_3_ULTRA_TOKENIZER_NAME#
‘nvidia–NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16’
- bridge.recipes.nemotronh.h100.nemotron_3_ultra.NEMOTRON_3_ULTRA_PRETRAIN_SEQ_LENGTH#
8192
- bridge.recipes.nemotronh.h100.nemotron_3_ultra.NEMOTRON_3_ULTRA_OPENMATHINSTRUCT2_SEQ_LENGTH#
4096
- bridge.recipes.nemotronh.h100.nemotron_3_ultra.NEMOTRON_3_ULTRA_SFT_LAYERS_PER_STAGE#
(9, 9, 9, 9, 9, 9, 9, 9, 10, 10, 11, 5)
- bridge.recipes.nemotronh.h100.nemotron_3_ultra._split_hybrid_layer_pattern(
- pattern: str,
- layers_per_stage: tuple[int, ...],
Insert pipeline stage separators into a hybrid layer pattern.
- Parameters:
pattern – Hybrid layer pattern without stage separators, optionally followed by an MTP suffix after “/”.
layers_per_stage – Number of decoder layers on each pipeline stage, in order.
- Returns:
The decoder pattern with “|” between stages, followed by the unchanged MTP suffix.
- Raises:
ValueError – If the pattern already has stage separators or the stage sizes do not cover it exactly.
- bridge.recipes.nemotronh.h100.nemotron_3_ultra.nemotron_3_ultra_pretrain_24gpu_h100_bf16_config() megatron.bridge.training.config.ConfigContainer#
Return a pre-training config for Nemotron 3 Ultra.
- Returns:
Pre-training configuration for Nemotron 3 Ultra.
- bridge.recipes.nemotronh.h100.nemotron_3_ultra._nemotron_3_ultra_large_scale_bf16_config() megatron.bridge.training.config.ConfigContainer#
Apply large-scale execution settings without benchmark-only policy.
- bridge.recipes.nemotronh.h100.nemotron_3_ultra.nemotron_3_ultra_pretrain_256gpu_h100_bf16_fsdp_config() megatron.bridge.training.config.ConfigContainer#
Return a convergence-safe Nemotron 3 Ultra pretrain config for 256 H100 GPUs.
The execution policy adopts the memory-feasible TP4/PP1/EP64 Megatron-FSDP shape derived from the performance configuration. It intentionally keeps BF16 compute, natural expert routing, numerical checks, and the library recipe’s training objective.
- Returns:
H100 BF16 Megatron-FSDP pretraining configuration.
- bridge.recipes.nemotronh.h100.nemotron_3_ultra.nemotron_3_ultra_sft_384gpu_h100_bf16_openmathinstruct2_packed_config() megatron.bridge.training.config.ConfigContainer#
Return a packed OpenMathInstruct-2 full SFT config for Nemotron 3 Ultra on 384 H100 GPUs.
The layout matches the H100 full-SFT starting point documented in the Ultra examples: 48 nodes at TP2/PP12/EP16 with selective activation recompute.
- Returns:
Full-parameter SFT configuration for OpenMathInstruct-2.
- bridge.recipes.nemotronh.h100.nemotron_3_ultra.nemotron_3_ultra_peft_32gpu_h100_bf16_openmathinstruct2_packed_config(
- peft_scheme: str | megatron.bridge.peft.base.PEFT | None = 'lora',
Return a packed OpenMathInstruct-2 PEFT config for Nemotron 3 Ultra.
- Parameters:
peft_scheme – PEFT scheme, PEFT instance, or “none”.
- Returns:
PEFT configuration for OpenMathInstruct-2.
- bridge.recipes.nemotronh.h100.nemotron_3_ultra.__all__#
[‘nemotron_3_ultra_peft_32gpu_h100_bf16_openmathinstruct2_packed_config’, ‘nemotron_3_ultra_pretrain…