bridge.recipes.nemotronh_multimodal.gb200.nemotron_35_super_vl#
Nemotron 3.5 Super VL GB200 training recipes.
Module Contents#
Functions#
Return the 64-GB200 BF16 pretraining config for Super VL. |
|
Return the 64-GB200 BF16 SFT configuration for Super VL. |
|
|
Return the 128-GB200 BF16 128K CoderForge SFT configuration. |
Return the 16-GB200 BF16 LoRA configuration for Super VL. |
Data#
API#
- bridge.recipes.nemotronh_multimodal.gb200.nemotron_35_super_vl._CODERFORGE_REVISION#
‘060fca96cf723b2ebab3181e9e59fafd273df3cb’
- bridge.recipes.nemotronh_multimodal.gb200.nemotron_35_super_vl.nemotron_35_super_vl_pretrain_64gpu_gb200_bf16_config() megatron.bridge.training.config.ConfigContainer#
Return the 64-GB200 BF16 pretraining config for Super VL.
The recipe preserves the H100 data, objective, optimizer schedule, natural routing, and numerical-safety contract. It uses the 8192-token workload shape of the corresponding Nemotron 3 Super GB200 recipe. Other GB200-specific changes are limited to the parallel layout, full-precision optimizer state, communication overlap, scoped CUDA graphs, and the single-NVL72 HybridEP environment.
- Returns:
The Super-VL GB200 pretraining configuration.
- bridge.recipes.nemotronh_multimodal.gb200.nemotron_35_super_vl.nemotron_35_super_vl_sft_64gpu_gb200_bf16_config() megatron.bridge.training.config.ConfigContainer#
Return the 64-GB200 BF16 SFT configuration for Super VL.
This preserves the H100 SFT data, objective, optimizer, schedule, batch, and trainable-parameter contract while mapping execution to one NVL72 domain. HybridEP keeps the EP64 dispatch path inside that NVLink domain; eager execution remains the correctness-first baseline for subsequent graph and overlap tuning.
- Returns:
The Super-VL GB200 SFT configuration.
- bridge.recipes.nemotronh_multimodal.gb200.nemotron_35_super_vl.nemotron_35_super_vl_sft_long_context_128gpu_gb200_bf16_config() megatron.bridge.training.config.ConfigContainer#
Return the 128-GB200 BF16 128K CoderForge SFT configuration.
The text-only CoderForge workload uses two prediction depths with a shared MTP block while exercising offline sequence packing and context parallelism. TP1/PP2 retains the verified 38/50 pipeline balance, CP32 leaves 4K tokens per rank, and EP64 shards one pipeline stage across one 64-GPU NVLink domain when the two stages receive topology-aligned ranks.
- Returns:
The Super-VL GB200 long-context SFT configuration.
- bridge.recipes.nemotronh_multimodal.gb200.nemotron_35_super_vl.nemotron_35_super_vl_peft_16gpu_gb200_bf16_config() megatron.bridge.training.config.ConfigContainer#
Return the 16-GB200 BF16 LoRA configuration for Super VL.
- Returns:
The Super-VL GB200 PEFT configuration.
- bridge.recipes.nemotronh_multimodal.gb200.nemotron_35_super_vl.__all__#
[‘nemotron_35_super_vl_peft_16gpu_gb200_bf16_config’, ‘nemotron_35_super_vl_pretrain_64gpu_gb200_bf1…