bridge.recipes.inkling.gb200.inkling2#
Bounded GB200 training recipes for Inkling2 Small (594B total, 17B active parameters).
Every recipe keeps each expert-parallel group of 16 or 32 GPUs inside one NVL72
NVLink domain and dispatches the 384 routed experts with HybridEP. Inkling2 is an
MCore HybridModel whose decoder blocks are an attention and a MoE layer (*E);
the Hybrid pattern’s | separators set the pipeline stages. The image encoder
stays frozen on text data.
Pretraining holds the optimizer state of every expert slice on 128 GPUs: two pipeline stages of 26 and 22 decoder blocks, each with two EP32 groups, and full activation recompute. The packed SFT and LoRA recipes train GBS 8 on eight data-parallel replicas, one pack each, so a step is a single pass through the 48 decoder blocks and more pipeline stages only add GPUs. Full SFT therefore also runs on 96 GPUs, the smallest layout that holds its optimizer state, and both full-SFT recipes spend their spare memory on recomputing fewer layers, none at 128 GPUs; LoRA runs the frozen base in a single stage on 32 or 16 GPUs. The long-context recipe splits every 32K pack over four context-parallel ranks. The image-text recipes train on CORD-v2 receipts through the checkpoint’s Inkling2 processor; their full SFT also trains the image encoder.
Each recipe is self-contained: it sets every model, data, and training option it relies on, even where that repeats another recipe.
Module Contents#
Functions#
Return a 100-step bounded text pretraining config on 128 GB200 GPUs. |
|
Return a 100-step full SFT config on packed 8K Tulu 3 with 128 GB200 GPUs. |
|
Return a 100-step full SFT config on packed 8K Tulu 3 with 96 GB200 GPUs. |
|
Return a 20-step full SFT config on packed 32K Tulu 3 with 128 GB200 GPUs. |
|
Return a 100-step LoRA config on packed 8K Tulu 3 with 32 GB200 GPUs. |
|
Return a 100-step LoRA config on packed 8K Tulu 3 with 16 GB200 GPUs. |
|
Return a 100-step full SFT config on image-text CORD-v2 with 128 GB200 GPUs. |
|
Return a 100-step LoRA config on image-text CORD-v2 with 32 GB200 GPUs. |
Data#
API#
- bridge.recipes.inkling.gb200.inkling2._MODEL_ID#
‘thinkingmachines/Inkling-2-Small’
- bridge.recipes.inkling.gb200.inkling2._TULU3_REVISION#
‘b14afda60f1bbebe55d5d2fa1e4df5042f97f8be’
- bridge.recipes.inkling.gb200.inkling2._CORD_V2_REVISION#
‘7f0115a4b758a71d6473b8d085751692da2fef98’
- bridge.recipes.inkling.gb200.inkling2.inkling2_small_pretrain_128gpu_gb200_bf16_config() megatron.bridge.training.config.ConfigContainer#
Return a 100-step bounded text pretraining config on 128 GB200 GPUs.
TP1/PP2/EP32: each pipeline stage is 64 contiguous ranks (16 nodes) in one NVL72 domain and holds two EP32 groups of 32 contiguous ranks (8 nodes), which dispatch the experts with HybridEP and reduce their expert gradients between each other; full activation recompute keeps the AttnRes residual blocks of in-flight microbatches small. The stages hold 26 and 22 decoder blocks: a block’s cost grows with depth, as the AttnRes state it reads grows, and the last stage also computes the output layer and loss. Two stages leave a smaller pipeline bubble than four and split the blocks more evenly, so the step is about 6% faster than on four stages of 13, 13, 12, and 10 blocks; in exchange each GPU holds the weights and gradients of twice as many expert slices, and the first stage peaks near 145 GB. EP64 on the same two stages is over 20% slower. Pass the imported checkpoint with
--pretrained_checkpointfor natural expert routing; the recipe leaves it unset. The peak learning rate is 3e-5 because the run continues a trained model rather than starting from random weights.
- bridge.recipes.inkling.gb200.inkling2.inkling2_small_sft_128gpu_gb200_bf16_config() megatron.bridge.training.config.ConfigContainer#
Return a 100-step full SFT config on packed 8K Tulu 3 with 128 GB200 GPUs.
TP4/PP4/EP32 with sequence parallelism leaves eight data-parallel replicas, one 8K pack each per step (GBS 8), so a step is a single pass through the 48 decoder blocks. That pass’s activations fit without recompute: the last stage peaks near 145 GB, and recomputing the first 6 of each stage’s 24 Hybrid layers makes the step about 13% slower. The 96-GPU recipe trains the same run on fewer GPUs. Conversations are rendered with the checkpoint’s chat template and packs are aligned to the TP4 sequence-parallel width. The final step saves the full model weights for HF export.
- bridge.recipes.inkling.gb200.inkling2.inkling2_small_sft_96gpu_gb200_bf16_config() megatron.bridge.training.config.ConfigContainer#
Return a 100-step full SFT config on packed 8K Tulu 3 with 96 GB200 GPUs.
Trains the same data, batch, and schedule as the 128-GPU recipe on three pipeline stages instead of four. TP4/PP3/EP32 with sequence parallelism leaves eight data-parallel replicas, one 8K pack each per step (GBS 8), so a step is a single pass through the 48 decoder blocks and a fourth stage only adds GPUs. Each stage holds 16 decoder blocks and one EP32 group of 32 contiguous ranks (8 nodes) inside one NVL72 domain. Only the first 20 of each stage’s 32 Hybrid layers recompute their activations, which keeps the last stage near 151 GB; full recompute makes the step about 8% slower. The final step saves the full model weights for HF export.
- bridge.recipes.inkling.gb200.inkling2.inkling2_small_sft_128gpu_gb200_bf16_long_context_config() megatron.bridge.training.config.ConfigContainer#
Return a 20-step full SFT config on packed 32K Tulu 3 with 128 GB200 GPUs.
TP4/PP4/EP32/CP4 with sequence parallelism. Context parallelism leaves two data-parallel replicas, so each replica runs four packs per step and keeps the four pipeline stages busy, where CP 1 would give each replica one pack and run the stages one after another. FlexAttention keeps attention memory linear in the sequence length. Only the first 20 of each stage’s 24 Hybrid layers recompute their activations: the first stage, with four packs in flight, peaks near 120 GB, and full recompute makes the step about 7% slower.
- bridge.recipes.inkling.gb200.inkling2.inkling2_small_peft_32gpu_gb200_bf16_config(
- peft_scheme: str | megatron.bridge.peft.base.PEFT = 'lora',
Return a 100-step LoRA config on packed 8K Tulu 3 with 32 GB200 GPUs.
LoRA rank 8 / alpha 16 adapts the attention Q, K, V, and output projections of the frozen base. The base runs in one pipeline stage at TP4/EP32 with sequence parallelism. The EP32 group is all 32 GPUs (8 nodes) in one NVL72 domain and dispatches the experts with HybridEP. Eight data-parallel replicas take one 8K pack each per step (GBS 8), so a step is a single pass through the 48 decoder blocks. The frozen base keeps activations small enough to skip activation recompute: peak memory stays near 121 GB.
- bridge.recipes.inkling.gb200.inkling2.inkling2_small_peft_16gpu_gb200_bf16_config(
- peft_scheme: str | megatron.bridge.peft.base.PEFT = 'lora',
Return a 100-step LoRA config on packed 8K Tulu 3 with 16 GB200 GPUs.
Trains the same adapters on the same data and schedule as the 32-GPU recipe. The frozen base runs in one pipeline stage at TP2/EP16 with sequence parallelism: the EP16 group is all 16 GPUs in one NVL72 domain, and eight data-parallel replicas take one 8K pack each per step (GBS 8), so a step is a single pass through the 48 decoder blocks with full activation recompute.
- bridge.recipes.inkling.gb200.inkling2.inkling2_small_sft_128gpu_gb200_bf16_multimodal_config() megatron.bridge.training.config.ConfigContainer#
Return a 100-step full SFT config on image-text CORD-v2 with 128 GB200 GPUs.
TP4/PP4/EP32 with sequence parallelism leaves eight data-parallel replicas, eight receipts each per step (GBS 64). Images take one token per 40x40-pixel patch, and a receipt over 3,072 image tokens is encoded at the largest same-aspect size within that budget; the image encoder and its projection train with the language model. The final step saves the full model weights for HF export.
- bridge.recipes.inkling.gb200.inkling2.inkling2_small_peft_32gpu_gb200_bf16_multimodal_config(
- peft_scheme: str | megatron.bridge.peft.base.PEFT = 'lora',
Return a 100-step LoRA config on image-text CORD-v2 with 32 GB200 GPUs.
LoRA rank 8 / alpha 16 adapts the language model’s attention Q, K, V, and output projections while the base model, image encoder included, stays frozen. TP4/PP2/EP16 with sequence parallelism leaves four data-parallel replicas, two receipts each per step (GBS 8), with the same 3,072-token image budget as full SFT. The frozen base needs no activation recompute: peak memory stays near 71 GB.
- bridge.recipes.inkling.gb200.inkling2.__all__#
[‘inkling2_small_peft_16gpu_gb200_bf16_config’, ‘inkling2_small_peft_32gpu_gb200_bf16_config’, ‘inkl…