Inkling 2#

Inkling 2 Small is Thinking Machines’ image-and-text mixture-of-experts model with 594B total and about 17B active parameters. Its 48 decoder layers replace the residual stream with Block Attention Residuals, interleave 1,024-token sliding-window attention with full attention on every sixth layer, and route each token to 8 of 384 experts. The release stores the routed experts in NVFP4 and MXFP8. Megatron Bridge builds it as a native Megatron-Core HybridModel through the builder-backed Inkling2ModelConfig, converts the complete checkpoint including the image encoder, and keeps the speculative-decoding draft network outside the Megatron model.

Verified configurations#

Choose an exact recorded configuration to see its command and expected result. These selectors are generated from the authoritative verification cards and never synthesize combinations.

Run a configuration#

Choose a workflow, precision, and exact recorded combination. The command and expected result update below.

Import · CPU

â—‹ Unverified
Hardware
not specified
Precision
BF16
Last verified
—
Exact command

No runnable command is recorded for this status.

Expected result

CPU import of the complete 594B checkpoint has not been attempted. GPU import is recorded separately.

Import · GPU

✓ Verified
Hardware
not specified
Precision
BF16
Last verified
2026-11-12
Exact command
Command
./scripts/conversion/convert.sh import --executor slurm --device gpu --nodes 4 --gpus-per-node 4 --hf-model thinkingmachines/Inkling-2-Small --hf-revision 78cb6c0ef68e1f089b77e78cfd94783d05b5c41b --megatron-path work/model-verification/inkling-2-small/gpu-megatron --torch-dtype bfloat16 --tp 1 --pp 1 --ep 16 --etp 1
Expected result

Observed with 16 GB200 GPUs at TP1/PP1/EP16/ETP1: the release loads by its Hub ID, builds the complete HybridModel with the image encoder, and dequantizes the NVFP4 and MXFP8 experts to BF16 one layer at a time, in 200.6 s. The export audit under megatron_to_hf_gpu establishes its tensor content. Caveats: this ran through a private srun wrapper that imports and exports in memory rather than through this launcher, so it persisted no iter_0000000 checkpoint. The training items start from a distributed checkpoint of the same weights that a pre-release tree of this support imported and saved.

Export · CPU

â—‹ Unverified
Hardware
not specified
Precision
BF16
Last verified
—
Exact command

No runnable command is recorded for this status.

Expected result

CPU export of the complete 594B checkpoint has not been attempted. GPU export is recorded separately.

Export · GPU

✓ Verified
Hardware
not specified
Precision
BF16
Last verified
2026-11-12
Exact command
Command
./scripts/conversion/convert.sh export --executor slurm --device gpu --nodes 4 --gpus-per-node 4 --hf-model thinkingmachines/Inkling-2-Small --hf-revision 78cb6c0ef68e1f089b77e78cfd94783d05b5c41b --megatron-path work/model-verification/inkling-2-small/gpu-megatron/iter_0000000 --hf-path work/model-verification/inkling-2-small/gpu-hf-export --torch-dtype bfloat16 --tp 1 --pp 1 --ep 16 --etp 1 --distributed-save
Expected result

Observed with 16 GB200 GPUs at TP1/PP1/EP16/ETP1: the export re-quantizes the routed experts against the source scales, and an exhaustive comparison against the release finds all 1,770 tensors byte-identical, including the NVFP4 and MXFP8 experts and their scales, with no missing or extra keys. The tokenizer files, chat template, processor and quantization configurations, and the MTP draft are byte-identical to the release; config.json keeps every release key and adds dtype, image_token_id, tie_word_embeddings, and transformers_version. Caveats: the export ran in memory from the model imported under hf_to_megatron_gpu, through a private srun wrapper, rather than from a saved iter_0000000 checkpoint through this launcher. Export from a saved checkpoint was exercised by the plain BF16 export of an SFT checkpoint described under sft_export_inference.

Pretrain · GB200

✓ Verified
Hardware
GB200
Precision
BF16
Last verified
2026-11-16
Recorded metrics
Initial loss
1.827291
Final loss
1.745315
Step time · last 10 avg
16,240.320 ms
Model throughput · last 10 avg
225.530 TFLOP/s/GPU
Token throughput · last 10 avg
2,017.694 tokens/s/GPU
Exact command
Command
./scripts/training/train.sh --nodes 32 --gpus-per-node 4 --recipe inkling2_small_pretrain_128gpu_gb200_bf16_config --mode pretrain --pretrained_checkpoint work/model-verification/inkling-2-small/gpu-megatron --max_steps 50 'dataset.blend=[["work/data/rp2/head_01_inkling2_text_document"],null]' checkpoint.save=null checkpoint.load=null checkpoint.async_save=false ddp.check_for_large_grads=true rerun_state_machine.check_for_nan_in_loss=true logger.log_interval=1 logger.log_throughput=true logger.log_device_memory_used=true logger.tensorboard_dir=null logger.save_config_filepath=work/model-verification/inkling-2-small/pretrain-resolved.yaml
Expected result

Observed with 128 GB200 GPUs at TP1/PP2/EP32/ETP1 (pipeline stages of 26 and 22 decoder blocks, each on 16 consecutive nodes of one NVL72 rack holding two EP32 groups; HybridEP dispatch; full activation recompute; compiled Inkling2 operators; Inkling2 attention's FlexAttention backward tiles for Blackwell; expandable segments set in the environment as the example README instructs): the command warm-starts from the imported checkpoint with natural expert routing, keeps the image encoder frozen, and runs the first 50 of the recipe's 100 steps on the RedPajama2 head_01 prefix at GBS 1024 and sequence length 4096, at the recipe's peak LR of 3e-5 after 40 warmup steps (2.82e-5 at step 50). All 50 steps finish with finite losses and 0 skipped or NaN iterations. Caveats: run on 2026-11-16 through a private srun wrapper around scripts/training/run_recipe.py rather than train.sh, with Transformers 5.23.0 installed over the container's copy. Checkpoint saving was off, so the run saved no step-50 or step-100 checkpoint and checkpoint_resume stays unverified. An earlier 100-step run of the same data and schedule on older code and four pipeline stages reached a loss of 1.7301 at step 100.

SFT · GB200

✓ Verified
Hardware
GB200
Precision
BF16
Last verified
2026-11-11
Recorded metrics
Initial loss
1.323619
Final loss
0.6976618
Step time · last 10 avg
1,481.150 ms
Model throughput · last 10 avg
37.740 TFLOP/s/GPU
Token throughput · last 10 avg
345.677 tokens/s/GPU
Exact command
Command
./scripts/training/train.sh --nodes 32 --gpus-per-node 4 --recipe inkling2_small_sft_128gpu_gb200_bf16_config --mode sft --pretrained_checkpoint work/model-verification/inkling-2-small/gpu-megatron --max_steps 30 checkpoint.save=null checkpoint.load=null checkpoint.async_save=false ddp.check_for_large_grads=true rerun_state_machine.check_for_nan_in_loss=true logger.log_interval=1 logger.log_throughput=true logger.log_device_memory_used=true logger.tensorboard_dir=null logger.save_config_filepath=work/model-verification/inkling-2-small/sft-resolved.yaml
Expected result

Observed with 128 GB200 GPUs at TP4/PP4/EP32/ETP1 with sequence parallelism, no activation recompute, HybridEP dispatch, and compiled Inkling2 operators: the command warm-starts from the imported checkpoint and runs the first 30 of the recipe's 100 steps of full SFT on the first 10,000 Tulu 3 conversations, rendered with the release's chat template and packed to 8,192 tokens, at GBS 8 (eight data-parallel replicas with one pack each) and peak LR 5e-6 after 10 warmup steps. All 30 steps finish with finite losses and 0 skipped or NaN iterations. Caveats: run on 2026-11-11 through a private srun wrapper around scripts/training/run_recipe.py rather than train.sh, with checkpoint saving off, on a pre-release tree whose model code matches the recorded commit; its recipe reached the recorded recipe's settings through recompute overrides and rendered the data with a copy of the release's chat template that gives the same tokens and loss masks. A 100-step run of the same data and schedule on older code reached a loss of 0.5446 at step 100; sft_export_inference describes the export of its checkpoint.

Long Context · GB200

✓ Verified
Hardware
GB200
Precision
BF16
Last verified
2026-11-11
Recorded metrics
Initial loss
1.153563
Final loss
0.8579385
Step time · last 10 avg
4,711.280 ms
Model throughput · last 10 avg
47.460 TFLOP/s/GPU
Token throughput · last 10 avg
434.701 tokens/s/GPU
Exact command
Command
./scripts/training/train.sh --nodes 32 --gpus-per-node 4 --recipe inkling2_small_sft_128gpu_gb200_bf16_long_context_config --mode sft --pretrained_checkpoint work/model-verification/inkling-2-small/gpu-megatron --max_steps 20 checkpoint.save=null checkpoint.load=null checkpoint.async_save=false ddp.check_for_large_grads=true rerun_state_machine.check_for_nan_in_loss=true logger.log_interval=1 logger.log_throughput=true logger.log_device_memory_used=true logger.tensorboard_dir=null logger.save_config_filepath=work/model-verification/inkling-2-small/sft-long-context-resolved.yaml
Expected result

Observed with 128 GB200 GPUs at TP4/PP4/EP32/CP4/ETP1 with sequence parallelism, Inkling2's own context parallelism, recompute of the first 20 of 24 layers per stage, HybridEP dispatch, and compiled Inkling2 operators: the command warm-starts from the imported checkpoint and runs all 20 recipe steps of full SFT on Tulu 3 packed to 32,768 tokens at GBS 8 (two data-parallel replicas running four packs each), at peak LR 1e-6 after 2 warmup steps with decay to 0 at step 20. All 20 steps finish with finite losses and 0 skipped or NaN iterations. Caveats: run on 2026-11-11 through a private srun wrapper around scripts/training/run_recipe.py rather than train.sh, on a pre-release tree whose model code matches the recorded commit; its recipe reached the recorded recipe's settings through recompute overrides and rendered the data with a copy of the release's chat template that gives the same tokens and loss masks. The pre-release checkpoint's configuration starts full-attention length scaling at 8,192 tokens instead of this revision's 128,000, so tokens past position 8,192 of a document were scaled where this revision would not scale them.

LoRA · GB200

✓ Verified
Hardware
GB200
Precision
BF16
Last verified
2026-11-11
Recorded metrics
Initial loss
1.324133
Final loss
0.7994129
Step time · last 10 avg
1,427.080 ms
Model throughput · last 10 avg
142.310 TFLOP/s/GPU
Token throughput · last 10 avg
1,435.098 tokens/s/GPU
Exact command
Command
./scripts/training/train.sh --nodes 8 --gpus-per-node 4 --recipe inkling2_small_peft_32gpu_gb200_bf16_config --mode lora --pretrained_checkpoint work/model-verification/inkling-2-small/gpu-megatron --max_steps 30 checkpoint.save=null checkpoint.load=null checkpoint.async_save=false ddp.check_for_large_grads=true rerun_state_machine.check_for_nan_in_loss=true logger.log_interval=1 logger.log_throughput=true logger.log_device_memory_used=true logger.tensorboard_dir=null logger.save_config_filepath=work/model-verification/inkling-2-small/peft-resolved.yaml
Expected result

Observed with 32 GB200 GPUs at TP4/PP1/EP32/ETP1 with sequence parallelism, no activation recompute, HybridEP dispatch, and compiled Inkling2 operators: the command loads the imported checkpoint, freezes the base model, and trains rank-8, alpha-16 LoRA adapters on linear_q, linear_k, linear_v, and linear_proj for the first 30 of the recipe's 100 steps on the packed 8K Tulu 3 data of the sft item at GBS 8 and peak LR 1e-4 after 10 warmup steps. All 30 steps finish with finite losses and 0 skipped or NaN iterations. The TFLOP/s value is Bridge's generic estimate, which counts a full backward pass. Caveats: run on 2026-11-11 through a private srun wrapper around scripts/training/run_recipe.py rather than train.sh, with checkpoint saving off, on a pre-release tree whose model code matches the recorded commit; its recipe reached the recorded recipe's settings through recompute overrides and rendered the data with a copy of the release's chat template that gives the same tokens and loss masks. A 100-step run of the same data and schedule on older code reached a loss of 0.7560 at step 100.