Step-3.5-Flash#
Step-3.5-Flash is a MoE language model from StepFun. Megatron Bridge supports checkpoint conversion, inference, and continued pretraining through a dedicated Step-3.5 bridge, provider, and recipe.
Verified configurations#
Choose an exact recorded configuration to see its command and expected result. These selectors are generated from the authoritative verification cards and never synthesize combinations.
Run a configuration#
Choose a workflow, precision, and exact recorded combination. The command and expected result update below.
Import · CPU
✓ VerifiedExact command
./scripts/conversion/convert.sh import --executor slurm --device cpu --nodes 1 --hf-model stepfun-ai/Step-3.5-Flash --hf-revision ab446a3de5e171ea341227e24bb1f090e1b771f7 --megatron-path work/model-verification/step35-flash/cpu-megatron --torch-dtype bfloat16 --trust-remote-codeExpected result
CPU import completed all mappings for the immutable 44-shard source, constructed 199,997,706,240 parameters, and created iter_0000000. The checkpoint reloaded successfully for CPU export; its subsequent strict audit covered all 804 source tensors.
Import · GPU
✓ VerifiedExact command
./scripts/conversion/convert.sh import --executor slurm --device gpu --nodes 1 --gpus-per-node 8 --hf-model stepfun-ai/Step-3.5-Flash --hf-revision ab446a3de5e171ea341227e24bb1f090e1b771f7 --megatron-path work/model-verification/step35-flash/imported-megatron --torch-dtype bfloat16 --tp 1 --pp 1 --ep 8 --etp 1 --distributed-timeout-minutes 110 --trust-remote-code --low-memory-saveExpected result
Distributed import completed at TP1/PP1/EP8/ETP1, constructed 33,525,780,480 parameters per rank, and created iter_0000000. The persisted checkpoint then reloaded successfully for both distributed export and checkpoint-backed generation.
Export · CPU
✓ VerifiedExact command
./scripts/conversion/convert.sh export --executor slurm --device cpu --nodes 1 --hf-model stepfun-ai/Step-3.5-Flash --hf-revision ab446a3de5e171ea341227e24bb1f090e1b771f7 --megatron-path work/model-verification/step35-flash/cpu-megatron/iter_0000000 --hf-path work/model-verification/step35-flash/cpu-hf-export --torch-dtype bfloat16 --trust-remote-codeExpected result
CPU export reloaded the imported checkpoint and wrote 44 indexed BF16 shards with all 804 source keys, shapes, and dtypes. Of 398,768,626,944 serialized bytes, 395,600,878,848 bytes across 801 tensors matched the immutable source bitwise. The three published MTP output entries occupy 3,167,748,096 bytes and were intentionally materialized from Megatron's shared lm_head representation; each exported entry matched lm_head bitwise and used independent safetensors storage. A Transformers reload through the pinned checkpoint's metadata/RoPE compatibility view ignored only the three published inference-unused MTP layers, predicted the same next token as the source (' Paris'), and reached cosine similarity 0.997260 with maximum/mean absolute logit differences 1.320312/0.209649.
Export · GPU
✓ VerifiedExact command
./scripts/conversion/convert.sh export --executor slurm --device gpu --nodes 1 --gpus-per-node 8 --hf-model stepfun-ai/Step-3.5-Flash --hf-revision ab446a3de5e171ea341227e24bb1f090e1b771f7 --megatron-path work/model-verification/step35-flash/imported-megatron/iter_0000000 --hf-path work/model-verification/step35-flash/hf-export --torch-dtype bfloat16 --tp 1 --pp 1 --ep 8 --etp 1 --distributed-timeout-minutes 110 --distributed-save --save-every-n-ranks 1 --no-progress --trust-remote-codeExpected result
Distributed export reloaded the TP1/PP1/EP8/ETP1 checkpoint and wrote 44 indexed BF16 shards with all 804 source keys, shapes, and dtypes. The exact audit found 801 tensors and 395,600,878,848 bytes bit-identical to the source. Megatron-Core shares the output head with MTP, so the three source MTP output tensors were explicitly normalized to lm_head.weight; all three exported copies matched that tensor bitwise and had independent storage. The generic grouped-expert export path recovered from CUDA stack pressure by moving only the affected merge to CPU. A Transformers reload through the pinned checkpoint's metadata/RoPE compatibility view ignored only the three published inference-unused MTP layers, predicted the same next token as the source (' Paris'), and reached cosine similarity 0.997260 with maximum/mean absolute logit differences 1.320312/0.209649.
Pretrain · H100
â—‹ UnverifiedRecorded metrics
- Initial loss
- None
- Final loss
- None
- Step time · last 10 avg
- None ms
- Model throughput · last 10 avg
- None TFLOP/s/GPU
- Token throughput · last 10 avg
- None tokens/s/GPU
Exact command
No runnable command is recorded for this status.
Expected result
The public Step-3.5-Flash H100 recipe must complete a bounded 100-step run with finite loss, no skipped or NaN iterations, all five metrics, and a reloadable final checkpoint. Training is deferred by this card.
SFT · H100
â—‹ UnverifiedRecorded metrics
- Initial loss
- None
- Final loss
- None
- Step time · last 10 avg
- None ms
- Model throughput · last 10 avg
- None TFLOP/s/GPU
- Token throughput · last 10 avg
- None tokens/s/GPU
Exact command
No runnable command is recorded for this status.
Expected result
A pinned-data 100-step full-SFT run must finish with finite loss, no skipped or NaN iterations, all five metrics, and a reloadable final checkpoint. Training is deferred by this card.
Long Context · H100
â—‹ UnverifiedRecorded metrics
- Initial loss
- None
- Final loss
- None
- Step time · last 10 avg
- None ms
- Model throughput · last 10 avg
- None TFLOP/s/GPU
- Token throughput · last 10 avg
- None tokens/s/GPU
Exact command
No runnable command is recorded for this status.
Expected result
A dedicated packed long-context SFT run must complete 100 steps with finite loss, no skipped or NaN iterations, all five metrics, and a reloadable checkpoint. Training is deferred by this card.
LoRA · H100
â—‹ UnverifiedRecorded metrics
- Initial loss
- None
- Final loss
- None
- Step time · last 10 avg
- None ms
- Model throughput · last 10 avg
- None TFLOP/s/GPU
- Token throughput · last 10 avg
- None tokens/s/GPU
Exact command
No runnable command is recorded for this status.
Expected result
A Step-3.5-Flash PEFT recipe with an audited adapter target set must complete 100 steps with finite loss, all five metrics, and a reloadable adapter checkpoint. Training is deferred by this card.