MiniMax-M3#
MiniMax-M3 is a natively multimodal sparse MoE model from MiniMaxAI (428B total, ~23B active parameters). MiniMaxM3Bridge converts the vision tower, both multimodal projector stages, and sparse-MoE text backbone into a MiniMaxM3VLModel.
Verified configurations#
Choose an exact recorded configuration to see its command and expected result. These selectors are generated from the authoritative verification cards and never synthesize combinations.
Run a configuration#
Choose a workflow, precision, and exact recorded combination. The command and expected result update below.
Import · CPU
â—‹ UnverifiedExact command
No runnable command is recorded for this status.
Expected result
No CPU HF-to-Megatron result is claimed. The CPU-only attempt enabled CPU weight initialization, but the current distributed-model initialization path still selected a CUDA device before tensor loading. With no GPU available, it exited without persisting iter_0000000, so no checkpoint reload or exact tensor audit completed. CPU conversion remains unverified rather than unsupported.
Import · GPU
✓ VerifiedExact command
./scripts/conversion/convert.sh import --executor slurm --device gpu --nodes 4 --gpus-per-node 8 --hf-model MiniMaxAI/MiniMax-M3 --hf-revision 50942730318c7943fe83db7ec8e9f9177ecb1cf8 --megatron-path work/model-verification/minimax-m3/imported-megatron --torch-dtype bfloat16 --tp 1 --pp 1 --ep 32 --etp 1 --distributed-timeout-minutes 180 --trust-remote-codeExpected result
The pinned 32-H100 full-VLM import exited successfully, persisted iter_0000000, and reloaded it at TP1/PP1/EP32/ETP1. A strict projection of the reloaded Megatron model covered exactly 23,416 of 23,416 published language, vision, projector, and lightning-indexer tensors and 854,172,958,720 actual tensor-payload bytes, with zero duplicate, missing, unexpected, shape, invalid-dtype, or value mismatches. The source index's declared total_size is 869,157,697,024 bytes. No dtype widening was required. The audit covered 1,804 total Megatron state tensors; model-state structure matched across all 32 EP ranks, and all 1,348 replicated tensors comprising 13,917,973,376 elements matched exactly.
Export · CPU
✓ VerifiedExact command
./scripts/conversion/convert.sh export --executor slurm --device cpu --nodes 1 --hf-model MiniMaxAI/MiniMax-M3 --hf-revision 50942730318c7943fe83db7ec8e9f9177ecb1cf8 --megatron-path work/model-verification/minimax-m3/imported-megatron/iter_0000000 --hf-path work/model-verification/minimax-m3/cpu-hf-export --torch-dtype bfloat16 --no-progress --trust-remote-codeExpected result
The single-node CPU export exited successfully and wrote the complete indexed MiniMax-M3 VLM checkpoint. An exhaustive audit matched all 23,416 source and export tensors, 427,040,140,160 values, and 854,172,958,720 tensor-payload bytes with zero missing, unexpected, shape, dtype, or value mismatches and no dtype conversions (atol=0, rtol=0). A meta-device Transformers 5.14.0 reload instantiated MiniMaxM3SparseForConditionalGeneration with 1,582 state entries and zero missing, unexpected, mismatched, or error keys.
Export · GPU
✓ VerifiedExact command
./scripts/conversion/convert.sh export --executor slurm --device gpu --nodes 4 --gpus-per-node 8 --hf-model MiniMaxAI/MiniMax-M3 --hf-revision 50942730318c7943fe83db7ec8e9f9177ecb1cf8 --megatron-path work/model-verification/minimax-m3/imported-megatron/iter_0000000 --hf-path work/model-verification/minimax-m3/hf-export --torch-dtype bfloat16 --tp 1 --pp 1 --ep 32 --etp 1 --distributed-timeout-minutes 180 --distributed-save --save-every-n-ranks 1 --no-progress --trust-remote-codeExpected result
The 32-H100 distributed export exited successfully and wrote an indexed full-VLM checkpoint at TP1/PP1/EP32/ETP1. Its key-to-shard map and all 59 safetensors shards are byte-for-byte identical to the pinned source, covering all 23,416 tensors and 854,172,958,720 actual tensor-payload bytes. The exported configuration preserves 60 text layers, 128 top-4 experts, 32 vision layers, both projector stages, and 57 lightning-indexer layers. Three tokenizer probes and multimodal processor inputs match the source. Transformers 5.14.0 natively reloads both source and export as MiniMaxM3SparseForConditionalGeneration through its checkpoint conversion mapping with zero missing, unexpected, mismatched, or error keys. The export consolidates the tokenizer in tokenizer.json and does not reproduce four redundant source-side legacy tokenizer files.
Pretrain · H100
â—‹ UnverifiedRecorded metrics
- Initial loss
- None
- Final loss
- None
- Step time · last 10 avg
- None ms
- Model throughput · last 10 avg
- None TFLOP/s/GPU
- Token throughput · last 10 avg
- None tokens/s/GPU
Exact command
No runnable command is recorded for this status.
Expected result
Complete a 100-step HybridEP language-backbone run with finite loss, reported metrics, and a reloadable checkpoint.
SFT · H100
â—‹ UnverifiedRecorded metrics
- Initial loss
- None
- Final loss
- None
- Step time · last 10 avg
- None ms
- Model throughput · last 10 avg
- None TFLOP/s/GPU
- Token throughput · last 10 avg
- None tokens/s/GPU
Exact command
No runnable command is recorded for this status.
Expected result
Complete a 100-step packed HybridEP language-backbone SFT run with finite loss, reported metrics, and a reloadable checkpoint.
Long Context · H100
â—‹ UnverifiedRecorded metrics
- Initial loss
- None
- Final loss
- None
- Step time · last 10 avg
- None ms
- Model throughput · last 10 avg
- None TFLOP/s/GPU
- Token throughput · last 10 avg
- None tokens/s/GPU
Exact command
No runnable command is recorded for this status.
Expected result
A dedicated packed long-context recipe must complete 100 steps with context parallelism, finite loss, no skipped or NaN iterations, and all five metrics. Beyond 2,048 tokens, Bridge uses full causal attention instead of MiniMax-M3's lightning-indexer sparse attention.
LoRA · H100
â—‹ UnverifiedRecorded metrics
- Initial loss
- None
- Final loss
- None
- Step time · last 10 avg
- None ms
- Model throughput · last 10 avg
- None TFLOP/s/GPU
- Token throughput · last 10 avg
- None tokens/s/GPU
Exact command
No runnable command is recorded for this status.
Expected result
A MiniMax-M3 PEFT recipe with an audited adapter target set must complete 100 steps with finite loss, no skipped or NaN iterations, all four metrics, and a reloadable adapter checkpoint.