bridge.models.minimax_m3.minimax_m3_bridge

Module Contents

Classes

MiniMaxM3TopKRouter	MiniMax-M3 router that computes its projection in the weight dtype.
MiniMaxM3ModelProvider	GPT provider that preserves MiniMax-M3’s FP32 router parameters.
MiniMaxM3VLModelProvider	Provider for the complete MiniMax-M3 vision-language model.
MiniMaxM3Bridge	Megatron Bridge for the complete MiniMax-M3 vision-language model.

Functions

minimax_m3_block_spec	Build a GPT block spec that uses MiniMax-M3’s FP32 router projection.
_promote_router_weights_to_float32	Keep MiniMax-M3 router parameters in FP32 for every load path.
_config_value	Read a value from either a Hugging Face config object or a dictionary.
_config_to_dict	Return a detached dictionary representation of an HF config.

API

class bridge.models.minimax_m3.minimax_m3_bridge.MiniMaxM3TopKRouter

Bases: megatron.core.transformer.moe.router.TopKRouter

MiniMax-M3 router that computes its projection in the weight dtype.

gating(input: torch.Tensor) → torch.Tensor

Match HF by widening router inputs to the FP32 router weight dtype.

bridge.models.minimax_m3.minimax_m3_bridge.minimax_m3_block_spec(

config: megatron.core.transformer.transformer_config.TransformerConfig,

use_transformer_engine: bool = True,

normalization: str | None = None,

qk_l2_norm: bool | None = False,

vp_stage: int | None = None,

pp_rank: int | None = None,

**kwargs: object,

) → megatron.core.transformer.transformer_block.TransformerBlockSubmodules

Build a GPT block spec that uses MiniMax-M3’s FP32 router projection.

bridge.models.minimax_m3.minimax_m3_bridge._promote_router_weights_to_float32(

model: list[torch.nn.Module],

) → list[torch.nn.Module]

Keep MiniMax-M3 router parameters in FP32 for every load path.

Megatron initializes router parameters in params_dtype even when moe_router_dtype="fp32". Promoting them immediately after construction prevents truncation when loading either HF weights or a native Megatron checkpoint.

class bridge.models.minimax_m3.minimax_m3_bridge.MiniMaxM3ModelProvider

Bases: megatron.bridge.models.gpt_provider.GPTModelProvider

GPT provider that preserves MiniMax-M3’s FP32 router parameters.

__post_init__() → None

Install MiniMax-M3 router behavior on fresh and deserialized providers.

bridge.models.minimax_m3.minimax_m3_bridge._config_value(

config: Any,

name: str,

default: Any = None,

) → Any

Read a value from either a Hugging Face config object or a dictionary.

bridge.models.minimax_m3.minimax_m3_bridge._config_to_dict(config: Any) → dict[str, Any]

Return a detached dictionary representation of an HF config.

class bridge.models.minimax_m3.minimax_m3_bridge.MiniMaxM3VLModelProvider

Bases: bridge.models.minimax_m3.minimax_m3_bridge.MiniMaxM3ModelProvider

Provider for the complete MiniMax-M3 vision-language model.

scatter_embedding_sequence_parallel: bool

False

vision_config: Any

None

hf_config_dict: dict[str, Any]

‘field(…)’

image_token_id: int

200025

video_token_id: int

200026

projector_hidden_size: int

6144

multimodal_projector_bias: bool

True

spatial_merge_size: int

2

temporal_patch_size: int

2

lightning_indexer_layers: list[int]

‘field(…)’

index_n_heads: int

4

index_head_dim: int

128

freeze_language_model: bool

False

freeze_vision_model: bool

False

freeze_vision_projection: bool

False

property special_token_ids: dict[str, int]

Return the token IDs used by multimodal data pipelines.

provide(

pre_process=None,

post_process=None,

vp_stage=None,

) → megatron.bridge.models.minimax_m3.modeling_minimax_m3_vl.MiniMaxM3VLModel

Construct the complete MiniMax-M3 VLM.

provide_language_model(

pre_process=None,

post_process=None,

vp_stage=None,

) → megatron.core.models.gpt.gpt_model.GPTModel

Construct only the Megatron text component used inside the VLM.

to_text_provider() → bridge.models.minimax_m3.minimax_m3_bridge.MiniMaxM3ModelProvider

Return the equivalent text-only provider.

This preserves the lightweight model shape and native checkpoint keys used by the existing text pretraining and SFT recipes.

class bridge.models.minimax_m3.minimax_m3_bridge.MiniMaxM3Bridge

Bases: megatron.bridge.models.conversion.model_bridge.MegatronModelBridge

Megatron Bridge for the complete MiniMax-M3 vision-language model.

MiniMax-M3 ships as a natively multimodal checkpoint (MiniMaxM3SparseForConditionalGeneration): a CLIP-style vision tower plus a sparse-MoE text backbone. This bridge converts the vision tower, both multimodal projector MLPs, and the language model into a MiniMaxM3VLModel. The vision stack is replicated across TP ranks while the language model retains Megatron TP/PP/EP sharding.

Text backbone architecture: - Mixed dense/MoE decoder: the first layers are dense (dense_intermediate_size MLP), the rest use 128 routed experts (top-4) plus one shared expert. - Sigmoid router scoring with expert-bias correction and routed_scaling_factor applied to the normalized top-k weights (same routing math as DeepSeek-V3). - SwiGLU-OAI expert/MLP activation: clamped gate/up projections with a +1 linear offset (same as GPT-OSS), expressed via activation_func_clamp_value and glu_linear_offset. - Gemma-style RMSNorm (x * (1 + w)) on every norm, expressed via layernorm_zero_centered_gamma. - GQA attention with per-head QK RMSNorm and partial RoPE (rotary_dim of head_dim channels rotated).

Known limitations: - The lightning-indexer block-sparse attention branch (self_attn.index_{q,k}_{proj,norm}) is stored as frozen model state but is not executed; Megatron runs full causal attention on every layer. Selection happens at index_block_size granularity with index_topk_blocks kept per query, so full attention is mathematically identical for sequences up to index_topk_blocks * index_block_size tokens (2048 for the released checkpoint) and an approximation beyond that. - MTP (Multi-Token Prediction) modules are not mapped. The released checkpoint advertises num_nextn_predict_layers in its config but ships no mtp.* weights, so mtp_num_layers is forced to None. - The vision forward matches the native Transformers implementation, which concatenates multiple image/video patch grids into one bidirectional attention sequence. Segmented multi-image/video attention remains future work.

.. rubric:: Example

from megatron.bridge import AutoBridge bridge = AutoBridge.from_hf_pretrained(“MiniMaxAI/MiniMax-M3”, trust_remote_code=True) provider = bridge.to_megatron_provider()

classmethod hf_to_megatron_activation(hidden_act: str)

Convert HF activation name to Megatron activation function.

The released MiniMax-M3 checkpoint declares hidden_act="swigluoai", which is not a standard ACT2FN key (transformers normalizes it to silu and computes the gate inline from swiglu_alpha / swiglu_limit). Map it to quick_gelu — the SwiGLU-OAI gate is gate * sigmoid(1.702 * gate), i.e. exactly quick-GELU; the clamp and +1 offset are carried by separate provider fields.

provider_bridge(

hf_pretrained: megatron.bridge.models.hf_pretrained.causal_lm.PreTrainedCausalLM,

) → bridge.models.minimax_m3.minimax_m3_bridge.MiniMaxM3VLModelProvider

Convert the Hugging Face MiniMax-M3 config to a full VLM provider.

classmethod megatron_to_hf_config(

provider: megatron.bridge.models.gpt_provider.GPTModelProvider,

) → dict[str, Any]

Build the nested MiniMax-M3 VLM config used for Hugging Face export.

mapping_registry() → megatron.bridge.models.conversion.mapping_registry.MegatronMappingRegistry

Return parameter mappings for the MiniMax-M3 VLM.

Text mappings target the nested language_model. Vision and projector modules intentionally preserve the legacy checkpoint names, so their mappings are replicated identity mappings.