> This page is for version 0.5.0 · 26.06.
> For other versions, use one of these documentation indexes:
> - Nightly (default): https://docs.nvidia.com/nemo/automodel/nightly/llms.txt
> - Latest: https://docs.nvidia.com/nemo/automodel/latest/llms.txt
> - 0.5.0 · 26.06: https://docs.nvidia.com/nemo/automodel/v0.5/llms.txt
> - 0.4.0 · 26.04: https://docs.nvidia.com/nemo/automodel/v0.4/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/automodel/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/automodel/_mcp/server.

# nemo_automodel.components.models.minimax_m3_vl

## Subpackages

* **[`nemo_automodel.components.models.minimax_m3_vl.kernels`](/nemo-automodel/nemo_automodel/components/models/minimax_m3_vl/kernels)**

## Submodules

* **[`nemo_automodel.components.models.minimax_m3_vl.config`](/nemo-automodel/nemo_automodel/components/models/minimax_m3_vl/config)**
* **[`nemo_automodel.components.models.minimax_m3_vl.cp_sparse_attn`](/nemo-automodel/nemo_automodel/components/models/minimax_m3_vl/cp_sparse_attn)**
* **[`nemo_automodel.components.models.minimax_m3_vl.flops`](/nemo-automodel/nemo_automodel/components/models/minimax_m3_vl/flops)**
* **[`nemo_automodel.components.models.minimax_m3_vl.layers`](/nemo-automodel/nemo_automodel/components/models/minimax_m3_vl/layers)**
* **[`nemo_automodel.components.models.minimax_m3_vl.model`](/nemo-automodel/nemo_automodel/components/models/minimax_m3_vl/model)**
* **[`nemo_automodel.components.models.minimax_m3_vl.msa`](/nemo-automodel/nemo_automodel/components/models/minimax_m3_vl/msa)**
* **[`nemo_automodel.components.models.minimax_m3_vl.msa_bindings`](/nemo-automodel/nemo_automodel/components/models/minimax_m3_vl/msa_bindings)**
* **[`nemo_automodel.components.models.minimax_m3_vl.mtp`](/nemo-automodel/nemo_automodel/components/models/minimax_m3_vl/mtp)**
* **[`nemo_automodel.components.models.minimax_m3_vl.processing`](/nemo-automodel/nemo_automodel/components/models/minimax_m3_vl/processing)**
* **[`nemo_automodel.components.models.minimax_m3_vl.recompute`](/nemo-automodel/nemo_automodel/components/models/minimax_m3_vl/recompute)**
* **[`nemo_automodel.components.models.minimax_m3_vl.state_dict_adapter`](/nemo-automodel/nemo_automodel/components/models/minimax_m3_vl/state_dict_adapter)**
* **[`nemo_automodel.components.models.minimax_m3_vl.vision_encoder`](/nemo-automodel/nemo_automodel/components/models/minimax_m3_vl/vision_encoder)**

## Package Contents

### Classes

| Name                                                                                                                                       | Description                                                                     |
| ------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------- |
| [`MiniMaxM3SparseForCausalLM`](#nemo_automodel-components-models-minimax_m3_vl-model-MiniMaxM3SparseForCausalLM)                           | Standalone M3 text backbone for causal LM (Stage 1 parity target).              |
| [`MiniMaxM3SparseForConditionalGeneration`](#nemo_automodel-components-models-minimax_m3_vl-model-MiniMaxM3SparseForConditionalGeneration) | MiniMax M3 VL: CLIP-style vision tower + projector/merger + M3 text backbone.   |
| [`MiniMaxM3VLConfig`](#nemo_automodel-components-models-minimax_m3_vl-config-MiniMaxM3VLConfig)                                            | Top-level configuration for MiniMax M3 vision-language checkpoints.             |
| [`MiniMaxM3VLTextConfig`](#nemo_automodel-components-models-minimax_m3_vl-config-MiniMaxM3VLTextConfig)                                    | Configuration for the MiniMax M3 (mixed sparse/dense MoE) text backbone.        |
| [`MiniMaxM3VLVisionConfig`](#nemo_automodel-components-models-minimax_m3_vl-config-MiniMaxM3VLVisionConfig)                                | Configuration for the MiniMax M3 VL CLIP-style vision tower (Conv3d + 3D RoPE). |

### Data

[`ModelClass`](#nemo_automodel-components-models-minimax_m3_vl-ModelClass)

### API

```python
class nemo_automodel.components.models.minimax_m3_vl.model.MiniMaxM3SparseForCausalLM(
    config: typing.Any,
    moe_config: nemo_automodel.components.moe.layers.MoEConfig | None = None,
    backend: nemo_automodel.components.models.common.BackendConfig | None = None,
    kwargs = {}
)
```

**Bases:** [HFCheckpointingMixin](/nemo-automodel/nemo_automodel/components/models/common/hf_checkpointing_mixin#nemo_automodel-components-models-common-hf_checkpointing_mixin-HFCheckpointingMixin), `Module`, [MoEFSDPSyncMixin](/nemo-automodel/nemo_automodel/components/moe/fsdp_mixin#nemo_automodel-components-moe-fsdp_mixin-MoEFSDPSyncMixin)

Standalone M3 text backbone for causal LM (Stage 1 parity target).

**`_keep_in_fp32_modules_strict`**

---

**`_pp_keep_self_forward`** `bool = True`

---

**`backend`**

---

**`consumes_packed_seq_ids`** `bool`

Whether the packed loader should hand this model `_packed_seq_ids` (the compact document map)
for every pack, single-document ones included, instead of a `[batch, 1, sequence, sequence]` mask.

---

**`lm_head`**

---

**`model`**

---

**`state_dict_adapter`**

---

**`tie_word_embeddings_support`** `TieSupport = TieSupport.UNTIED_ONLY`

---

```python
nemo_automodel.components.models.minimax_m3_vl.model.MiniMaxM3SparseForCausalLM.forward(
    input_ids: torch.Tensor,
    position_ids: torch.Tensor | None = None,
    attention_mask: torch.Tensor | None = None,
    padding_mask: torch.Tensor | None = None,
    attn_kwargs: typing.Any = {}
) -> torch.Tensor | nemo_automodel.components.models.minimax_m3_vl.model.MiniMaxM3CausalLMOutput
```

Map ids/positions/padding\[B,S] and mask\[B,S] or \[B,1,S,S] to logits\[B,S,V] or MTP logits; MSA uses BSHD.

```python
nemo_automodel.components.models.minimax_m3_vl.model.MiniMaxM3SparseForCausalLM.from_config(
    config: typing.Any,
    moe_config: nemo_automodel.components.moe.layers.MoEConfig | None = None,
    backend: nemo_automodel.components.models.common.BackendConfig | None = None,
    kwargs = {}
)
```

classmethod

```python
nemo_automodel.components.models.minimax_m3_vl.model.MiniMaxM3SparseForCausalLM.from_pretrained(
    pretrained_model_name_or_path: str,
    model_args = (),
    kwargs = {}
)
```

classmethod

```python
nemo_automodel.components.models.minimax_m3_vl.model.MiniMaxM3SparseForCausalLM.get_input_embeddings()
```

```python
nemo_automodel.components.models.minimax_m3_vl.model.MiniMaxM3SparseForCausalLM.get_output_embeddings()
```

```python
nemo_automodel.components.models.minimax_m3_vl.model.MiniMaxM3SparseForCausalLM.initialize_weights(
    buffer_device: torch.device | None = None,
    dtype: torch.dtype = torch.bfloat16
) -> None
```

```python
nemo_automodel.components.models.minimax_m3_vl.model.MiniMaxM3SparseForCausalLM.set_input_embeddings(
    value
)
```

```python
nemo_automodel.components.models.minimax_m3_vl.model.MiniMaxM3SparseForCausalLM.set_output_embeddings(
    new_embeddings
)
```

```python
class nemo_automodel.components.models.minimax_m3_vl.model.MiniMaxM3SparseForConditionalGeneration(
    config: typing.Any,
    moe_config: nemo_automodel.components.moe.layers.MoEConfig | None = None,
    backend: nemo_automodel.components.models.common.BackendConfig | None = None,
    kwargs = {}
)
```

**Bases:** [HFCheckpointingMixin](/nemo-automodel/nemo_automodel/components/models/common/hf_checkpointing_mixin#nemo_automodel-components-models-common-hf_checkpointing_mixin-HFCheckpointingMixin), `Module`, [MoEFSDPSyncMixin](/nemo-automodel/nemo_automodel/components/moe/fsdp_mixin#nemo_automodel-components-moe-fsdp_mixin-MoEFSDPSyncMixin)

MiniMax M3 VL: CLIP-style vision tower + projector/merger + M3 text backbone.

Vision features (`vision_tower(pixel_values, grid_thw)`) are spliced into
the text embeddings at `image_token_index` / `video_token_index`
positions, then run through the (sparse/dense MoE) language model + lm\_head.

**`_keep_in_fp32_modules`** `= ['rotary_emb', 'inv_freq']`

---

**`_keep_in_fp32_modules_strict`**

---

**`_pp_keep_self_forward`** `bool = True`

---

**`backend`**

---

**`consumes_packed_seq_ids`** `bool`

Whether the packed loader should hand this model `_packed_seq_ids` (the compact document map)
for every pack, single-document ones included, instead of a `[batch, 1, sequence, sequence]` mask.

---

**`image_token_index`** `= config.image_token_index`

---

**`lm_head`**

---

**`model`**

---

**`state_dict_adapter`**

---

**`tie_word_embeddings_support`** `TieSupport = TieSupport.UNTIED_ONLY`

---

**`video_token_index`** `= config.video_token_index`

---

**`vision_tower`**

---

**`vocab_size`** `= text_config.vocab_size`

---

```python
nemo_automodel.components.models.minimax_m3_vl.model.MiniMaxM3SparseForConditionalGeneration._embed_and_splice(
    input_ids: torch.Tensor,
    pixel_values: torch.Tensor | None = None,
    image_grid_thw = None,
    pixel_values_videos: torch.Tensor | None = None,
    video_grid_thw = None
) -> torch.Tensor
```

Embed token ids and splice vision/video features into the embeddings.

Shared by `forward` and `prepare_model_inputs_for_cp` so the splice logic
lives in one place.

```python
nemo_automodel.components.models.minimax_m3_vl.model.MiniMaxM3SparseForConditionalGeneration._is_pipeline_parallel_stage() -> bool
```

True when this is a partial pipeline stage (some text modules nulled).

```python
nemo_automodel.components.models.minimax_m3_vl.model.MiniMaxM3SparseForConditionalGeneration._splice_multimodal(
    inputs_embeds: torch.Tensor,
    input_ids: torch.Tensor,
    pixel_values: torch.Tensor,
    grid_thw,
    token_index: int
) -> torch.Tensor
```

```python
nemo_automodel.components.models.minimax_m3_vl.model.MiniMaxM3SparseForConditionalGeneration._to_grid_list(
    grid_thw
) -> list[list[int]]
```

staticmethod

```python
nemo_automodel.components.models.minimax_m3_vl.model.MiniMaxM3SparseForConditionalGeneration.customize_pipeline_stage_modules(
    module_names_per_stage: list[list[str]],
    layers_prefix: str,
    text_model: torch.nn.Module | None = None
) -> list[list[str]]
```

Rewrite auto-generated pipeline FQNs to M3's real module paths.

M3's text stack lives directly under `self.model` and the vision tower
is a top-level sibling (`vision_tower`). The framework, seeing the
`language_model` property, derives a nested `model.language_model.`
prefix for the text modules and a `model.` prefix for the multimodal
encoders. Map both back to M3's actual paths so per-stage module nulling
keeps/drops the correct submodules.

```python
nemo_automodel.components.models.minimax_m3_vl.model.MiniMaxM3SparseForConditionalGeneration.forward(
    input_ids: torch.Tensor | None = None,
    pixel_values: torch.Tensor | None = None,
    image_grid_thw = None,
    pixel_values_videos: torch.Tensor | None = None,
    video_grid_thw = None,
    position_ids: torch.Tensor | None = None,
    attention_mask: torch.Tensor | None = None,
    inputs_embeds: torch.Tensor | None = None,
    logits_to_keep: int | None = None,
    kwargs: typing.Any = {}
) -> torch.Tensor | nemo_automodel.components.models.minimax_m3_vl.model.MiniMaxM3CausalLMOutput | dict[str, torch.Tensor]
```

```python
nemo_automodel.components.models.minimax_m3_vl.model.MiniMaxM3SparseForConditionalGeneration.from_config(
    config: typing.Any,
    moe_config: nemo_automodel.components.moe.layers.MoEConfig | None = None,
    backend: nemo_automodel.components.models.common.BackendConfig | None = None,
    kwargs = {}
)
```

classmethod

```python
nemo_automodel.components.models.minimax_m3_vl.model.MiniMaxM3SparseForConditionalGeneration.from_pretrained(
    pretrained_model_name_or_path: str,
    model_args = (),
    kwargs = {}
)
```

classmethod

```python
nemo_automodel.components.models.minimax_m3_vl.model.MiniMaxM3SparseForConditionalGeneration.get_input_embeddings()
```

```python
nemo_automodel.components.models.minimax_m3_vl.model.MiniMaxM3SparseForConditionalGeneration.get_output_embeddings()
```

```python
nemo_automodel.components.models.minimax_m3_vl.model.MiniMaxM3SparseForConditionalGeneration.get_pipeline_stage_metas(
    is_first: bool,
    microbatch_size: int,
    seq_len: int,
    dtype: torch.dtype
) -> tuple[tuple[torch.Tensor, ...], tuple[torch.Tensor, ...]]
```

Per-stage input/output meta tensors for the PP schedule's shape inference.

First stage consumes the FULL token ids `[mb, seq]`; later stages
consume hidden states. The final stage (owning `lm_head`) emits logits;
earlier stages emit hidden states.

Under context parallelism the first stage embeds the full sequence and
shards it to this rank's round-robin chunk pair inside forward
(see `shard_sequence_for_cp_round_robin`), so every stage output and every
later-stage input carries the LOCAL (padded-to-`2*cp` then `//cp`)
sequence length while the first stage's input stays full-length. At
`cp_size == 1` the lengths coincide and the layout is symmetric.

```python
nemo_automodel.components.models.minimax_m3_vl.model.MiniMaxM3SparseForConditionalGeneration.initialize_weights(
    buffer_device: torch.device | None = None,
    dtype: torch.dtype = torch.bfloat16
) -> None
```

```python
nemo_automodel.components.models.minimax_m3_vl.model.MiniMaxM3SparseForConditionalGeneration.prepare_model_inputs_for_cp(
    batch: dict[str, typing.Any],
    num_chunks: int = 1
) -> dict[str, typing.Any]
```

Return a sharder-only CP backend; embed + splice + shard happen in forward.

The returned `ContextParallelSharder` round-robin-shards only the
no-grad aux streams (labels/position\_ids/loss\_mask/padding\_mask) via
`shard_batch_aux_only`, leaving `input_ids` and the multimodal inputs
full-length; the forward then embeds + splices and calls
`shard_sequence_for_cp_round_robin` per microbatch, so embeddings and vision stay
trainable under CP. Nothing is consumed here.
Defining this method is the opt-in signal the recipe checks
(`hasattr(model, "prepare_model_inputs_for_cp")`).

**Parameters:**

**`batch`** `dict[str, Any]`

The full-sequence batch; left intact (nothing consumed).

---

**`num_chunks`** `int` — default: 1

Accepted for hook-signature parity; unused (round-robin CP).

---

```python
nemo_automodel.components.models.minimax_m3_vl.model.MiniMaxM3SparseForConditionalGeneration.set_input_embeddings(
    value
)
```

```python
class nemo_automodel.components.models.minimax_m3_vl.config.MiniMaxM3VLConfig(
    vision_config: typing.Union[dict, nemo_automodel.components.models.minimax_m3_vl.config.MiniMaxM3VLVisionConfig] | None = None,
    text_config: typing.Union[dict, nemo_automodel.components.models.minimax_m3_vl.config.MiniMaxM3VLTextConfig] | None = None,
    image_token_index: int = 200025,
    video_token_index: int = 200026,
    image_seq_length: int = 576,
    process_image_mode: str = 'dynamic_res',
    projector_hidden_act: str = 'gelu',
    projector_hidden_size: int = 6144,
    multimodal_projector_bias: bool = True,
    patch_merge_bias: bool = True,
    vision_feature_layer: int = -1,
    vision_feature_select_strategy: str = 'full',
    img_token_compression_config: dict | None = None,
    image_grid_pinpoints: str | None = None,
    kwargs = {}
)
```

**Bases:** `PretrainedConfig`

Top-level configuration for MiniMax M3 vision-language checkpoints.

**`hidden_size`** `= text_config.hidden_size`

---

**`img_token_compression_config`** `= img_token_compression_config or {}`

---

**`max_position_embeddings`** `= text_config.max_position_embeddings`

---

**`model_type`** `= 'minimax_m3_vl'`

---

**`sub_configs`**

---

```python
nemo_automodel.components.models.minimax_m3_vl.config.MiniMaxM3VLConfig.to_dict()
```

```python
class nemo_automodel.components.models.minimax_m3_vl.config.MiniMaxM3VLTextConfig(
    hidden_size: int = 6144,
    intermediate_size: int = 3072,
    dense_intermediate_size: int = 12288,
    shared_intermediate_size: int = 3072,
    num_hidden_layers: int = 60,
    num_attention_heads: int = 64,
    num_key_value_heads: int = 4,
    head_dim: int = 128,
    vocab_size: int = 200064,
    max_position_embeddings: int = 524288,
    rms_norm_eps: float = 1e-06,
    use_gemma_norm: bool = True,
    attention_output_gate: bool = False,
    rope_theta: float = 5000000.0,
    rotary_dim: int = 64,
    partial_rotary_factor: float = 0.5,
    hidden_act: str = 'swigluoai',
    use_qk_norm: bool = True,
    qk_norm_type: str = 'per_head',
    tie_word_embeddings: bool = False,
    num_local_experts: int = 128,
    num_experts_per_tok: int = 4,
    n_shared_experts: int = 1,
    scoring_func: str = 'sigmoid',
    use_routing_bias: bool = True,
    routed_scaling_factor: float = 2.0,
    moe_layer_freq: list[int] | None = None,
    swiglu_alpha: float = 1.702,
    swiglu_limit: float = 7.0,
    sparse_attention_config: dict | None = None,
    num_mtp_modules: int = 1,
    pad_token_id: int | None = None,
    kwargs = {}
)
```

**Bases:** `PretrainedConfig`

Configuration for the MiniMax M3 (mixed sparse/dense MoE) text backbone.

**`architectures`** `= ['MiniMaxM3SparseForCausalLM']`

---

**`model_type`** `= 'minimax_m3'`

---

```python
nemo_automodel.components.models.minimax_m3_vl.config.MiniMaxM3VLTextConfig.to_dict()
```

```python
class nemo_automodel.components.models.minimax_m3_vl.config.MiniMaxM3VLVisionConfig(
    hidden_size: int = 1280,
    num_attention_heads: int = 16,
    num_hidden_layers: int = 32,
    intermediate_size: int = 5120,
    patch_size: int = 14,
    image_size: int = 672,
    projection_dim: int = 6144,
    num_channels: int = 3,
    position_embedding_type: str = 'rope',
    rope_mode: str = '3d',
    rope_theta: float = 10000.0,
    attention_dropout: float = 0.0,
    hidden_act: str = 'gelu',
    layer_norm_eps: float = 1e-05,
    img_token_compression_config: dict | None = None,
    vision_segment_max_frames: int = 4,
    kwargs = {}
)
```

**Bases:** `PretrainedConfig`

Configuration for the MiniMax M3 VL CLIP-style vision tower (Conv3d + 3D RoPE).

**`img_token_compression_config`**

---

**`model_type`** `= 'minimax_m3_vision'`

---

```python
nemo_automodel.components.models.minimax_m3_vl.config.MiniMaxM3VLVisionConfig.to_dict()
```

```python
nemo_automodel.components.models.minimax_m3_vl.ModelClass = MiniMaxM3SparseForConditionalGeneration
```