> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/automodel/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/automodel/_mcp/server.

# nemo_automodel.components.models.mistral3.model

## Module Contents

### Classes

| Name                                                                                                                    | Description                                |
| ----------------------------------------------------------------------------------------------------------------------- | ------------------------------------------ |
| [`GradientCheckpointingLayer`](#nemo_automodel-components-models-mistral3-model-GradientCheckpointingLayer)             | -                                          |
| [`Ministral3Attention`](#nemo_automodel-components-models-mistral3-model-Ministral3Attention)                           | -                                          |
| [`Ministral3CausalLMOutputWithPast`](#nemo_automodel-components-models-mistral3-model-Ministral3CausalLMOutputWithPast) | -                                          |
| [`Ministral3Config`](#nemo_automodel-components-models-mistral3-model-Ministral3Config)                                 | Configuration for Ministral3 text decoder. |
| [`Ministral3DecoderLayer`](#nemo_automodel-components-models-mistral3-model-Ministral3DecoderLayer)                     | -                                          |
| [`Ministral3ForCausalLM`](#nemo_automodel-components-models-mistral3-model-Ministral3ForCausalLM)                       | -                                          |
| [`Ministral3MLP`](#nemo_automodel-components-models-mistral3-model-Ministral3MLP)                                       | -                                          |
| [`Ministral3Model`](#nemo_automodel-components-models-mistral3-model-Ministral3Model)                                   | -                                          |
| [`Ministral3ModelOutputWithPast`](#nemo_automodel-components-models-mistral3-model-Ministral3ModelOutputWithPast)       | -                                          |
| [`Ministral3PreTrainedModel`](#nemo_automodel-components-models-mistral3-model-Ministral3PreTrainedModel)               | -                                          |
| [`Ministral3RMSNorm`](#nemo_automodel-components-models-mistral3-model-Ministral3RMSNorm)                               | -                                          |
| [`Ministral3RotaryEmbedding`](#nemo_automodel-components-models-mistral3-model-Ministral3RotaryEmbedding)               | -                                          |

### Functions

| Name                                                                                                                                | Description                                                          |
| ----------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------- |
| [`_get_llama_4_attn_scale`](#nemo_automodel-components-models-mistral3-model-_get_llama_4_attn_scale)                               | -                                                                    |
| [`_register_ministral3_with_transformers`](#nemo_automodel-components-models-mistral3-model-_register_ministral3_with_transformers) | Register Ministral3Config and models with transformers Auto classes. |
| [`apply_rotary_pos_emb`](#nemo_automodel-components-models-mistral3-model-apply_rotary_pos_emb)                                     | -                                                                    |
| [`eager_attention_forward`](#nemo_automodel-components-models-mistral3-model-eager_attention_forward)                               | -                                                                    |
| [`repeat_kv`](#nemo_automodel-components-models-mistral3-model-repeat_kv)                                                           | -                                                                    |
| [`rotate_half`](#nemo_automodel-components-models-mistral3-model-rotate_half)                                                       | -                                                                    |

### Data

[`ModelClass`](#nemo_automodel-components-models-mistral3-model-ModelClass)

[`logger`](#nemo_automodel-components-models-mistral3-model-logger)

### API

```python
class nemo_automodel.components.models.mistral3.model.GradientCheckpointingLayer()
```

**Bases:** `Module`

```python
nemo_automodel.components.models.mistral3.model.GradientCheckpointingLayer.forward(
    args = (),
    kwargs = {}
)
```

```python
class nemo_automodel.components.models.mistral3.model.Ministral3Attention(
    config: nemo_automodel.components.models.mistral3.model.Ministral3Config,
    layer_idx: int
)
```

**Bases:** `Module`

```python
nemo_automodel.components.models.mistral3.model.Ministral3Attention.forward(
    hidden_states: torch.Tensor,
    position_embeddings: tuple[torch.Tensor, torch.Tensor],
    attention_mask: typing.Optional[torch.Tensor],
    past_key_values: typing.Optional[transformers.cache_utils.Cache] = None,
    cache_position: typing.Optional[torch.LongTensor] = None,
    kwargs: transformers.processing_utils.Unpack[transformers.modeling_flash_attention_utils.FlashAttentionKwargs] = {}
) -> tuple[torch.Tensor, typing.Optional[torch.Tensor]]
```

```python
class nemo_automodel.components.models.mistral3.model.Ministral3CausalLMOutputWithPast()
```

Dataclass

**Bases:** `CausalLMOutputWithPast`

```python
class nemo_automodel.components.models.mistral3.model.Ministral3Config(
    vocab_size: typing.Optional[int] = 131072,
    hidden_size: typing.Optional[int] = 4096,
    intermediate_size: typing.Optional[int] = 14336,
    num_hidden_layers: typing.Optional[int] = 34,
    num_attention_heads: typing.Optional[int] = 32,
    num_key_value_heads: typing.Optional[int] = 8,
    head_dim: typing.Optional[int] = 128,
    hidden_act: typing.Optional[str] = 'silu',
    max_position_embeddings: typing.Optional[int] = 262144,
    initializer_range: typing.Optional[float] = 0.02,
    rms_norm_eps: typing.Optional[float] = 1e-05,
    use_cache: typing.Optional[bool] = True,
    pad_token_id: typing.Optional[int] = 11,
    bos_token_id: typing.Optional[int] = 1,
    eos_token_id: typing.Optional[int] = 2,
    tie_word_embeddings: typing.Optional[bool] = False,
    rope_parameters: typing.Optional[dict] = None,
    sliding_window: typing.Optional[int] = None,
    attention_dropout: typing.Optional[float] = 0.0,
    kwargs = {}
)
```

**Bases:** `PretrainedConfig`

Configuration for Ministral3 text decoder.

```python
class nemo_automodel.components.models.mistral3.model.Ministral3DecoderLayer(
    config: nemo_automodel.components.models.mistral3.model.Ministral3Config,
    layer_idx: int
)
```

**Bases:** [GradientCheckpointingLayer](#nemo_automodel-components-models-mistral3-model-GradientCheckpointingLayer)

```python
nemo_automodel.components.models.mistral3.model.Ministral3DecoderLayer.forward(
    hidden_states: torch.Tensor,
    attention_mask: typing.Optional[torch.Tensor] = None,
    position_ids: typing.Optional[torch.LongTensor] = None,
    past_key_values: typing.Optional[transformers.cache_utils.Cache] = None,
    use_cache: typing.Optional[bool] = False,
    cache_position: typing.Optional[torch.LongTensor] = None,
    position_embeddings: typing.Optional[tuple[torch.Tensor, torch.Tensor]] = None,
    kwargs: transformers.processing_utils.Unpack[transformers.utils.TransformersKwargs] = {}
) -> torch.Tensor
```

```python
class nemo_automodel.components.models.mistral3.model.Ministral3ForCausalLM(
    config: nemo_automodel.components.models.mistral3.model.Ministral3Config
)
```

**Bases:** [HFCheckpointingMixin](/nemo-automodel/nemo_automodel/components/models/common/hf_checkpointing_mixin#nemo_automodel-components-models-common-hf_checkpointing_mixin-HFCheckpointingMixin), [Ministral3PreTrainedModel](#nemo_automodel-components-models-mistral3-model-Ministral3PreTrainedModel), `GenerationMixin`

```python
nemo_automodel.components.models.mistral3.model.Ministral3ForCausalLM.forward(
    input_ids: typing.Optional[torch.LongTensor] = None,
    attention_mask: typing.Optional[torch.Tensor] = None,
    position_ids: typing.Optional[torch.LongTensor] = None,
    past_key_values: typing.Optional[transformers.cache_utils.Cache] = None,
    inputs_embeds: typing.Optional[torch.FloatTensor] = None,
    labels: typing.Optional[torch.LongTensor] = None,
    use_cache: typing.Optional[bool] = None,
    cache_position: typing.Optional[torch.LongTensor] = None,
    logits_to_keep: typing.Union[int, torch.Tensor] = 0,
    output_hidden_states: typing.Optional[bool] = None,
    kwargs: transformers.processing_utils.Unpack[transformers.utils.TransformersKwargs] = {}
) -> transformers.modeling_outputs.CausalLMOutputWithPast
```

```python
nemo_automodel.components.models.mistral3.model.Ministral3ForCausalLM.get_input_embeddings()
```

```python
nemo_automodel.components.models.mistral3.model.Ministral3ForCausalLM.get_output_embeddings()
```

```python
nemo_automodel.components.models.mistral3.model.Ministral3ForCausalLM.set_input_embeddings(
    value
)
```

```python
nemo_automodel.components.models.mistral3.model.Ministral3ForCausalLM.set_output_embeddings(
    new_embeddings
)
```

```python
class nemo_automodel.components.models.mistral3.model.Ministral3MLP(
    config: nemo_automodel.components.models.mistral3.model.Ministral3Config
)
```

**Bases:** `Module`

```python
nemo_automodel.components.models.mistral3.model.Ministral3MLP.forward(
    x
)
```

```python
class nemo_automodel.components.models.mistral3.model.Ministral3Model(
    config: nemo_automodel.components.models.mistral3.model.Ministral3Config
)
```

**Bases:** [Ministral3PreTrainedModel](#nemo_automodel-components-models-mistral3-model-Ministral3PreTrainedModel)

```python
nemo_automodel.components.models.mistral3.model.Ministral3Model.forward(
    input_ids: typing.Optional[torch.LongTensor] = None,
    attention_mask: typing.Optional[torch.Tensor] = None,
    position_ids: typing.Optional[torch.LongTensor] = None,
    past_key_values: typing.Optional[transformers.cache_utils.Cache] = None,
    inputs_embeds: typing.Optional[torch.FloatTensor] = None,
    use_cache: typing.Optional[bool] = None,
    cache_position: typing.Optional[torch.LongTensor] = None,
    kwargs: transformers.processing_utils.Unpack[transformers.utils.TransformersKwargs] = {}
) -> transformers.modeling_outputs.BaseModelOutputWithPast
```

```python
class nemo_automodel.components.models.mistral3.model.Ministral3ModelOutputWithPast(
    image_hidden_states: typing.Optional[torch.FloatTensor] = None
)
```

Dataclass

**Bases:** `BaseModelOutputWithPast`

```python
class nemo_automodel.components.models.mistral3.model.Ministral3PreTrainedModel()
```

**Bases:** `PreTrainedModel`

```python
class nemo_automodel.components.models.mistral3.model.Ministral3RMSNorm(
    hidden_size,
    eps = 1e-06
)
```

**Bases:** `Module`

```python
nemo_automodel.components.models.mistral3.model.Ministral3RMSNorm.forward(
    hidden_states
)
```

```python
class nemo_automodel.components.models.mistral3.model.Ministral3RotaryEmbedding(
    config: nemo_automodel.components.models.mistral3.model.Ministral3Config,
    device = None
)
```

**Bases:** `Module`

```python
nemo_automodel.components.models.mistral3.model.Ministral3RotaryEmbedding.compute_default_rope_parameters(
    config: typing.Optional[nemo_automodel.components.models.mistral3.model.Ministral3Config] = None,
    device: typing.Optional[torch.device] = None,
    seq_len: typing.Optional[int] = None
) -> tuple[torch.Tensor, float]
```

staticmethod

```python
nemo_automodel.components.models.mistral3.model.Ministral3RotaryEmbedding.forward(
    x,
    position_ids
)
```

```python
nemo_automodel.components.models.mistral3.model._get_llama_4_attn_scale(
    positions_ids: torch.Tensor,
    beta: float,
    max_position_embeddings: int
) -> torch.Tensor
```

```python
nemo_automodel.components.models.mistral3.model._register_ministral3_with_transformers()
```

Register Ministral3Config and models with transformers Auto classes.

This uses the official transformers registration API. Registration is idempotent
(re-registering the same config/model is a no-op in recent transformers versions).

```python
nemo_automodel.components.models.mistral3.model.apply_rotary_pos_emb(
    q,
    k,
    cos,
    sin,
    position_ids = None,
    unsqueeze_dim = 1
)
```

```python
nemo_automodel.components.models.mistral3.model.eager_attention_forward(
    module: torch.nn.Module,
    query: torch.Tensor,
    key: torch.Tensor,
    value: torch.Tensor,
    attention_mask: typing.Optional[torch.Tensor],
    scaling: float,
    dropout: float = 0.0,
    kwargs: transformers.processing_utils.Unpack[transformers.utils.TransformersKwargs] = {}
)
```

```python
nemo_automodel.components.models.mistral3.model.repeat_kv(
    hidden_states: torch.Tensor,
    n_rep: int
) -> torch.Tensor
```

```python
nemo_automodel.components.models.mistral3.model.rotate_half(
    x
)
```

```python
nemo_automodel.components.models.mistral3.model.ModelClass = Ministral3ForCausalLM
```

```python
nemo_automodel.components.models.mistral3.model.logger = logging.get_logger(__name__)
```