> This page is for version 0.7.0 · 26.10.
> For other versions, use one of these documentation indexes:
> - Nightly (default): https://docs.nvidia.com/nemo/automodel/nightly/llms.txt
> - Latest: https://docs.nvidia.com/nemo/automodel/latest/llms.txt
> - 0.7.0 · 26.10: https://docs.nvidia.com/nemo/automodel/v0.7/llms.txt
> - 0.5.0 · 26.06: https://docs.nvidia.com/nemo/automodel/v0.5/llms.txt
> - 0.4.0 · 26.04: https://docs.nvidia.com/nemo/automodel/v0.4/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/automodel/llms.txt.

# nemo_automodel.components.models.lfm3.config

LFM3 and LFM3-VL configurations for Transformers releases that predate them.

## Module Contents

### Classes

| Name                                                                                     | Description                                                                                    |
| ---------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------- |
| [`Lfm3Config`](#nemo_automodel-components-models-lfm3-config-Lfm3Config)                 | Configure the LFM3 decoder of gated short convolutions and grouped-query attention.            |
| [`Lfm3VlConfig`](#nemo_automodel-components-models-lfm3-config-Lfm3VlConfig)             | Configure LFM3-VL: an LFM3 decoder fed by the LFM3-VL vision encoder through an MLP projector. |
| [`Lfm3VlVisionConfig`](#nemo_automodel-components-models-lfm3-config-Lfm3VlVisionConfig) | Configure the LFM3-VL vision encoder of gated 2D convolutions and bidirectional attention.     |

### Data

[`_JsonValue`](#nemo_automodel-components-models-lfm3-config-_JsonValue)

### API

```python
class nemo_automodel.components.models.lfm3.config.Lfm3Config()
```

**Bases:** `PreTrainedConfig`

Configure the LFM3 decoder of gated short convolutions and grouped-query attention.

`layer_types` names each layer `"conv"`, `"full_attention"` or
`"sliding_attention"`. `rope_parameters` holds one entry per attention
type. Full-attention layers rotate only the first `partial_rotary_factor`
share of each head and leave the rest unrotated.

Released checkpoints label that partial rotation `"default"`, which
Transformers 5.23 reads as a full rotation. Such an entry is relabelled
`"proportional"` here, the name under which Transformers and the
publisher's code both build the trained frequencies, so saved configs load
correctly in either.

**`_rope_type_labels`** `= ('full_attention', 'sliding_attention')`

---

**`attention_bias`** `bool = False`

---

**`attention_dropout`** `float = 0.0`

---

**`attribute_map`** `= {'rms_norm_eps': 'norm_eps'}`

---

**`bos_token_id`** `int | None = 124894`

---

**`conv_L_cache`** `int = 7`

---

**`conv_bias`** `bool = False`

---

**`default_theta`** `= 1000000.0`

---

**`eos_token_id`** `int | list[int] | None = 124900`

---

**`head_dim`** `int = 128`

---

**`hidden_size`** `int = 2048`

---

**`initializer_range`** `float = 0.02`

---

**`intermediate_size`** `int = 7680`

---

**`keys_to_ignore_at_inference`** `= ['past_key_values']`

---

**`layer_types`** `list[str] | None = None`

---

**`max_position_embeddings`** `int = 128000`

---

**`model_type`** `= 'lfm3'`

---

**`norm_eps`** `float = 1e-05`

---

**`num_attention_heads`** `int = 16`

---

**`num_hidden_layers`** `int = 24`

---

**`num_key_value_heads`** `int = 4`

---

**`pad_token_id`** `int | None = 124893`

---

**`rope_parameters`** `dict | None = None`

---

**`sliding_window`** `int | None = 256`

---

**`tie_word_embeddings`** `bool = True`

---

**`use_attention_gate`** `bool = True`

---

**`use_cache`** `bool = True`

---

**`use_sandwich_norm`** `bool = True`

---

**`vocab_size`** `int = 128000`

---

```python
nemo_automodel.components.models.lfm3.config.Lfm3Config.__post_init__(
    kwargs: nemo_automodel.components.models.lfm3.config._JsonValue = {}
) -> None
```

```python
nemo_automodel.components.models.lfm3.config.Lfm3Config.validate_architecture() -> None
```

Reject RoPE settings this implementation does not run.

Transformers runs this before saving. The config runs it on construction and
the model runs it again when built, because loader overrides are written onto
an existing config and skip construction.

**Raises:**

* `ValueError`: If an attention layer type has no RoPE entry, uses a RoPE type
  other than `default` or `proportional`, or scales its frequencies
  with a non-unit `factor`.

```python
class nemo_automodel.components.models.lfm3.config.Lfm3VlConfig()
```

**Bases:** `PreTrainedConfig`

Configure LFM3-VL: an LFM3 decoder fed by the LFM3-VL vision encoder through an MLP projector.

**`image_token_id`** `int = 125030`

---

**`model_type`** `= 'lfm3_vl'`

---

**`projector_bias`** `bool = True`

---

**`projector_hidden_act`** `str = 'gelu'`

---

**`projector_hidden_size`** `int = 4096`

---

**`sub_configs`**

---

**`text_config`** `dict | PreTrainedConfig | None = None`

---

**`tie_word_embeddings`** `bool = True`

---

**`video_token_id`** `int = 125029`

---

**`vision_config`** `dict | PreTrainedConfig | None = None`

---

**`vision_end_token_id`** `int = 125010`

---

**`vision_start_token_id`** `int = 125009`

---

```python
nemo_automodel.components.models.lfm3.config.Lfm3VlConfig.__post_init__(
    kwargs: nemo_automodel.components.models.lfm3.config._JsonValue = {}
) -> None
```

```python
class nemo_automodel.components.models.lfm3.config.Lfm3VlVisionConfig()
```

**Bases:** `PreTrainedConfig`

Configure the LFM3-VL vision encoder of gated 2D convolutions and bidirectional attention.

`layer_types` names each layer `"conv"` or `"full_attention"`. Patches
are `patch_size` pixels square and `temporal_patch_size` frames deep;
the projector later merges each `spatial_merge_size` square of them.

**`base_config_key`** `= 'vision_config'`

---

**`conv_kernel_size`** `int = 3`

---

**`default_rope_type`** `= 'axial'`

---

**`default_theta`** `= 1000000.0`

---

**`hidden_size`** `int = 1024`

---

**`in_channels`** `int = 3`

---

**`initializer_range`** `float = 0.02`

---

**`intermediate_size`** `int = 2560`

---

**`layer_types`** `list[str] | None = None`

---

**`model_type`** `= 'lfm3_vl_vision'`

---

**`norm_eps`** `float = 1e-05`

---

**`num_attention_heads`** `int = 16`

---

**`num_hidden_layers`** `int = 14`

---

**`num_key_value_heads`** `int = 8`

---

**`patch_size`** `int = 16`

---

**`rope_parameters`** `dict | None = None`

---

**`spatial_merge_size`** `int = 2`

---

**`temporal_patch_size`** `int = 2`

---

```python
nemo_automodel.components.models.lfm3.config.Lfm3VlVisionConfig.__post_init__(
    kwargs: nemo_automodel.components.models.lfm3.config._JsonValue = {}
) -> None
```

```python
nemo_automodel.components.models.lfm3.config._JsonValue = str | int | float | bool | list | dict | None
```