> This page is for version 0.5.0 · 26.06.
> For other versions, use one of these documentation indexes:
> - Nightly (default): https://docs.nvidia.com/nemo/automodel/nightly/llms.txt
> - Latest: https://docs.nvidia.com/nemo/automodel/latest/llms.txt
> - 0.7.0 · 26.10: https://docs.nvidia.com/nemo/automodel/v0.7/llms.txt
> - 0.5.0 · 26.06: https://docs.nvidia.com/nemo/automodel/v0.5/llms.txt
> - 0.4.0 · 26.04: https://docs.nvidia.com/nemo/automodel/v0.4/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/automodel/llms.txt.

# nemo_automodel.components.models.lfm3

## Submodules

* **[`nemo_automodel.components.models.lfm3.config`](/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/models/lfm3/config)**
* **[`nemo_automodel.components.models.lfm3.model`](/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/models/lfm3/model)**
* **[`nemo_automodel.components.models.lfm3.processing`](/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/models/lfm3/processing)**
* **[`nemo_automodel.components.models.lfm3.vl_model`](/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/models/lfm3/vl_model)**

## Package Contents

### Classes

| Name                                                                              | Description                                                                 |
| --------------------------------------------------------------------------------- | --------------------------------------------------------------------------- |
| [`Lfm3ForCausalLM`](#nemo_automodel-components-models-lfm3-model-Lfm3ForCausalLM) | LFM3 with a vocabulary head, loadable from LiquidAI's released checkpoints. |

### API

```python
class nemo_automodel.components.models.lfm3.model.Lfm3ForCausalLM(
    config: nemo_automodel.components.models.lfm3.config.Lfm3Config
)
```

**Bases:** [HFCheckpointingMixin](/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/models/common/hf_checkpointing_mixin#nemo_automodel-components-models-common-hf_checkpointing_mixin-HFCheckpointingMixin), [Lfm3PreTrainedModel](#nemo_automodel-components-models-lfm3-model-Lfm3PreTrainedModel), `GenerationMixin`

LFM3 with a vocabulary head, loadable from LiquidAI's released checkpoints.

**`_tied_weights_keys`** `= {'lm_head.weight': 'model.embed_tokens.weight'}`

---

**`lm_head`**

---

**`model`** `= Lfm3Model(config)`

---

**`tie_word_embeddings_support`** `TieSupport = TieSupport.BOTH`

---

```python
nemo_automodel.components.models.lfm3.model.Lfm3ForCausalLM.forward(
    input_ids: torch.LongTensor | None = None,
    attention_mask: torch.Tensor | dict[str, torch.Tensor | None] | None = None,
    position_ids: torch.LongTensor | None = None,
    past_key_values: transformers.cache_utils.Cache | None = None,
    inputs_embeds: torch.FloatTensor | None = None,
    labels: torch.LongTensor | None = None,
    use_cache: bool | None = None,
    logits_to_keep: int | torch.Tensor = 0,
    kwargs: transformers.processing_utils.Unpack[transformers.utils.TransformersKwargs] = {}
) -> transformers.modeling_outputs.CausalLMOutputWithPast
```

Predict next-token logits and, given labels, the causal LM loss.

**Parameters:**

**`input_ids`** `torch.LongTensor | None` — default: None

Long Tensor of shape \[batch, sequence]. Exclusive with `inputs_embeds`.

---

**`attention_mask`** `torch.Tensor | dict[str, torch.Tensor | None] | None` — default: None

Optional padding Tensor of shape \[batch, key_sequence], where
`key_sequence` also counts cached tokens.

---

**`position_ids`** `torch.LongTensor | None` — default: None

Optional long Tensor of shape \[batch, sequence].

---

**`past_key_values`** `Cache | None` — default: None

Optional generation cache, updated in place.

---

**`inputs_embeds`** `torch.FloatTensor | None` — default: None

Tensor of shape \[batch, sequence, hidden].

---

**`labels`** `torch.LongTensor | None` — default: None

Optional long Tensor of shape \[batch, sequence]; `-100` is ignored.
Shifted internally.

---

**`use_cache`** `bool | None` — default: None

Whether to create and return a generation cache.

---

**`logits_to_keep`** `int | torch.Tensor` — default: 0

Project only the last `logits_to_keep` positions (0 keeps
all), or the positions in a long Tensor of shape \[kept].

---

**`**kwargs`** `Unpack[TransformersKwargs]` — default: \{}

Hugging Face output-capture, attention and loss arguments.

---

**Returns:** `CausalLMOutputWithPast`

Output whose `logits` is a Tensor of shape \[batch, kept, vocab] and whose

```python
nemo_automodel.components.models.lfm3.model.Lfm3ForCausalLM.get_input_embeddings() -> torch.nn.Embedding
```

Return the token embedding table.

```python
nemo_automodel.components.models.lfm3.model.Lfm3ForCausalLM.get_output_embeddings() -> torch.nn.Linear
```

Return the vocabulary head.

```python
nemo_automodel.components.models.lfm3.model.Lfm3ForCausalLM.set_input_embeddings(
    value: torch.nn.Embedding
) -> None
```

Replace the token embedding table and retie the head.

```python
nemo_automodel.components.models.lfm3.model.Lfm3ForCausalLM.tie_weights(
    kwargs: bool | set[str] | None = {}
) -> None
```

Share the vocabulary head with the token embeddings when the config asks for it.