Skip to navigation

nemo_automodel.components.models.lfm3

View as MarkdownOpen in Claude

Submodules

Package Contents

Classes

NameDescription
Lfm3ForCausalLMLFM3 with a vocabulary head, loadable from LiquidAI’s released checkpoints.

API

class nemo_automodel.components.models.lfm3.model.Lfm3ForCausalLM(
)

Bases: HFCheckpointingMixin, Lfm3PreTrainedModel, GenerationMixin

LFM3 with a vocabulary head, loadable from LiquidAI’s released checkpoints.

_tied_weights_keys
= {'lm_head.weight': 'model.embed_tokens.weight'}
lm_head
model
= Lfm3Model(config)
tie_word_embeddings_support
TieSupport = TieSupport.BOTH
nemo_automodel.components.models.lfm3.model.Lfm3ForCausalLM.forward(
input_ids: torch.LongTensor | None = None,
attention_mask: torch.Tensor | dict[str, torch.Tensor | None] | None = None,
position_ids: torch.LongTensor | None = None,
past_key_values: transformers.cache_utils.Cache | None = None,
inputs_embeds: torch.FloatTensor | None = None,
labels: torch.LongTensor | None = None,
use_cache: bool | None = None,
logits_to_keep: int | torch.Tensor = 0,
kwargs: transformers.processing_utils.Unpack[transformers.utils.TransformersKwargs] = {}
) -> transformers.modeling_outputs.CausalLMOutputWithPast

Predict next-token logits and, given labels, the causal LM loss.

Parameters:

input_ids
torch.LongTensor | NoneDefaults to None

Long Tensor of shape [batch, sequence]. Exclusive with inputs_embeds.

attention_mask
torch.Tensor | dict[str, torch.Tensor | None] | NoneDefaults to None

Optional padding Tensor of shape [batch, key_sequence], where key_sequence also counts cached tokens.

position_ids
torch.LongTensor | NoneDefaults to None

Optional long Tensor of shape [batch, sequence].

past_key_values
Cache | NoneDefaults to None

Optional generation cache, updated in place.

inputs_embeds
torch.FloatTensor | NoneDefaults to None

Tensor of shape [batch, sequence, hidden].

labels
torch.LongTensor | NoneDefaults to None

Optional long Tensor of shape [batch, sequence]; -100 is ignored. Shifted internally.

use_cache
bool | NoneDefaults to None

Whether to create and return a generation cache.

logits_to_keep
int | torch.TensorDefaults to 0

Project only the last logits_to_keep positions (0 keeps all), or the positions in a long Tensor of shape [kept].

**kwargs
Unpack[TransformersKwargs]Defaults to {}

Hugging Face output-capture, attention and loss arguments.

Returns: CausalLMOutputWithPast

Output whose logits is a Tensor of shape [batch, kept, vocab] and whose

nemo_automodel.components.models.lfm3.model.Lfm3ForCausalLM.get_input_embeddings() -> torch.nn.Embedding

Return the token embedding table.

nemo_automodel.components.models.lfm3.model.Lfm3ForCausalLM.get_output_embeddings() -> torch.nn.Linear

Return the vocabulary head.

nemo_automodel.components.models.lfm3.model.Lfm3ForCausalLM.set_input_embeddings(
value: torch.nn.Embedding
) -> None

Replace the token embedding table and retie the head.

nemo_automodel.components.models.lfm3.model.Lfm3ForCausalLM.tie_weights(
kwargs: bool | set[str] | None = {}
) -> None

Share the vocabulary head with the token embeddings when the config asks for it.