Skip to navigation

LFM3-1.7B has a checked-in NeMo AutoModel recipe for text generation. The Hugging Face configuration declares the Lfm3ForCausalLM architecture: gated short convolutions interleaved with full and sliding-window grouped-query attention.

Fine-Tune LFM3-1.7B

Follow the installation instructions, then run the recipe from the repository root:

automodel examples/llm_finetune/lfm3/lfm3_1p7b_squad.yaml --nproc-per-node 8

Choose a Workflow

GoalStart Here
Run the primary recipe for this modelUse the recipe configuration.

Configuration

SettingConfiguration
Hardware-
StrategyFSDP2; tp_size=1, cp_size=1
Nodes-
Features-
Advanced-

NeMo AutoModel runs its own Lfm3ForCausalLM, so the checkpoint loads without remote code on the pinned Transformers release. Full-attention layers rotate half of each attention head, as the released weights were trained. Transformers 5.23.0 reads the released config.json as a full rotation, which changes its outputs on long inputs. Checkpoints exported by NeMo AutoModel record the rotation as proportional and load correctly in both.

Model Reference

Model Architecture

PropertyValue
TaskText generation
ArchitectureLfm3ForCausalLM

Available Models