Large Language Models (LLMs)
Large Language Models (LLMs)
Introduction
Large Language Models (LLMs) power a variety of tasks such as dialogue systems, text classification, summarization, and more. NeMo AutoModel provides a simple interface for loading and fine-tuning LLMs hosted on the Hugging Face Hub.
Run LLMs with NeMo AutoModel
To run LLMs with NeMo AutoModel, make sure you use NeMo container version 26.06.00 or later. If the model you intend to fine-tune requires a newer source than the container provides, use a current checkout so the package and its example recipes stay on one revision:
For other installation options (such as uv), refer to the NeMo AutoModel Installation Guide.
Supported Models
NeMo AutoModel supports the AutoModelForCausalLM in the Text Generation category. During preprocessing, it uses transformers.AutoTokenizer, which is sufficient for most LLM cases. If your model requires custom text handling, override the tokenizer in your recipe YAML or provide a custom dataset _target_. Refer to LLM datasets and dataset overview.
Fine-Tuning LLMs with NeMo AutoModel
The models listed above can be fine-tuned using NeMo AutoModel. NeMo AutoModel supports two primary fine-tuning approaches:
- Parameter-Efficient Fine-Tuning (PEFT): Updates only a small subset of parameters (typically less than 1%) using techniques such as Low-Rank Adaptation (LoRA).
- Supervised Fine-Tuning (SFT): Updates all or most model parameters for deeper adaptation.
See the Fine-Tuning Guide to learn how to apply both methods to your data.
In these guides, the SQuAD v1.1 dataset is used for demonstration purposes, but you can use custom data. Update the recipe YAML dataset and validation_dataset sections accordingly. Refer to LLM Datasets and Dataset Overview.