Model CoverageLarge Language ModelsMicrosoftphi-2

phi-2

View as Markdown

Microsoft’s Phi are compact, high-capability language models designed to punch above their weight class. Phi-1.5 and Phi-2 use a standard transformer decoder architecture (PhiForCausalLM). For Phi-3 and Phi-4 see Phi-3 / Phi-4.

Set up NeMo AutoModel with the latest container or follow the installation instructions.

Fine-Tune phi-2

From the repository root, run:

uv run automodel --nproc-per-node=8 examples/llm_finetune/phi/phi_2_squad.yaml

Choose a Workflow

GoalStart Here
Supervised fine-tuning (SFT) - Phi-2 on SQuADUse phi_2_squad.yaml.
Low-rank adaptation (LoRA) - Phi-2 on SQuADUse phi_2_squad_peft.yaml.

Model Reference

Model Architecture

PropertyValue
TaskText Generation
ArchitecturePhiForCausalLM
Parameters2.7B
Hugging Face Organizationmicrosoft

Available Models

ModelHF ID
Phi-2microsoft/phi-2