Model CoverageLarge Language ModelsMicrosoftPhi-4

Phi-4

View as Markdown

Phi-3 and Phi-4 are Microsoft’s high-capability small language models using a shared transformer decoder architecture (Phi3ForCausalLM). Phi-4-mini and Phi-4 achieve strong benchmark results at relatively small parameter counts.

Set up NeMo AutoModel with the latest container or follow the installation instructions.

Fine-Tune Phi-4

From the repository root, run:

uv run automodel --nproc-per-node=8 examples/llm_finetune/phi/phi_4_squad.yaml

Choose a Workflow

GoalStart Here
Supervised fine-tuning (SFT) - Phi-4 on SQuADUse phi_4_squad.yaml.
Low-rank adaptation (LoRA) - Phi-4 on SQuADUse phi_4_squad_peft.yaml.

Model Reference

Model Architecture

PropertyValue
TaskText Generation
ArchitecturePhi3ForCausalLM
Parameters14B
Hugging Face Organizationmicrosoft

Available Models

ModelHF ID
Phi-4microsoft/Phi-4
phi-4microsoft/phi-4