Model CoverageLarge Language ModelsMicrosoftPhi-3-small-8k-instruct

Phi-3-small-8k-instruct

View as Markdown

Phi-3-Small is Microsoft’s 7B model using a distinct Phi3SmallForCausalLM architecture with blocksparse attention, separate from the standard Phi-3 family.

Use this page as a checkpoint and architecture reference. Set up NeMo AutoModel with the latest container or follow the installation instructions.

Model Reference

Model Architecture

PropertyValue
TaskText Generation
ArchitecturePhi3SmallForCausalLM
Parameters7B
Hugging Face Organizationmicrosoft

Available Models

ModelHF ID
Phi-3-small-8k-instructmicrosoft/Phi-3-small-8k-instruct