starcoder

View as Markdown

StarCoder is BigCode’s code language model trained on the Stack dataset. It uses Multi-Query Attention and Fill-in-the-Middle (FIM) objectives. WizardCoder also uses this architecture.

Use this page as a checkpoint and architecture reference. Set up NeMo AutoModel with the latest container or follow the installation instructions.

Model Reference

Model Architecture

PropertyValue
TaskCode Generation
ArchitectureGPTBigCodeForCausalLM
Parameters15.5B
Hugging Face Organizationbigcode

Available Models

ModelHF ID
StarCoderbigcode/starcoder