> This page is for version Latest.
> For other versions, use one of these documentation indexes:
> - Nightly (default): https://docs.nvidia.com/nemo/automodel/nightly/llms.txt
> - Latest: https://docs.nvidia.com/nemo/automodel/latest/llms.txt
> - 0.7.0 · 26.10: https://docs.nvidia.com/nemo/automodel/v0.7/llms.txt
> - 0.5.0 · 26.06: https://docs.nvidia.com/nemo/automodel/v0.5/llms.txt
> - 0.4.0 · 26.04: https://docs.nvidia.com/nemo/automodel/v0.4/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/automodel/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/automodel/_mcp/server.

# starcoder

> Reference starcoder checkpoint and architecture details for NeMo AutoModel, with model-family information, setup guidance, and upstream resources.

[StarCoder](https://huggingface.co/blog/starcoder) is BigCode's code language model trained on the Stack dataset. It uses Multi-Query Attention and Fill-in-the-Middle (FIM) objectives. WizardCoder also uses this architecture.

Use this page as a checkpoint and architecture reference. Set up NeMo AutoModel with the
[latest container](https://catalog.ngc.nvidia.com/orgs/nvidia/containers/nemo-automodel) or follow the
[installation instructions](/nemo/automodel/latest/get-started/installation).

## Model Reference

### Model Architecture

| Property                  | Value                                     |
| ------------------------- | ----------------------------------------- |
| Task                      | Code Generation                           |
| Architecture              | `GPTBigCodeForCausalLM`                   |
| Parameters                | 15.5B                                     |
| Hugging Face Organization | [bigcode](https://huggingface.co/bigcode) |

### Available Models

| Model     | HF ID                                                           |
| --------- | --------------------------------------------------------------- |
| StarCoder | [`bigcode/starcoder`](https://huggingface.co/bigcode/starcoder) |

## Related Resources

* [LLM Fine-Tuning Guide](/nemo/automodel/latest/recipes-e2e-examples/sft-peft)