> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/automodel/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/automodel/_mcp/server.

# gpt-neox-20b

> Reference gpt-neox-20b checkpoint and architecture details for NeMo AutoModel, with model-family information, setup guidance, and upstream resources.

[GPT-NeoX](https://github.com/EleutherAI/gpt-neox) is EleutherAI's large-scale language model architecture. The same `GPTNeoXForCausalLM` architecture is used by the Pythia scaling suite, OpenAssistant, Databricks Dolly, and StableLM models.

Use this page as a checkpoint and architecture reference. Set up NeMo AutoModel with the
[latest container](https://catalog.ngc.nvidia.com/orgs/nvidia/containers/nemo-automodel) or follow the
[installation instructions](/get-started/installation).

## Model Reference

### Model Architecture

| Property                  | Value                                           |
| ------------------------- | ----------------------------------------------- |
| Task                      | Text Generation                                 |
| Architecture              | `GPTNeoXForCausalLM`                            |
| Parameters                | 20B                                             |
| Hugging Face Organization | [EleutherAI](https://huggingface.co/EleutherAI) |

### Available Models

| Model        | HF ID                                                                       |
| ------------ | --------------------------------------------------------------------------- |
| GPT-NeoX 20B | [`EleutherAI/gpt-neox-20b`](https://huggingface.co/EleutherAI/gpt-neox-20b) |

## Related Resources

* [LLM Fine-Tuning Guide](/recipes-e2e-examples/sft-peft)