Model CoverageLarge Language ModelsEleutherAIgpt-neox-20b

gpt-neox-20b

View as Markdown

GPT-NeoX is EleutherAI’s large-scale language model architecture. The same GPTNeoXForCausalLM architecture is used by the Pythia scaling suite, OpenAssistant, Databricks Dolly, and StableLM models.

Use this page as a checkpoint and architecture reference. Set up NeMo AutoModel with the latest container or follow the installation instructions.

Model Reference

Model Architecture

PropertyValue
TaskText Generation
ArchitectureGPTNeoXForCausalLM
Parameters20B
Hugging Face OrganizationEleutherAI

Available Models

ModelHF ID
GPT-NeoX 20BEleutherAI/gpt-neox-20b