> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/automodel/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/automodel/_mcp/server.

# gpt-j-6b

> Reference gpt-j-6b checkpoint and architecture details for NeMo AutoModel, with model-family information, setup guidance, and upstream resources.

[GPT-J](https://github.com/kingoflolz/mesh-transformer-jax) is a 6B parameter transformer language model trained by EleutherAI on the Pile dataset. It was one of the earliest large open-weight models.

Use this page as a checkpoint and architecture reference. Set up NeMo AutoModel with the
[latest container](https://catalog.ngc.nvidia.com/orgs/nvidia/containers/nemo-automodel) or follow the
[installation instructions](/get-started/installation).

## Model Reference

### Model Architecture

| Property                  | Value                                           |
| ------------------------- | ----------------------------------------------- |
| Task                      | Text Generation                                 |
| Architecture              | `GPTJForCausalLM`                               |
| Parameters                | 6B                                              |
| Hugging Face Organization | [EleutherAI](https://huggingface.co/EleutherAI) |

### Available Models

| Model    | HF ID                                                               |
| -------- | ------------------------------------------------------------------- |
| GPT-J 6B | [`EleutherAI/gpt-j-6b`](https://huggingface.co/EleutherAI/gpt-j-6b) |

## Related Resources

* [LLM Fine-Tuning Guide](/recipes-e2e-examples/sft-peft)