Models
The following are the LLMs available as NIM. Click the model name link to view the hardware requirements for each model.
Model(Hardware Requirements) |
Organization |
Docker Image |
Versions Supported |
LoRA Support |
Tool Calling Support |
Parallel Tool Calling Support |
|---|---|---|---|---|---|---|
| Llama 3.1 8B Base | Meta | nvcr.io/nim/meta/llama-3.1-8b-base:latest |
1.1.2, 1.1.0 |
Hugging Face, NeMo Formats | No | No |
| Llama 3.1 8B Instruct | Meta | nvcr.io/nim/meta/llama-3.1-8b-instruct:latest |
1.2.0, 1.1.2, 1.1.0 |
Hugging Face, NeMo Formats | Yes | No |
| Llama 3.1 70B Instruct | Meta | nvcr.io/nim/meta/llama-3.1-70b-instruct:latest |
1.2.1, 1.1.2, 1.1.0 |
Hugging Face, NeMo Formats | Yes | No |
| Llama 3.1 405B Instruct | Meta | nvcr.io/nim/meta/llama-3.1-405b-instruct:latest |
1.2.0, 1.1.2, 1.1.0 |
- | Yes | No |
| Llama 3 8B Instruct | Meta | nvcr.io/nim/meta/llama3-8b-instruct:latest |
1.0.3, 1.0.0 |
Hugging Face, NeMo Formats | No | No |
| Llama 3 70B Instruct | Meta | nvcr.io/nim/meta/llama3-70b-instruct:latest |
1.0.3, 1.0.0 |
Hugging Face, NeMo Formats | No | No |
| Mistral 7B Instruct v0.3 | Mistral | nvcr.io/nim/mistralai/mistral-7b-instruct-v03:latest |
1.1.2, 1.0.0 |
Hugging Face, NeMo Formats | Yes | Yes |
| Mixtral 8x7B Instruct v0.1 | Mistral | nvcr.io/nim/mistralai/mixtral-8x7b-instruct-v01:latest |
1.2.1, 1.0.0 |
- | No | No |
| Mixtral 8x22B Instruct v0.1 | Mistral | nvcr.io/nim/mistralai/mixtral-8x22b-instruct-v01:latest |
1.0.0 |
- | Yes | Yes |
| Nemotron 4 340B Reward | NVIDIA | nvcr.io/nim/nvidia/nemotron-4-340b-instruct:latest |
1.2.0, 1.1.2 |
- | Yes | Yes |