Models#

The following are the LLMs available as NIM. Click the model name link to view the hardware requirements for each model.

Model(Hardware Requirements)

Organization

Docker Image

Versions Supported

LoRA Support

Llama 3.1 8B Base

Meta

nvcr.io/nim/meta/llama-3.1-8b-base:latest

1.1.2, 1.1.0

Hugging Face, NeMo Formats

Llama 3.1 8B Instruct

Meta

nvcr.io/nim/meta/llama-3.1-8b-instruct:latest

1.1.2, 1.1.0

Hugging Face, NeMo Formats

Llama 3.1 70B Instruct

Meta

nvcr.io/nim/meta/llama-3.1-70b-instruct:latest

1.1.2, 1.1.0

Hugging Face, NeMo Formats

Llama 3.1 405B Instruct

Meta

nvcr.io/nim/meta/llama-3.1-405b-instruct:latest

1.1.2, 1.1.0

-

Llama 3 8B Instruct

Meta

nvcr.io/nim/meta/llama3-8b-instruct:latest

1.0.3, 1.0.0

Hugging Face, NeMo Formats

Llama 3 70B Instruct

Meta

nvcr.io/nim/meta/llama3-70b-instruct:latest

1.0.3, 1.0.0

Hugging Face, NeMo Formats

Llama 3 Swallow 70B Instruct V0.1

Meta

nvcr.io/nim/tokyotech-llm/llama-3-swallow-70b-instruct-v0.1:latest

1.1.2

Hugging Face, NeMo Formats

Llama 3 Taiwan 70B Instruct

Meta

nvcr.io/nim/yentinglin/llama-3-taiwan-70b-instruct:latest

1.1.2

Hugging Face, NeMo Formats

Mistral 7B Instruct v0.3

Mistral

nvcr.io/nim/mistralai/mistral-7b-instruct-v03:latest

1.0.0

Hugging Face, NeMo Formats

Mixtral 8x7B Instruct v0.1

Mistral

nvcr.io/nim/mistralai/mixtral-8x7b-instruct-v01:latest

1.0.0

-

Mixtral 8x22B Instruct v0.1

Mistral

nvcr.io/nim/mistralai/mixtral-8x22b-instruct-v01:latest

1.0.0

-

Nemotron 4 340B Instruct

NVIDIA

nvcr.io/nim/nvidia/nemotron-4-340b-instruct:latest

1.1.2

-