Large Language Models (1.1.0)
Large Language Models (1.1.0)

Models

The following are the LLMs available as NIM. Click the model name link to view the hardware requirements for each model.

Model(Hardware Requirements)

Organization

Docker Image

Versions Supported

LoRA Support

Llama 3.1 8B Base Meta nvcr.io/nim/meta/llama-3.1-8b-base:latest 1.1.2, 1.1.0 Hugging Face, NeMo Formats
Llama 3.1 8B Instruct Meta nvcr.io/nim/meta/llama-3.1-8b-instruct:latest 1.1.2, 1.1.0 Hugging Face, NeMo Formats
Llama 3.1 70B Instruct Meta nvcr.io/nim/meta/llama-3.1-70b-instruct:latest 1.1.2, 1.1.0 Hugging Face, NeMo Formats
Llama 3.1 405B Instruct Meta nvcr.io/nim/meta/llama-3.1-405b-instruct:latest 1.1.2, 1.1.0 -
Llama 3 8B Instruct Meta nvcr.io/nim/meta/llama3-8b-instruct:latest 1.0.3, 1.0.0 Hugging Face, NeMo Formats
Llama 3 70B Instruct Meta nvcr.io/nim/meta/llama3-70b-instruct:latest 1.0.3, 1.0.0 Hugging Face, NeMo Formats
Llama 3 Swallow 70B Instruct V0.1 Meta nvcr.io/nim/tokyotech-llm/llama-3-swallow-70b-instruct-v0.1:latest 1.1.2 Hugging Face, NeMo Formats
Llama 3 Taiwan 70B Instruct Meta nvcr.io/nim/yentinglin/llama-3-taiwan-70b-instruct:latest 1.1.2 Hugging Face, NeMo Formats
Mistral 7B Instruct v0.3 Mistral nvcr.io/nim/mistralai/mistral-7b-instruct-v03:latest 1.0.0 Hugging Face, NeMo Formats
Mixtral 8x7B Instruct v0.1 Mistral nvcr.io/nim/mistralai/mixtral-8x7b-instruct-v01:latest 1.0.0 -
Mixtral 8x22B Instruct v0.1 Mistral nvcr.io/nim/mistralai/mixtral-8x22b-instruct-v01:latest 1.0.0 -
Nemotron 4 340B Instruct NVIDIA nvcr.io/nim/nvidia/nemotron-4-340b-instruct:latest 1.1.2 -
Previous Introduction to LLM Inference Benchmarking
Next Support Matrix
© Copyright © 2024, NVIDIA Corporation. Last updated on Sep 9, 2024.