Large Language Models (Latest)
NVIDIA Docs Hub  NVIDIA NIM  Large Language Models (Latest)  Models
Large Language Models (Latest)

Models

The following are the LLMs available as NIM. Select the model name link to view the hardware requirements for each model. Select the Catalog Page link to get additional information about the model.

Model(Hardware Requirements)

Organization

Catalog Page

Versions Supported

LoRA Support

Tool Calling Support

Parallel Tool Calling Support
Llama 2 13B Chat Meta meta-llama-2-13b-chat 1.2.1 H100, A100, L40S - -
Llama 2 70B Chat Meta meta-llama-2-70b-chat 1.1.2, 1.1.0 H100, A100 - -
Llama 3 SQLCoder 8B Instruct Meta Llama-3-SQLCoder-8B-Instruct 1.2.3 - - -
Llama 3 Swallow 70B Instruct V0.1 Meta Llama-3-Swallow-70B-Instruct-v0.1 1.2.0, 1.1.2, 1.1.0 - - -
Llama 3 Taiwan 70B Instruct Meta Llama-3-Taiwan-70B-Instruct 1.2.1, 1.1.2, 1.1.0 - - -
Llama 3.1 8B Instruct Meta Llama-3.1-8b-instruct 1.3.0 - Yes Yes
Llama 3.1 70B Instruct Meta Llama-3.1-70b-instruct 1.3.0 - Yes Yes
Llama 3.1 405B Instruct Meta Llama-3.1-405b-instruct 1.3.0 - Yes Yes
Llama 3.1 Nemotron 70B Instruct Meta Llama-3.1 Nemotron 70B-Instruct 1.2.3 - - -
Llama 3.1 Nemotron 70B Reward Meta Llama-3.1 Nemotron 70B-Reward 1.2.3 - - -
Llama 3.1 405B Instruct Meta Llama-3.1-405b-instruct 1.2.0, 1.1.2, 1.1.0 - Yes Yes
Mistral 7B Instruct v0.3 Mistral Mistral-07B-Instruct-v0.3 1.3, 1.1.2, 1.0.0 Hugging Face, NeMo Formats Yes Yes
Mistral NeMo 12B Instruct Mistral Mistral-Nemo-12B-Instruct 1.1.2, 1.0.0 - - -
Mistral NeMo Minitron 8B 8K Instruct Mistral Mistral-NeMo-Minitron-8B-instruct 1.2.3 - - -
Mixtral 8x7B Instruct v0.1 Mistral Mixtral-8x7B-Instruct-v0.1 1.2.1, 1.0.0 Hugging Face, NeMo Formats No No
Mixtral 8x22B Instruct v0.1 Mistral Mixtral-8x22B-Instruct-v0.1 1.0.0 - Yes Yes
Nemotron 4 340B Instruct NVIDIA nemotron-4-340b-instruct 1.2.0, 1.1.2 - - -
Nemotron 4 340B Instruct 128K NVIDIA nemotron-4-340b-instruct-128k 1.3, 1.2.3 - - -
Nemotron 4 340B Reward NVIDIA Nemotron-4-340B-Reward 1.3, 1.2.0, 1.1.2 - Yes Yes
Phi 3 Mini 4K Instruct Microsoft Phi-3-4K-Instruct 1.2.3 - - -
Phind Codellama 34B v2 Microsoft Phind-Codellama-34B-v2 1.2.3 - - -
Previous Introduction to LLM Inference Benchmarking
Next Support Matrix
© Copyright © 2024, NVIDIA Corporation. Last updated on Nov 28, 2024.
content here