Important

NeMo 2.0 is an experimental feature and currently released in the dev container only: nvcr.io/nvidia/nemo:dev. Please refer to NeMo 2.0 overview for information on getting started.

Community Model Converter User Guide

This guide provides instructions on how to use the conversion scripts to convert models between Community model and NVIDIA’s NeMo format.

Support Matrix

Conversion	From	To	Github Link
Baichuan	Hugging Face	NeMo	Link
Baichuan	NeMo	Hugging Face	Link
BERT	Hugging Face	NeMo	Link
BERT	NeMo	Hugging Face	Link
Falcon	Hugging Face	NeMo	Link
Falcon	NeMo	Hugging Face	Link
Gemma	Hugging Face	NeMo	Link
Gemma	JAX	NeMo	Link
Gemma	PyTorch	NeMo	Link
GPT/LLaMA	NeMo (Legacy)	NeMo (Megatron-Core)	Link
LLaMA	Hugging Face	NeMo	Link
LLaMA	NeMo	Hugging Face	Link
Mistral 7B	Hugging Face	NeMo	Link
Mistral 7B	NeMo	Hugging Face	Link
Mixtral	Hugging Face	NeMo	Link
Mixtral	NeMo	Hugging Face	Link
MPT	Hugging Face	NeMo	Link
Starcoder	Hugging Face	NeMo	Link

Convert Hugging Face LLaMA Checkpoints to NeMo

To convert a Hugging Face LLaMA checkpoint into a NeMo checkpoint, use the following command:

python convert_llama_hf_to_nemo.py \
 --input_name_or_path <path_to_hf_checkpoints_folder> \
 --output_path <path_to_output_nemo_file>

Convert NeMo Checkpoint to Hugging Face LLaMA

To convert a NeMo checkpoint into a Hugging Face LLaMA checkpoint, you have two options:

1. Generate only the Hugging Face weights:

python convert_<model>_nemo_to_hf.py \
--input_name_or_path /path/to/file.nemo or /path/to/extracted_folder \
--output_path /path/to/pytorch_model.bin

2. Generate the full Hugging Face model folder:

python convert_<model>_nemo_to_hf.py \
--input_name_or_path /path/to/file.nemo or /path/to/extracted_folder \
--output_path /path/to/model_folder \
--hf_input_path /path/to/input_hf_folder \
--hf_output_path /path/to/output_hf_folder

Replace <model> with the specific model you are converting.

Use the --cpu-only flag if the model cannot fit in the GPU, such as for Llama2 70b models. Note that using this option will significantly slow down the conversion process.

Command-Line Arguments

• --input_name_or_path: Path to the input .nemo file or the Hugging Face model folder.

• --output_path: Path to the output file or folder, depending on the conversion direction.

• --hf_input_path: (Optional) Path to the input Hugging Face model folder.

• --hf_output_path: (Optional) Path to the output Hugging Face model folder.