Important
You are viewing the NeMo 2.0 documentation. This release introduces significant changes to the API and a new library, NeMo Run. We are currently porting all features from NeMo 1.0 to 2.0. For documentation on previous versions or features not yet available in 2.0, please refer to the NeMo 24.07 documentation.
Community Model Converter User Guide#
This guide provides instructions on how to use the conversion scripts to convert models between Community model and NVIDIA’s NeMo format.
Support Matrix#
Conversion |
From |
To |
Github Link |
|---|---|---|---|
Baichuan |
Hugging Face |
NeMo |
|
Baichuan |
NeMo |
Hugging Face |
|
BERT |
Hugging Face |
NeMo |
|
BERT |
NeMo |
Hugging Face |
|
Falcon |
Hugging Face |
NeMo |
|
Falcon |
NeMo |
Hugging Face |
|
Gemma |
Hugging Face |
NeMo |
|
Gemma |
JAX |
NeMo |
|
Gemma |
PyTorch |
NeMo |
|
GPT/LLaMA |
NeMo (Legacy) |
NeMo (Megatron-Core) |
|
LLaMA |
Hugging Face |
NeMo |
|
LLaMA |
NeMo |
Hugging Face |
|
Mistral 7B |
Hugging Face |
NeMo |
|
Mistral 7B |
NeMo |
Hugging Face |
|
Mixtral |
Hugging Face |
NeMo |
|
Mixtral |
NeMo |
Hugging Face |
|
MPT |
Hugging Face |
NeMo |
|
Starcoder |
Hugging Face |
NeMo |
Convert Hugging Face LLaMA Checkpoints to NeMo#
To convert a Hugging Face LLaMA checkpoint into a NeMo checkpoint, use the following command:
python convert_llama_hf_to_nemo.py \
--input_name_or_path <path_to_hf_checkpoints_folder> \
--output_path <path_to_output_nemo_file>
Convert NeMo Checkpoint to Hugging Face LLaMA#
To convert a NeMo checkpoint into a Hugging Face LLaMA checkpoint, you have two options:
Generate only the Hugging Face weights:
python convert_<model>_nemo_to_hf.py \
--input_name_or_path /path/to/file.nemo or /path/to/extracted_folder \
--output_path /path/to/pytorch_model.bin
Generate the full Hugging Face model folder:
python convert_<model>_nemo_to_hf.py \
--input_name_or_path /path/to/file.nemo or /path/to/extracted_folder \
--output_path /path/to/model_folder \
--hf_input_path /path/to/input_hf_folder \
--hf_output_path /path/to/output_hf_folder
Replace <model> with the specific model you are converting.
Use the --cpu-only flag if the model cannot fit in the GPU, such as for Llama2 70b models. Note that using this option will significantly slow down the conversion process.
Command-Line Arguments#
--input_name_or_path: Path to the input .nemo file or the Hugging Face model folder.--output_path: Path to the output file or folder, depending on the conversion direction.--hf_input_path: (Optional) Path to the input Hugging Face model folder.--hf_output_path: (Optional) Path to the output Hugging Face model folder.