Important
NeMo 2.0 is an experimental feature and currently released in the dev container only: nvcr.io/nvidia/nemo:dev. Please refer to NeMo 2.0 overview for information on getting started.
Community Model Converter User Guide
This guide provides instructions on how to use the conversion scripts to convert models between Community model and NVIDIA’s NeMo format.
Support Matrix
Conversion |
From |
To |
Github Link |
---|---|---|---|
Baichuan |
Hugging Face |
NeMo |
|
Baichuan |
NeMo |
Hugging Face |
|
BERT |
Hugging Face |
NeMo |
|
BERT |
NeMo |
Hugging Face |
|
Falcon |
Hugging Face |
NeMo |
|
Falcon |
NeMo |
Hugging Face |
|
Gemma |
Hugging Face |
NeMo |
|
Gemma |
JAX |
NeMo |
|
Gemma |
PyTorch |
NeMo |
|
GPT/LLaMA |
NeMo (Legacy) |
NeMo (Megatron-Core) |
|
LLaMA |
Hugging Face |
NeMo |
|
LLaMA |
NeMo |
Hugging Face |
|
Mistral 7B |
Hugging Face |
NeMo |
|
Mistral 7B |
NeMo |
Hugging Face |
|
Mixtral |
Hugging Face |
NeMo |
|
Mixtral |
NeMo |
Hugging Face |
|
MPT |
Hugging Face |
NeMo |
|
Starcoder |
Hugging Face |
NeMo |
Convert Hugging Face LLaMA Checkpoints to NeMo
To convert a Hugging Face LLaMA checkpoint into a NeMo checkpoint, use the following command:
python convert_llama_hf_to_nemo.py \
--input_name_or_path <path_to_hf_checkpoints_folder> \
--output_path <path_to_output_nemo_file>
Convert NeMo Checkpoint to Hugging Face LLaMA
To convert a NeMo checkpoint into a Hugging Face LLaMA checkpoint, you have two options:
Generate only the Hugging Face weights:
python convert_<model>_nemo_to_hf.py \
--input_name_or_path /path/to/file.nemo or /path/to/extracted_folder \
--output_path /path/to/pytorch_model.bin
Generate the full Hugging Face model folder:
python convert_<model>_nemo_to_hf.py \
--input_name_or_path /path/to/file.nemo or /path/to/extracted_folder \
--output_path /path/to/model_folder \
--hf_input_path /path/to/input_hf_folder \
--hf_output_path /path/to/output_hf_folder
Replace <model> with the specific model you are converting.
Use the --cpu-only
flag if the model cannot fit in the GPU, such as for Llama2 70b models. Note that using this option will significantly slow down the conversion process.
Command-Line Arguments
--input_name_or_path
: Path to the input .nemo file or the Hugging Face model folder.--output_path
: Path to the output file or folder, depending on the conversion direction.--hf_input_path
: (Optional) Path to the input Hugging Face model folder.--hf_output_path
: (Optional) Path to the output Hugging Face model folder.