Community Model Converter User Guide

This guide provides instructions on how to use the conversion scripts to convert models between Community model and NVIDIA’s NeMo format.

Conversion

From

To

Github Link

Baichuan Hugging Face NeMo Link
Baichuan NeMo Hugging Face Link
BERT Hugging Face NeMo Link
BERT NeMo Hugging Face Link
Falcon Hugging Face NeMo Link
Falcon NeMo Hugging Face Link
Gemma Hugging Face NeMo Link
Gemma JAX NeMo Link
Gemma PyTorch NeMo Link
GPT/LLaMA NeMo (Legacy) NeMo (Megatron-Core) Link
LLaMA Hugging Face NeMo Link
LLaMA NeMo Hugging Face Link
Mistral 7B Hugging Face NeMo Link
Mistral 7B NeMo Hugging Face Link
Mixtral Hugging Face NeMo Link
Mixtral NeMo Hugging Face Link
MPT Hugging Face NeMo Link
Starcoder Hugging Face NeMo Link

To convert a Hugging Face LLaMA checkpoint into a NeMo checkpoint, use the following command:

Copy
Copied!
            

python convert_llama_hf_to_nemo.py \ --input_name_or_path <path_to_hf_checkpoints_folder> \ --output_path <path_to_output_nemo_file>

To convert a NeMo checkpoint into a Hugging Face LLaMA checkpoint, you have two options:

  1. Generate only the Hugging Face weights:

Copy
Copied!
            

python convert_<model>_nemo_to_hf.py \ --input_name_or_path /path/to/file.nemo or /path/to/extracted_folder \ --output_path /path/to/pytorch_model.bin

  1. Generate the full Hugging Face model folder:

Copy
Copied!
            

python convert_<model>_nemo_to_hf.py \ --input_name_or_path /path/to/file.nemo or /path/to/extracted_folder \ --output_path /path/to/model_folder \ --hf_input_path /path/to/input_hf_folder \ --hf_output_path /path/to/output_hf_folder

Replace <model> with the specific model you are converting.

Use the --cpu-only flag if the model cannot fit in the GPU, such as for Llama2 70b models. Note that using this option will significantly slow down the conversion process.

  • --input_name_or_path: Path to the input .nemo file or the Hugging Face model folder.

  • --output_path: Path to the output file or folder, depending on the conversion direction.

  • --hf_input_path: (Optional) Path to the input Hugging Face model folder.

  • --hf_output_path: (Optional) Path to the output Hugging Face model folder.

Previous Community Checkpoint Converter
Next Community Model Converter Development Guide
© Copyright 2023-2024, NVIDIA. Last updated on May 17, 2024.