Checkpoint Conversion

NVIDIA provides scripts to convert the external StarCoder2 checkpoints from HuggingFace format to .nemo format. The .nemo checkpoint will be used for SFT, PEFT, and inference. NVIDIA also provides scripts to convert .nemo format back to HuggingFace format.

Run the container using the following command:

Copy
Copied!
            

docker run --gpus device=1 --shm-size=2g --net=host --ulimit memlock=-1 --rm -it -v ${PWD}:/workspace -w /workspace -v ${PWD}/results:/results nvcr.io/nvidia/nemo:24.01.starcoder2 bash

Convert the HuggingFace StarCoder2 model to .nemo model:

Copy
Copied!
            

python3 /opt/NeMo/scripts/checkpoint_converters/convert_starcoder2_hf_to_nemo.py \ --input_name_or_path /path/to/starcoder2/checkpoints/hf \ --output_path /path/to/starcoder2.nemo

The generated starcoder2.nemo file uses distributed checkpointing and can be loaded with any tensor parallel (tp) or pipeline parallel (pp) combination without reshaping/splitting.

Convert the Starcoder2 .nemo model to HuggingFace

Copy
Copied!
            

python3 /opt/NeMo/scripts/checkpoint_converters/convert_starcoder2_hf_to_nemo.py \ --input_name_or_path /path/to/starcoder2/nemo/checkpoint \ --output_path /path/to/hf/folder

The generated HuggingFace checkpoint folder can be loaded using HuggingFace transformers pipeline and upload to the hub.

Previous Data Preparation for SFT and PEFT
Next Model Evaluation
© Copyright 2023-2024, NVIDIA. Last updated on May 17, 2024.