Checkpoint Conversion

User Guide (Latest Version)

NVIDIA provides scripts to convert the external StarCoder2 checkpoints from HuggingFace format to .nemo format. The .nemo checkpoint will be used for SFT, PEFT, and inference. NVIDIA also provides scripts to convert .nemo format back to HuggingFace format.

Run the container using the following command:


docker run --gpus device=1 --shm-size=2g --net=host --ulimit memlock=-1 --rm -it -v ${PWD}:/workspace -w /workspace -v ${PWD}/results:/results bash

Convert the HuggingFace StarCoder2 model to .nemo model:


python3 /opt/NeMo/scripts/checkpoint_converters/ \ --input_name_or_path /path/to/starcoder2/checkpoints/hf \ --output_path /path/to/starcoder2.nemo

The generated starcoder2.nemo file uses distributed checkpointing and can be loaded with any tensor parallel (tp) or pipeline parallel (pp) combination without reshaping/splitting.

Convert the Starcoder2 .nemo model to HuggingFace


python3 /opt/NeMo/scripts/checkpoint_converters/ \ --input_name_or_path /path/to/starcoder2/nemo/checkpoint \ --output_path /path/to/hf/folder

The generated HuggingFace checkpoint folder can be loaded using HuggingFace transformers pipeline and upload to the hub.

Previous Data Preparation for SFT and PEFT
Next Model Evaluation
© | | | | | | |. Last updated on Jun 14, 2024.