PEFT Training and Inference

NVIDIA Docs Hub NVIDIA NeMo Framework User Guide PEFT Training and Inference

Below is an example of how to use the training scripts for adapter tuning. The TRAIN_FILEs (and VALIDATION_FILEs) follow the same format as SFT:

Copy
Copied!

            
            python /opt/NeMo/examples/nlp/language_modeling/tuning/megatron_t5_adapter_tuning.py \
    model.language_model_path=<BASE_T5_MODEL> \
    model.data.train_ds=[<TRAIN_FILE1>,<TRAIN_FILE2>,...] \
    model.data.validation_ds=[<VALIDATION_FILE1>, <VALIDATION_FILE2>,...]

At the end of tuning, a ‘.nemo’ model is generated which contains the parameters for the PEFT model. Similarly, the PEFT framework has an inference script as well:

Copy
Copied!

            
            python /data/NeMo/examples/nlp/language_modeling/tuning/megatron_t5_adapter_eval.py \
    data.test_ds=[<TEST_FILE>] \
    language_model_path=[BASE_T5_MODEL] \
    adapter_model_file=[PEFT_MODEL] \
    pred_file_path=<OUTPUT_FILE>

You can switch to IA3, P-tuning, or LoRA methods by using the same input arguments to a different script. Below is the table including filepaths for each PEFT method:

PEFT Method	Filepath
Adapter tuning	/opt/NeMo/examples/nlp/language_modeling/tuning/megatron_t5_adapter_tuning.py
IA3 tuning	/opt/NeMo/examples/nlp/language_modeling/tuning/megatron_t5_ia3_tuning.py
P-tuning	/opt/NeMo/examples/nlp/language_modeling/tuning/megatron_t5_prompt_learning.py
LoRA tuning	/opt/NeMo/examples/nlp/language_modeling/tuning/megatron_t5_lora_tuning.py

Similarly, the inference script filepaths are provided below:

PEFT Method	Filepath
Adapter tuning	/opt/NeMo/examples/nlp/language_modeling/tuning/megatron_t5_adapter_eval.py
IA3 tuning	/opt/NeMo/examples/nlp/language_modeling/tuning/megatron_t5_ia3_eval.py
P-tuning	/opt/NeMo/examples/nlp/language_modeling/tuning/megatron_t5_prompt_learning_eval.py
LoRA tuning	/opt/NeMo/examples/nlp/language_modeling/tuning/megatron_t5_lora_eval.py

Previous Model Evaluation

Next Model Fine-Tuning