PEFT Training and Inference

Below is an example of how to use the training scripts for adapter tuning. The TRAIN_FILEs (and VALIDATION_FILEs) follow the same format as SFT:

Copy
Copied!
            

python /opt/NeMo/examples/nlp/language_modeling/tuning/megatron_t5_adapter_tuning.py \ model.language_model_path=<BASE_T5_MODEL> \ model.data.train_ds=[<TRAIN_FILE1>,<TRAIN_FILE2>,...] \ model.data.validation_ds=[<VALIDATION_FILE1>, <VALIDATION_FILE2>,...]

At the end of tuning, a ‘.nemo’ model is generated which contains the parameters for the PEFT model. Similarly, the PEFT framework has an inference script as well:

Copy
Copied!
            

python /data/NeMo/examples/nlp/language_modeling/tuning/megatron_t5_adapter_eval.py \ data.test_ds=[<TEST_FILE>] \ language_model_path=[BASE_T5_MODEL] \ adapter_model_file=[PEFT_MODEL] \ pred_file_path=<OUTPUT_FILE>

You can switch to IA3, P-tuning, or LoRA methods by using the same input arguments to a different script. Below is the table including filepaths for each PEFT method:

PEFT Method

Filepath

Adapter tuning /opt/NeMo/examples/nlp/language_modeling/tuning/megatron_t5_adapter_tuning.py
IA3 tuning /opt/NeMo/examples/nlp/language_modeling/tuning/megatron_t5_ia3_tuning.py
P-tuning /opt/NeMo/examples/nlp/language_modeling/tuning/megatron_t5_prompt_learning.py
LoRA tuning /opt/NeMo/examples/nlp/language_modeling/tuning/megatron_t5_lora_tuning.py

Similarly, the inference script filepaths are provided below:

PEFT Method

Filepath

Adapter tuning /opt/NeMo/examples/nlp/language_modeling/tuning/megatron_t5_adapter_eval.py
IA3 tuning /opt/NeMo/examples/nlp/language_modeling/tuning/megatron_t5_ia3_eval.py
P-tuning /opt/NeMo/examples/nlp/language_modeling/tuning/megatron_t5_prompt_learning_eval.py
LoRA tuning /opt/NeMo/examples/nlp/language_modeling/tuning/megatron_t5_lora_eval.py
Previous Model Evaluation
Next Model Fine-Tuning
© Copyright 2023-2024, NVIDIA. Last updated on May 17, 2024.