Fine-tuning

NSFW Content filtering essentially performs linear probing on top of existing CLIP checkpoint. The recommended configuration can be found in the conf/fine_tuning/nsfw which corresponds to CLIP L-14 version. You can access and modify parameters to customize the hyperparameters according to your specific training requirements and base model needs.

To enable the training stage with an NSFW model, configure the following configuration files:

1. In the defaults section, update the training field to point to the desired NSFW configuration file. The only provided configuration for now is nsfw_L_14.yaml.

   Copy

   Copied!

               
   
               
   defaults:
     - _self_
     - cluster: bcm
     - fine_tuning: nsfw/nsfw_L_14
   ...
           

2. In the stages field of conf/config.yaml, make sure the training stage is included. For example,

   Copy

   Copied!

               
   
               
   stages:
     - fine_tuning
     ...
           

3. Execute launcher pipeline: python3 main.py

Remarks:

1. You should feed the trained CLIP checkpoint into NSFW training by specifying training.model.restore_from_path (or set restore_from_path field in the model section of nsfw/nsfw_L_14.yaml). The checkpoint can be sourced from either NeMo or converted from Hugging Face in the form of a .nemo file.