Model Fine-tune


If you are using Clara-Train to train your model, AIAA provides functionality to let you fine-tune your trained models based on new annotation samples you make.

AIAA will take care of fine-tuning the model and re-load it back to AIAA for serving.


This is an Admin API.


If you have only 1 GPU in your system, then AIAA can’t serve inference through segmentation/annotation APIs during fine-tuning.

You need to make sure you put new samples in /workspace/samples and have your MMAR in /workspace/mmars


To see an example of MMAR check nvmidl/mmar:Medical Model Archive (MMAR).

Below is an example directory for clara_ct_seg_spleen_amp model


workspace/ samples/ clara_pt_spleen_ct_segmentation/ options.conf #optional dataset.json #optional images/ spleen_2.nii.gz spleen_3.nii.gz ... labels/ spleen_2.nii.gz spleen_3.nii.gz ... mmars/ clara_pt_spleen_ct_segmentation/ commands/ configs/ ...

The uri to call is /admin/finetune/[model]. Note that you can pass all options that nvmidl/ takes. There are two ways of passing those options, one is via curl, which is demonstrated below.


# basic call curl -X POST "$AIAA_PORT/admin/finetune/clara_ct_seg_spleen_amp" # fine-tune for 5 epochs, %3D means space curl -X POST "$AIAA_PORT/admin/finetune/clara_ct_seg_spleen_amp?options=epochs%3D5"

The other is to edit the options.conf file. An example is below. You should put this file inside your /workspace/samples/[model]/options.conf.


epochs=5 learning_rate=0.00001


If you want to pick up specific images for training and validation, then you can provide your own configuration at: /workspace/samples/{model}/dataset.json.

You can also set automatic fine-tuning by adding flag --fine_tune true when starting AIAA. AIAA will run model fine-tune on the fine_tune_hour every day for all the models.

© Copyright 2020, NVIDIA. Last updated on Feb 2, 2023.