To load an AIAA model you need a model config that describes the inference workflow and usually a model file that contains either the weights or the whole network structure.
There are multiple options to load a model into AIAA.
AIAA allows you to load the model directly from NVIDIA GPU Cloud (NGC).
A list of available pre-trained models are in NGC clara models. (“Annotation” models that required user inputs can be found at NGC annotation models)
You can also use ngc registry model list nvidia/med/clara_pt_* to get a list of models.
The following example is to load the clara_pt_spleen_ct_segmentation pre-trained model.
# note that the version in this command means the version on NGC
# which differs from the Clara-Train version
curl -X PUT "http://127.0.0.1:$AIAA_PORT/admin/model/clara_pt_spleen_ct_segmentation" \
-H "accept: application/json" \
-H "Content-Type: application/json" \
-d '{"path":"nvidia/med/clara_pt_spleen_ct_segmentation","version":"1"}'
You can also download the model from NGC and load it.
ngc registry model download-version nvidia/med/clara_pt_spleen_ct_segmentation:1
curl -X PUT "http://127.0.0.1:$AIAA_PORT/admin/model/clara_pt_spleen_ct_segmentation" \
-F "config=@clara_pt_spleen_ct_segmentation_v1/config/config_aiaa.json;type=application/json" \
-F "data=@clara_pt_spleen_ct_segmentation_v1/models/model.trt.pb"
Follow NGC CLI installation to setup NGC CLI first.
If you have already downloaded the MMAR into a local disk, you can use the following approach to load it from the disk.
# loading segmentation spleen model
curl -X PUT "http://127.0.0.1:$AIAA_PORT/admin/model/clara_pt_spleen_ct_segmentation" \
-F "data=@clara_pt_spleen_ct_segmentation.tgz"
If you have a PyTorch trained model in TorchScript format, you can load it as follows.
curl -X PUT "http://127.0.0.1:$AIAA_PORT/admin/model/clara_pt_spleen_ct_segmentation" \
-F "config=@config_aiaa.json;type=application/json" \
-F "data=@model.ts"
If you are using Clara to train your models, you can use nvmidl/mmar:export.sh to convert your model to a TorchScript format.
If you are using PyTorch directly, you can refer to Convert PyTorch trained network.
Before running inference or using clients, make sure you can see your models in http://127.0.0.1:$AIAA_PORT/v1/models. If not, please follow instructions in Frequently Asked Questions to debug.