Loading Models

To load an AIAA model you need a model config that describes the inference workflow and usually a model file that contains either the weights or the whole network structure.

There are multiple options to load a model into AIAA.

Loading from NGC

AIAA allows you to load the model directly from NVIDIA GPU Cloud (NGC).

A list of available pre-trained models are in NGC clara models. (“Annotation” models that required user inputs can be found at NGC annotation models)

Attention

For Clara v4.0+, all supported MMARs should carry the name prefix clara_pt_*

The following example is to load the clara_pt_spleen_ct_segmentation pre-trained model.

# note that the version in this command means the version on NGC
# which differs from the Clara-Train version
curl -X PUT "http://127.0.0.1:$AIAA_PORT/admin/model/clara_pt_spleen_ct_segmentation" \
     -F 'ngc={"path":"nvidia/med/clara_pt_spleen_ct_segmentation","version":"1"}'

You can also download the model from NGC and load it.

ngc registry model download-version nvidia/med/clara_pt_spleen_ct_segmentation:1

curl -X PUT "http://127.0.0.1:$AIAA_PORT/admin/model/clara_pt_spleen_ct_segmentation" \
     -F "config=@clara_pt_spleen_ct_segmentation_v1/config/config_aiaa.json;type=application/json" \
     -F "data=@clara_pt_spleen_ct_segmentation_v1/models/model.ts"

Attention

Follow NGC CLI installation to setup NGC CLI first.

Loading from MMAR

If you have already downloaded the MMAR into a local disk, you can use the following approach to load it from the disk.

# loading segmentation spleen model
curl -X PUT "http://127.0.0.1:$AIAA_PORT/admin/model/clara_pt_spleen_ct_segmentation" \
     -F "data=@clara_pt_spleen_ct_segmentation.tgz"

Loading TorchScript Model

If you have a PyTorch trained model in TorchScript format, you can load it as follows.

curl -X PUT "http://127.0.0.1:$AIAA_PORT/admin/model/clara_pt_spleen_ct_segmentation" \
     -F "config=@config_aiaa.json;type=application/json" \
     -F "data=@model.ts"

Tip

If you are using Clara to train your models, you can use export.sh to convert your model to a TorchScript format.

Attention

Before running inference or using clients, make sure you can see your models in http://127.0.0.1:$AIAA_PORT/v1/models. If not, please follow instructions in Frequently Asked Questions to debug.