Loading Models

To load an AIAA model you need a model config that describes the inference workflow and usually a model file that contains either the weights or the whole network structure.

There are multiple options to load a model into AIAA.

Loading from NGC

AIAA allows you to load the model directly from NVIDIA GPU Cloud (NGC).

A list of available pre-trained models are in NGC clara models. (“Annotation” models that required user inputs can be found at NGC annotation models)

The following example is to load the clara_pt_spleen_ct_segmentation pre-trained model.

# note that the version in this command means the version on NGC
# which differs from the Clara-Train version
curl -X PUT "http://127.0.0.1:$AIAA_PORT/admin/model/clara_pt_spleen_ct_segmentation" \
     -H "accept: application/json" \
     -H "Content-Type: application/json" \
     -d '{"path":"nvidia/med/clara_pt_spleen_ct_segmentation","version":"1"}'

You can also download the model from NGC and load it.

ngc registry model download-version nvidia/med/clara_pt_spleen_ct_segmentation:1

curl -X PUT "http://127.0.0.1:$AIAA_PORT/admin/model/clara_pt_spleen_ct_segmentation" \
     -F "config=@clara_pt_spleen_ct_segmentation_v1/config/config_aiaa.json;type=application/json" \
     -F "data=@clara_pt_spleen_ct_segmentation_v1/models/model.trt.pb"

Attention

Follow NGC CLI installation to setup NGC CLI first.

Loading from MMAR

If you have already downloaded the MMAR into a local disk, you can use the following approach to load it from the disk.

# loading segmentation spleen model
curl -X PUT "http://127.0.0.1:$AIAA_PORT/admin/model/clara_pt_spleen_ct_segmentation" \
     -F "data=@clara_pt_spleen_ct_segmentation.tgz"

Loading TorchScript Model

If you have a PyTorch trained model in TorchScript format, you can load it as follows.

curl -X PUT "http://127.0.0.1:$AIAA_PORT/admin/model/clara_pt_spleen_ct_segmentation" \
     -F "config=@config_aiaa.json;type=application/json" \
     -F "data=@model.ts"

Tip

If you are using Clara to train your models, you can use export.sh to convert your model to a TorchScript format.

If you are using PyTorch directly, you can refer to Convert PyTorch trained network.

Attention

Before running inference or using clients, make sure you can see your models in http://127.0.0.1:$AIAA_PORT/v1/models. If not, please follow instructions in Frequently Asked Questions to debug.