Upgrading from previous versions of Clara Train

Upgrading MMAR from Clara 4.0 to 4.1

The Clara 4.1 package was developed to align with the latest MONAI v0.8 APIs for better performance and flexibility. It is necessary to add to configurations in Clara 4.0 in order to make previous MMARs compatible with Clara Train 4.1.

Note

Objects can now be set with as config argument with “@”, for example:

{"dataloader": {"args": {"dataset": "@dataset", ...}}}

Code can be set with “#” to execute directly, for example:

{"optimizer": {"args": {"params": "#@model.parameters()", ...}}}

Please follow the steps below to update your MMAR if they are applicable:

  • Add new variables below to environment.json and update train.json, validation.json, and inference.json to use these variables:

    "CLARA_TRAIN_VERSION": "4.1.0",
    "TRAIN_DATALIST_KEY": "training",
    "VAL_DATALIST_KEY": "validation",
    "INFER_DATALIST_KEY": "test",
    "INPUT_CHANNELS": 1,
    "OUTPUT_CHANNELS": 2
    
  • Use the “Invertd” post processing transform instead of the “TransformInverter” handler.

  • Use the “SaveImaged” transform instead of the “SegmentationSaver” handler.

  • Change the output_transform lambda, because now the model output is decollated data (list of channel-first shape). For more information about decollate, please refer to: https://github.com/Project-MONAI/MONAI/wiki/v0.5-to-v0.6-migration-guide

    "output_transform": "lambda x: x['loss']"
    

    to

    "output_transform": "lambda x: x[0]['loss']"
    
  • Split the “metrics” section of JSON config to “key_metric” and “addtional_metrics”. For example:

    ...
    "key_metric": {
      "name": "MeanDice",
      "log_label": "val_mean_dice",
      "args": {
        "include_background": false,
        "output_transform": "lambda x: ([i['pred'] for i in x], [i['label'] for i in x])"
      }
    },
    "additional_metrics": [
      {
        "name": "Accuracy",
        "log_label": "val_acc",
        "args": {
          "output_transform": "lambda x: ([i['pred'] for i in x], [i['label'] for i in x])"
        }
      }
    ],
    
  • Pass arg “dataset” into DataLoader:

    ...
    "dataloader": {
      "name": "DataLoader",
      "args": {
        "dataset": "@dataset",
        "batch_size": 2,
        "shuffle": true,
        "num_workers": 2
      }
    },
    
  • Pass args “data” and “transform” into Dataset:

    ...
    "dataset": {
      "name": "CacheDataset",
      "data_list_file_path": "{DATASET_JSON}",
      "data_file_base_dir": "{DATA_ROOT}",
      "data_list_key": "{TRAIN_DATALIST_KEY}",
      "args": {
        "data": "@datalist",
        "transform": "@pre_transforms",
        "cache_num": 32,
        "cache_rate": 1.0,
        "num_workers": 2
      }
    },
    
  • Pass arg “val_data_loader” and “network” into SupervisedEvaluator:

    ...
    "evaluator": {
      "name": "SupervisedEvaluator",
      "args": {
        "device": "cuda",
        "val_data_loader": "@dataloader",
        "network": "@model",
        "inferer": "@inferer",
        "postprocessing": "@post_transforms",
        "key_val_metric": "@key_metric",
        "additional_metrics": "@additional_metrics",
        "val_handlers": "@handlers",
        "amp": "{amp}"
      }
    }
    
  • Pass arg “param” into Adam optimizer:

    ...
    "optimizer": {
      "name": "Adam",
      "args": {
        "params": "#@model.parameters()",
        "lr": "{learning_rate}"
      }
    },
    
  • Pass arg “optimizer” into LR scheduler

    ...
    {
      "name": "LrScheduleHandler",
      "args": {
        "lr_scheduler": "@lr_scheduler",
        "print_lr": true
      }
    },
    
  • CheckpointLoader and CheckpointSaver args load_dict and save_dict now require a dictionary instead of a list:

    ...
    {
      "name": "CheckpointLoader",
      "disabled": "{dont_load_ckpt_model}",
      "args": {
        "load_path": "{MMAR_CKPT}",
        "load_dict": {"model": "@model"}
      }
    },
    
    ...
    {
      "name": "CheckpointSaver",
      "rank": 0,
      "args": {
        "save_dir": "{MMAR_CKPT_DIR}",
        "save_dict": {
          "model": "@model",
          "optimizer": "@optimizer",
          "lr_scheduler": "@lr_scheduler",
          "train_conf": "@conf"
        },
        "save_interval": 400,
        "save_final": true
      }
    },
    
  • Pass arg “lr_scheduler” in LrScheduleHandler:

    ...
    {
      "name": "LrScheduleHandler",
      "args": {
        "lr_scheduler": "@lr_scheduler",
        "print_lr": true
      }
    },
    
  • Pass arg “validator” in ValidationHandler

    ...
    {
      "name": "ValidationHandler",
      "args": {
        "validator": "@evaluator",
        "interval": 1,
        "epoch_level": true
      }
    },
    

Converting from Clara 3.1 to Clara 4.0

Prior to Clara v4.0, TensorFlow was used as a back end instead of MONAI and PyTorch Ignite, and Clara Train SDK contained its own implementations of components instead of now using components from MONAI.

Unfortunately, this means some things have changed behind the scenes, and the following will need to be done for MMARs prior to Clara v4.0 to work with Clara Train v4.0 and later:

  • TF versions of components must be replaced with corresponding MONAI and PyTorch components

  • Arguments and parameters may have to be adjusted accordingly, and for some components there may not be a direct one to one mapping, so understanding what is happening in the pipeline is important

  • Please see our example Clara Train v4.0 MMARs on NGC and details below for specific changes

Only PT models are compatible with 4.0, and older models on NGC and for Clara 3.1 would need to be converted to be compatible.

MMAR Conversion Tool

A conversion tool has been provided in the docker image to convert a Clara 3.1 MMAR with TensorFlow to a Clara v4.0 MMAR. It will convert the workflow logic and components and where it is able and add warning messages in the converted JSON files to give tips.

To use the tool, include the path to the original MMAR after the -m flag and the directory to output the new MMAR to after the -o flag:

python3 -u -m medl.tools.mmar_converter.convert_from_tf -m $MMAR_ROOT -o ./

Please note that the conversion is a best effort, and the underlying implementations of many functions have changed. A list of warnings are included to highlight differences to pay attention to, but for the MONAI components, you can look at the underlying source code and decide if the implementation makes the most sense for your use case. Default args will be used for many components, so it is a good idea to go through the converted configurations and modify or delete anything that is not applicable.

Some unsupported parts may need to be manually modified based on the warnings at the end of the converted configurations. For example, some args of TensorFlow components may not map to exactly the same logic in PyTorch, or some necessary args for a Clara 4.0 MMAR are not in the Clara 3.1 MMAR.

Changes from Clara v3.X MMARs

In Clara Train 3.X, the concept of ShapeImage took care of different formats and allowed transforms to work with various dimensions, but for PT there is a convention for the dimensions to be channels first. Transforms in the example MMARs add the channel dimension to front to get data to conform to the channels first expectations, and you may have to do this depending on your data.

In MONAI, class names ending with ‘d’ denote dictionary-based transforms, and those are the ones used for Clara Train.

“dataset” and “dataloader” now take the place of what used to be configured in “image_pipeline”.

At the top level of the config: “num_training_epoch_per_valid” is now “num_interval_per_valid”.

For PT, ToTensord is needed at end of pre_transforms.

Attention

In general throughout the transforms, in args, the key “fields” has now been replaced with “keys”, but this may require additional attention in certain transforms like cropping where “label_key”, “source_key”, or “image_key” may be used.

Note

The tables below list many corresponding components, but for details or when in doubt, check the documentation and open source code for the implementations of the underlying MONAI transforms

IO Transforms

For loading images, LoadImaged is used for everything. This would replace LoadNifti, LoadPng, or LoadJpg for example.

Intensity transforms

Before v4.0

Component to use in v4.0

Notes

AddGaussianNoise

RandGaussianNoised

ShiftIntensityd

ScaleIntensityOscillation

RandShiftIntensityd

probability -> prob, magnitude -> offsets

ScaleIntensityd

ScaleShiftIntensity

RandScaleIntensityd

the previous component did what now is split into this and RandShiftIntensityd, and both components have additional args available for controlling the amount of scale/shift

NormalizeNonzeroIntensities

NormalizeIntensityd

bool to choose whether or not to normalize non-zero

CenterData

^ see above

ThresholdValues

ThresholdIntensityd

ScaleIntensityRange

ScaleIntensityRanged

ScaleIntensityRangePercentilesd

AdjustContrast

AdjustContrastd

RandAdjustContrastd

MaskIntensityd

GaussianSmoothd

RandomGaussianSmooth

RandGaussianSmoothd

GaussianSharpend

RandomSharpen

RandGaussianSharpend

RanHistogramShiftd

Cropping and padding transforms

Before v4.0

Component to use in v4.0

Notes

PadSymmetric

SpatialPadd

BorderPadd

SymmetricPadderDiv

DivisiblePadd

SpatialCropd

CropSubVolumeCenter

CenterSpatialCropd

CropFixedSizeRandomCenter

RandSpatialCropd

CropRandomSizeWithDisplacement

^ see above

RandSpatialCropSamplesd

CropForegroundObject

CropForegroundd

ResizeWithPadOrCropd

FastPosNegRatioCropROI

RandCropByPosNegLabeld

The Rand3DElasticD transform is added afterward if using the mapping tool

CropByPosNegRatio

CropByPosNegRatioLabelOnly

^ see above for options

FastCropByPosNegRatio

ClassificationFastCropByPosNegRatio

Spatial transforms

Before v4.0

Component to use in v4.0

Notes

ScaleBySpacing

Spacingd

target_resolution/target_spacing -> pixdim, is_label => mode; this can be used in training if data is not pre-converted to 1mm

ScaleByResolution

^ see above

Orientationd

Flipd

RandomSpatialFlip

RandFlipd

probability -> prob; clara-train will select random axis to flip while MONAI will flip all by default although “spatial_axis” can be set to specify which to flip

RandomAxisFlip

^ see above

Rotated

RandomRotate2D

RandRotated

angle -> range_x

ScaleByFactor

Zoomd

RandomZoom

RandZoomd

[image_field, label_field] -> keys, lower_limits -> min_zoom, upper_limits -> max_zoom, keep_size -> keep_size, probability -> prob, no need for use_gpu argument anymore

RandomRotate3D

RandRotate90d

probability -> prob

Rotate90d

ScaleToShape

Resized

target_shape -> spatial_size, is_label => mode

RandAffined

Rand2DElasticd

covers the rotation, deformation, and scaling that was previously done for TF with FastPosNegRatioCropROI

Rand3DElasticd

Post processing transforms

Before v4.0

Component to use in v4.0

Notes

SplitAcrossChannels

SplitChanneld

KeepLargestCC

KeepLargestConnectedComponentd

Activationsd

MONAI networks do not have activations, so we will need to add this transform to post-transform chain when converting from TF

AsDiscreted

LabelToContourd

Ensembled

MeanEnsembled

VoteEnsembled

Utility transforms

Before v4.0

Component to use in v4.0

Notes

Identityd

ConvertToChannelsFirst

EnsureChannelFirstd

ConvertToChannelsLast

AsChannelLastd

RepeatChannel

RepeatChanneld

CastToType

CastToTyped

AddExtremePointsChannel

AddExtremePointsChanneld

ConvertToMultiChannelBasedOnBratsClasses

ConvertToMultiChannelBasedOnBratsClassesd

ToNumpyd

DeleteItemsd

SqueezeDimd

DataStatsd

SimulateDelayd

CopyProperties

CopyItemsd

ConcatItemsd

Lambdad

MaskImage

LabelToMaskd

FgBgToIndicesd

ToTensord

now needed at end of pre-transforms

Dataset and DataLoader for v4.0

ImagePipelines from Clara with TF have now been replaced with Dataset and DataLoader. For more information see: https://docs.monai.io/en/latest/data.html

Loss functions for v4.0

“Dice” is now “DiceLoss” and “BinaryClassificationLoss” can now be replaced with “BCEWithLogitsLoss”. For more, see Loss functions in MONAI <https://docs.monai.io/en/latest/losses.html>.

Handlers for v4.0

Handlers are very important for extending the functionality of the training engine, from LrScheduleHandler to modify the learning rate, ValidationHandler to specify the frequency validation during training, to all kinds of stats and metrics that can be customized base on the use case. See MONAI handlers for more information and the example MMARs for how some are used.

Metrics for v4.0

Metrics now conform to how they are in PyTorch Ignite, so they must conform to the signature expected by the PyTorch Ignite metrics.

This changes how they must be configured in the json configs, so please see the example MMARs to see how that is done.