ai4med.workflows.evaluators package

class BulkEvaluator(data_dict_list: ( , ) , data_prop: ai4med.common.data_prop.DataProperty, model_loader: ai4med.components.model_loaders.model_loader.ModelLoader, inferer: ai4med.components.inferers.inferer.Inferer, batch_size=1, pre_transforms=None, post_transforms=None, output_writers=None, label_transforms=None, val_metrics=None, do_validation=False, output_infer_result=True, image_key_name='image', overwrite_previous_result=True, image_dtype='float32', label_dtype='float32', data_list_key=None)

Bases: ai4med.workflows.evaluators.evaluator.Evaluator

The BulkEvaluator evaluates a set of data files. It can do both inference and validation.

  • data_dict_list – list of data dicts. Each dict contains an image element and a label element,

  • of which are complete paths to the data files. (both) –

  • model_loader – loader for loading pre-trained model file

  • inferer – inferer for making inference on images

  • batch_size (int) – size of batch. Images are batched for inference. Note that batch_size > 1 may not

  • in all cases. Specifically (work) –

  • does not currently support batch_size > 1. (ScanWindowInferer) –

  • pre_transforms – transforms to be applied to image before inference

  • post_transforms – transforms to be applied to result of inference

  • output_writers – list of output eval_writers

  • label_transforms – transforms to be applied to label

  • val_metrics – list of validation metrics

  • do_validation (bool) – whether or not to do validation

  • output_infer_result (bool) – whether or not to write inference results to disk

  • image_key_name (str) – key name for image data in data dict list

  • overwrite_previous_result (bool) – whether or not to overwrite results from previous runs

  • image_dtype (str) –


Implements the required ‘evaluate’ method of evaluator.

Performs inference and/or validation on data samples defined in the data dict list.


inputs – not used


class Evaluator

Bases: abc.ABC

abstract close()
abstract evaluate(inputs)

Evaluate the inputs to generate output :param inputs: input values to be evaluated :return: evaluation result

© Copyright 2020, NVIDIA. Last updated on Feb 2, 2023.