ai4med.components.metrics package

Submodules

ai4med.components.metrics.auc module

class ComputeAUC(name, field, label_field, report_path=None, auc_average='macro', class_index=None, do_summary=True, do_print=True, is_key_metric=False)

Bases: ai4med.components.metrics.metric.Metric

This is a wrapper over the MetricAUC library class.

ai4med.components.metrics.avg module

class ComputeAverage(name, field, invalid_value=nan, report_path=None, do_summary=True, do_print=True, is_key_metric=False, negate_value=False)

Bases: ai4med.components.metrics.metric.Metric

This is a wrapper over the MetricAverage library class.

ai4med.components.metrics.avg_from_array_dice module

class ComputeAverageDice(name, field, label_field='label', is_2d=False, remove_bg=True, logit_thresh=0.5, is_one_hot_targets=False, report_path=None, do_summary=True, do_print=True, is_key_metric=False)

Bases: ai4med.components.metrics.metric.Metric

This is a wrapper over the MetricAverageFromArrayDice library class.

ai4med.components.metrics.classification module

class ComputeMulticlassAverage(name, field, label_field, report_path=None, auc_average='macro', label_index=None, do_summary=True, do_print=True, is_key_metric=False)

Bases: ai4med.components.metrics.metric.Metric

This is a wrapper over the MulticlassAverage library class.

ai4med.components.metrics.metric module

class Metric(metric: ai4med.libs.metrics.metric.Metric, field, label_field=None, do_summary=True, do_print=True, is_key_metric=False)

Bases: object

A Metric Component computes aggregated values for some aspect of the model predictions.

Metric components are used in two cases: validation during training, and evaluation on trained model. In both cases, we want to see how the model performs by evaluating a set of Metric Components against a validation dataset. For each subject in the dataset, model is evaluated to produce predictions. Metric components are then called to compute a value based on the predictions and labels. The per-subject values are collected/accumulated. After all subjects are evaluated, accumulated values are aggregated to produce the final result.

Any number of metric components can be used. For training validation, one and only one component must be designated as the ‘key metric’, whose value will be used as the overall score of the current snapshot of the model. The snapshot that has the highest score will be saved as the final result of the training workflow.

NOTE: the Metric Component class is just a wrapper over a Metric Library Class that implements the actual computation. The Component Class interacts with the MetricContext to get input values based on configured field names, call the Library Metric component with the input values to perform metric computation.

Parameters
  • metric – the library metric object

  • field (str) – name of the field containing a prediction value in MetricContext

  • label_field (str) – name of the label field in MetricContext

  • do_summary (bool) – whether to do summary on this metric (record in Tensorboard)

  • do_print (bool) – whether to print metric value to console

  • is_key_metric (bool) – whether this is the key metric

generate_report()

Puts metric values in a report file. Each metric determines what values (individual, aggregated, or both) should be included in the report, the report’s file name, and creates the file.

Returns:

get()

Get the aggregated value of the metric. Usually called at the end of the round of model validation for the final result.

Returns: aggregated result

name()

Get the name of the metric.

Returns: name of the metric

reduce_across_ranks()

Gather results from all processes into rank 0. Used in multi-GPU training, where the same metric is evaluated in multiple processes. Process of rank 0 is responsible for computing and reporting the final result of the metric.

Returns:

reset()

Reset the metric to its initialized state. Called before a new round of model validation starts.

Returns:

update(metric_ctx: ai4med.common.metric_ctx.MetricContext, filename='')

Update the metric with the current prediction and label values in the metric context.

Parameters
  • metric_ctx – contextual data for metric computation

  • filename – file name of the training subject.

  • used for metric reporting purpose (Usually) –

Returns:

Module contents