-
class
ComputeAUC(name, field, label_field, report_path=None, auc_average='macro', class_index=None, do_summary=True, do_print=True, is_key_metric=False, include_list=None) Bases:
ai4med.components.metrics.metric.MetricThis is a wrapper over the MetricAUC library class.
-
class
ComputeAverage(name, field, invalid_value=nan, report_path=None, do_summary=True, do_print=True, is_key_metric=False, negate_value=False) Bases:
ai4med.components.metrics.metric.MetricThis is a wrapper over the MetricAverage library class.
-
class
ComputeAverageDice(name, field, label_field='label', is_2d=False, remove_bg=True, logit_thresh=0.5, is_one_hot_targets=False, report_path=None, do_summary=True, do_print=True, is_key_metric=False) Bases:
ai4med.components.metrics.metric.MetricThis is a wrapper over the MetricAverageFromArrayDice library class.
-
class
ComputeMulticlassAverage(name, field, label_field, report_path=None, auc_average='macro', label_index=None, do_summary=True, do_print=True, is_key_metric=False) Bases:
ai4med.components.metrics.metric.MetricThis is a wrapper over the MulticlassAverage library class.
-
class
Metric(metric: ai4med.libs.metrics.metric.Metric, field, label_field=None, do_summary=True, do_print=True, is_key_metric=False) Bases:
objectA Metric Component computes aggregated values for some aspect of the model predictions.
Metric components are used in two cases: validation during training, and evaluation on trained model. In both cases, we want to see how the model performs by evaluating a set of Metric Components against a validation dataset. For each subject in the dataset, model is evaluated to produce predictions. Metric components are then called to compute a value based on the predictions and labels. The per-subject values are collected/accumulated. After all subjects are evaluated, accumulated values are aggregated to produce the final result.
Any number of metric components can be used. For training validation, one and only one component must be designated as the ‘key metric’, whose value will be used as the overall score of the current snapshot of the model. The snapshot that has the highest score will be saved as the final result of the training workflow.
NOTE: the Metric Component class is just a wrapper over a Metric Library Class that implements the actual computation. The Component Class interacts with the MetricContext to get input values based on configured field names, call the Library Metric component with the input values to perform metric computation.
- Parameters
metric – the library metric object
field (str) – name of the field containing a prediction value in MetricContext
label_field (str) – name of the label field in MetricContext
do_summary (bool) – whether to do summary on this metric (record in Tensorboard)
do_print (bool) – whether to print metric value to console
is_key_metric (bool) – whether this is the key metric
-
generate_report() Puts metric values in a report file. Each metric determines what values (individual, aggregated, or both) should be included in the report, the report’s file name, and creates the file.
Returns:
-
get() Get the aggregated value of the metric. Usually called at the end of the round of model validation for the final result.
Returns: aggregated result
-
name() Get the name of the metric.
Returns: name of the metric
-
reduce_across_ranks() Gather results from all processes into rank 0. Used in multi-GPU training, where the same metric is evaluated in multiple processes. Process of rank 0 is responsible for computing and reporting the final result of the metric.
Returns:
-
reset() Reset the metric to its initialized state. Called before a new round of model validation starts.
Returns:
-
update(metric_ctx: ai4med.common.metric_ctx.MetricContext, filename='') Update the metric with the current prediction and label values in the metric context.
- Parameters
metric_ctx – contextual data for metric computation
filename – file name of the training subject.
used for metric reporting purpose (Usually) –
Returns: