> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/sdgm/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/sdgm/_mcp/server.

# Classification

> Metrics used to evaluate binary, multiclass, and multilabel classification tasks in Kumo.

> **Prerelease documentation**
>
> This Kumo Studio information is prerelease and is not yet supported. Stay tuned for the Docker and Kubernetes releases.

## Binary Classification

Binary classification tasks are problems with target values that fall into two categories: one *positive* class, one *negative* class.
For example, in a churn prediction problem, you want to predict whether a customer leaves your service in the near future, so you can define the labels as:

* **Positive Class:** the customer discontinues your service.
* **Negative Class:** the customer continues your service.

Whenever you write a predictive query with a target that contains a conditional operation such as `=`, `>`, or `<`, Kumo defines the task as a binary classification task.

The prediction output for a binary classification task is the probability that the target value belongs to the positive class.
A classification threshold determines how high the predicted probability must be to be considered a positive class prediction.
Kumo uses accuracy, AUROC, and AUPRC as the evaluation metrics for this task, and provides a [confusion matrix](#confusion-matrix) and a [cumulative gain chart](#cumulative-gain-chart) to help understand the results.

### Accuracy

**Accuracy** is the percentage of correct predictions: the total number of correct predictions divided by the number of samples.

### AUROC

**AUROC** stands for *area under the receiver operating characteristic curve*: a curve that plots sensitivity against specificity under different classification thresholds.

**Sensitivity** is calculated by:

![image.png](file:images/image.png "image.png")

where TP (true positives) counts customers who stop purchasing and the model correctly predicts this. FN (false negatives) counts customers who stop purchasing but the model predicts they will continue.

**Specificity** is calculated by:

![Formula for specificity: TN divided by TN plus FP](file:images/Screenshot2025-06-27at10.36.54AM.png "Formula for specificity: TN divided by TN plus FP")

where TN (true negatives) counts customers who continue purchasing and the model correctly predicts this, and FP (false positives) counts customers who continue purchasing but the model predicts they will leave.

A perfect predictor yields an AUROC score of `1`; a completely random predictor yields `0.5`.
An AUROC of `0.8` is typically considered very good, though what qualifies as good enough depends on your specific business problem.

### AUPRC

**AUPRC** stands for *area under the precision-recall curve*: a curve that plots precision against recall under different classification thresholds.

**Precision** is calculated by:

![Formula for precision: TP divided by TP plus FP](file:images/Screenshot2025-06-27at10.37.18AM.png "Formula for precision: TP divided by TP plus FP")

where FP (false positives) counts customers who continue purchasing with us but the model predicts they will stop.

**Recall** is calculated by:

![image.png](file:images/image.png "image.png")

A perfect predictor yields an AUPRC score of `1`.
The baseline score varies by problem; specifically, it depends on how imbalanced the data is.
For example, if your problem has 10% positive class and 90% negative class, a predictor that always predicts positive would achieve an AUPRC of `0.1`.

AUPRC is typically a better metric when you have imbalanced data (for example, when predicting a very rare class), since it is more sensitive to changes in performance on that rare class.
By contrast, with a metric such as accuracy, you could achieve very high accuracy simply by predicting that the rare positive class never occurs.

Changing the classification threshold affects accuracy, specificity, sensitivity, precision, and recall scores.
A single AUROC or AUPRC score captures overall performance across all thresholds.

---

## Confusion Matrix

Kumo provides a confusion matrix showing the number of true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN). Read the confusion matrix as follows:

|                       | Predicted Negative Class | Predicted Positive Class |
| --------------------- | ------------------------ | ------------------------ |
| Actual negative class | TN                       | FP                       |
| Actual positive class | FN                       | TP                       |

A good model generally has significantly higher values in the TN and TP cells than in the FN and FP cells.
Always interpret these values in the context of your specific business problem.

---

## Cumulative Gain Chart

The cumulative gain chart shows the percentage of "gain" you can achieve in a certain class by taking a particular action.
For example, suppose that the graph below is the result of a churn prediction query:

![Cumulative gain chart showing model performance versus baseline and ideal curves](/sdgm/_fern-img/933d18361bd5759a056b7562445b658853fea6d3f6a4eebf154185d33219eb25.webp)

The (0.11, 0.16) datapoint means: if you reach out to 11% (the action) of all customers, you find 16% (the "gain") of all customers who leave in the near future.
The better your model, the fewer customers you need to contact to capture a disproportionately larger share of customers likely to churn.

A gain chart plots the TP rate against the rate of positive predictions.
The TP rate is given by:

![image.png](file:images/image.png "image.png")

and the rate of positive predictions is given by:

![Formula for rate of positive predictions: TP plus FP divided by total samples](file:images/Screenshot2025-06-27at10.38.36AM.png "Formula for rate of positive predictions: TP plus FP divided by total samples")

The straight orange line represents the baseline: the result of a random model.

The green curve represents the ideal model: one that is always correct when predicting the positive class.
The blue curve shows your model's actual performance.
A better model has a larger area between the blue and orange curves; the two numbers shown on the graph quantify this difference.

---

## Multiclass Classification

Kumo uses the accuracy score as the metric for multiclass classification problems.
Accuracy is the percentage of correct class predictions: the total number of correct class predictions divided by the number of samples.

---

## Multilabel Classification

For multilabel classification, Kumo calculates the same precision and recall scores for each class as in the binary classification case.
You can also view the F1 score, which takes both precision and recall into consideration, calculated by:

![Formula for F1 score: 2 times precision times recall divided by precision plus recall](/sdgm/_fern-img/fc5adfab9da2741faa76d66b904881ca773319d42de4cff5bf10af52b5954c7e.webp)

Kumo also provides the Hamming score as a measure of accuracy across multiple classes.
The Hamming score is calculated by:

![Screenshot2025 06 27at10 39 35AM Pn](file:images/Screenshot2025-06-27at10.39.35AM.png "Screenshot2025 06 27at10 39 35AM Pn")

which translates to the percentage of correct predictions divided by the total number of predictions.

---

* [Table of Contents](#)
* * [Binary Classification](#binary-classification)
    * [Accuracy](#accuracy)
    * [AUROC](#auroc)
    * [AUPRC](#auprc)
  * [Confusion Matrix](#confusion-matrix)
  * [Cumulative Gain Chart](#cumulative-gain-chart)
  * [Multiclass Classification](#multiclass-classification)
  * [Multilabel Classification](#multilabel-classification)