Prediction Explainability

Request explanations to inspect how a prediction responds to small input changes and how outcomes vary across the labelled examples behind it. The result contains two separate views: model gradients for each scored entity and column cohorts computed by the client.

Request an explanation

Use the schema from Build a Schema, with customers and orders linked by customer ID. Replace the example IDs with customers in your data and use your configured endpoint URL. The query predicts each customer’s order total over the next 30 days.

from sdm_client import PredictiveQuery, SDMClient, KumoRelational
query = PredictiveQuery(
    "EXPLAIN PREDICT SUM(orders.amount, 0, 30, days) "
    "FOR EACH customers.customer_id",
    schema,
)
request = query.materialize(indices=[42, 43])
with SDMClient(url="http://localhost:8000") as client:
    result = client.predict(request, model=KumoRelational())
predictions = result.to_pandas()
gradients = result.explanations.gradients.to_pandas()
columns = result.explanations.columns
cohorts = columns.to_pandas() if columns is not None else None

EXPLAIN adds explanations to the outputs the call would otherwise request. Alternatively, keep PREDICT in the query and pass outputs=["prediction", "explanation"] to client.predict(). For classification probabilities as well, include "probabilities" in that list. Using both forms is allowed but redundant.

Explanations cover every scored entity, including requests split into batches. There is no explanation-specific single-entity restriction or 1,000-row context limit; the ordinary request limits still apply. Ranking queries do not support explanations because the client assembles their answers from separate classification calls.

Work with the result

result.to_pandas() remains the prediction DataFrame. Explanations are separate:

Inspect gradients

For the example above, inspect customer 42 directly:

customer = result.explanations.gradients[42]
related = customer["related"]
print(related.keys())

Each record has two parts:

• outputs: what each gradient is taken of, in order. For classification, there is one entry per class probability. For regression, there is the prediction, then each quantile in the response.
• related: one block per table represented in the returned explanation, including the entity’s own table.

Each block has:

• rows: the entity’s rows in that table, as column-to-value mappings.
• columns: the model’s numerical inputs after preprocessing. These can differ from the fields in rows; for example, a date becomes several numbers.
• gradients: indexed as [output][row][column]. gradients[o][r][c] is how much outputs[o] changes per unit change in columns[c] of rows[r].
• keys: each row’s primary key, when the table has a single-column primary key present in the data sent to the model. Otherwise, identify a row by its values in rows.

A table block with no rows for the entity has an empty rows list. For an orders block with at least two rows and three preprocessed numerical columns, inspect the second row’s third column for the first output:

orders = result.explanations.gradients[42]["related"]["orders"]
orders["gradients"][0][1][2]  # How outputs[0] moves with columns[2] of rows[1].

The lookup key follows the prediction result:

The flattened gradient DataFrame includes table, hop, key, row, output, column, and gradient, alongside the prediction identity. Hop 0 is the entity’s own table; larger hops follow relationships away from it. An unreachable row has no hop value. output identifies a prediction column such as prediction, probability_True, or quantile_0.1.

The singular result.explanation property is deprecated. Use .explanations.gradients and .explanations.columns explicitly.

Inspect column cohorts

Column analysis summarizes observed outcomes in groups of labelled context examples. Check that the analysis is available before converting it to a DataFrame:

analysis = result.explanations.columns
if analysis is not None:
    cohorts = analysis.to_pandas()
    print(cohorts[["table", "hop", "column", "cohort", "class", "share", "mean"]])

Each row describes a cohort for a column at a graph hop and, for classification, a target class. share describes the cohort’s fraction of the analysed population; mean is its observed class rate or average regression target. See Cohort analysis for examples and the population included.

Fitted sessions and forecasts

A fitted session computes column cohorts once, from the pinned labelled rows after holdout rows are removed. Later predictions reuse that analysis. If fit() requested explanations, each prediction carries them; otherwise an EXPLAIN PREDICT query can request them for a prediction of the same fitted question. Adding EXPLAIN does not change the question pinned by the session.

A forecast requests gradients for each step and combines column analysis across all steps’ labelled examples, using each step’s outcome. Use the step in the gradient lookup key to distinguish predictions for the same entity. If only some steps return gradients, the client warns about the unexplained steps.

Troubleshoot explainability

• The endpoint returns UNSUPPORTED_OUTPUT: confirm that the deployed model supports explanations, not just that the client accepts the option.
• Gradients are missing: check result.absent(["explanation"]) and client warnings, especially for a partially explained forecast.
• Column analysis is None: confirm that explanations were requested and inspect SDMWarning messages. A column-analysis failure does not prevent the prediction from returning.
• A ranking request is refused: use the underlying classification question when you need an explanation of a candidate’s score.

See Understand Explanations to interpret the two views.