Explainability

KumoRFM explanations provide insights into how predictions are made by offering two complementary perspectives. The global view (cohorts) reveals broad patterns across all in-context examples, showing which data characteristics drive predictions. The local view (subgraphs) highlights specific values in an individual entity’s data neighborhood that influenced its prediction. Together, these views answer: “What patterns does the model see overall?” and “Why did this specific entity get this prediction?”

Use explanations to understand model reasoning, debug unexpected results, communicate insights to stakeholders, or validate that the model is learning meaningful patterns rather than spurious correlations.

Getting Started

Enable explanations by passing an ExplainConfig object to the predict method:

1 from kumoai.rfm import ExplainConfig
2 
3 # Full explanations
4 explanation = model.predict(
5     query="PREDICT COUNT(returns.*, 0, 30, days) > 0 FOR orders.order_id=333",
6     explain=True
7 )
8 
9 # Skip natural language summary for reduced latency
10 explanation = model.predict(
11     query="PREDICT COUNT(returns.*, 0, 30, days) > 0 FOR orders.order_id=333",
12     explain=ExplainConfig(skip_summary=True)
13 )

The return type changes from a DataFrame to an Explanation object with three components:

1 # Access the prediction scores
2 prediction_df = explanation.prediction
3 
4 # Access global column analysis
5 cohorts = explanation.details.cohorts
6 
7 # Access local subgraph with attribution scores
8 subgraph = explanation.details.subgraphs[0]
9 
10 # Access natural language summary (if skip_summary=False)
11 summary = explanation.summary

Global View: Column Analysis (Cohorts)

The cohort analysis reveals how different column values correlate with prediction outcomes across all in-context examples used by the model. This global perspective helps identify which features are generally predictive.

Structure

Each cohort object contains:

table_name: The table being analyzed (e.g., 'orders', 'users')
column_name: The column or aggregate (e.g., 'age', 'COUNT(*)')
hop: Distance from entity table (0 = entity itself, 1 = directly linked, 2 = second-degree neighbors)
stype: Semantic type (numerical, categorical, timestamp, None for aggregates)
cohorts: List of value ranges (numerical/timestamp) or categories
populations: Proportion of in-context examples in each cohort (sums to 1.0)
targets: Average prediction score for examples in each cohort

High-impact columns show large variance in targets across different cohorts. Compare cohort values to identify risk factors and protective factors.

Example: Predicting order returns

1  # orders.date at hop 0 (the order being predicted) 
2 {     
3 	'table_name': 'orders',     
4 	'column_name': 'date',     
5 	'hop': 0,     
6 	'cohorts': ['[0 months, 2 months)', '[2 months, 24 months+)'],     
7 	'populations': [0.098, 0.902],     
8 	'targets': [0.020, 0.000],
9 }

What this means: Recent orders (0-2 months old) have a 2% return rate, while older orders have 0% return rate. Order age is highly predictive—the model learned that return policies don’t apply to old purchases.

Calculating Column Importance

To quantify which columns matter most, compute weighted variance of targets:

1 import numpy as np
2 
3 def calculate_importance(cohort):
4     populations = np.array(cohort.populations)
5     targets = np.array(cohort.targets)
6     weighted_mean = np.sum(populations * targets)
7     weighted_variance = np.sum(populations * (targets - weighted_mean) ** 2)
8     return weighted_variance
9 
10 # Calculate for all cohorts and sort
11 importance_scores = [
12     {'column': f"{c.table_name}.{c.column_name}", 'score': calculate_importance(c)}
13     for c in explanation.details.cohorts
14 ]
15 importance_scores.sort(key=lambda x: x['score'], reverse=True)

Note on hops: The same column may appear at different hops. For example, orders.price at hop 0 is the price of the order being predicted, while orders.price at hop 2 represents prices of other orders connected through items or users. These are treated as distinct features in the global analysis.

Local View: Subgraph Attribution

The subgraph shows the actual data neighborhood around the specific entity being predicted, with attribution scores indicating which values influenced the prediction.

Structure

A subgraph contains:

seed_id: Always 0 (the entity being predicted)
seed_table: The entity’s table (e.g., 'orders')
seed_time: The anchor time for prediction
tables: Dictionary mapping table names to nodes

Each node represents a row from a table and contains:

cells: Dictionary of column names to cell objects
value: The actual data value (can be None if missing)
score: Attribution score from 0 to 1 (higher = more influential)
links: Dictionary mapping relationship names to sets of connected node IDs

Important: Node IDs are internal indices (0 to N), not primary keys. The entity being predicted always has ID 0.

Score Interpretation

Attribution scores indicate how much changing a value would change the prediction:

0.00 - 0.05: Negligible influence
0.05 - 0.15: Moderate influence
0.15 - 0.30: Strong influence
0.30+: Critical influence

Example: Order return prediction for order_id=333

1 # Node 0 in 'orders' table (the seed entity) 
2 {     
3 	'cells': {         
4 		'date': {'value': '2018-09-20', 'score': 1.0},
5 	    'price': {'value': 8.98, 'score': 0.291},
6 		'sales_channel_id': {'value': 2, 'score': 0.222}
7 		},     
8 	'links': {         
9 		'item_id->items': {5},         
10 		'user_id->users': {17}	     
11 	}
12 }

What this means: The order date (from 2018) is the strongest signal (score=1.0), indicating age is decisive. Price and sales channel show moderate influence. The order connects to 1 item and 1 user, forming a larger subgraph.

Key insight: Null values can have non-zero scores, meaning missingness itself is informative. For example, a missing email address might correlate with higher churn risk.

Context Example Weights

Alongside the subgraph attribution scores, the model returns a context weight for each in-context example — an importance score that reflects how strongly each example influenced the prediction.

Context weights appear on each subgraph as context_examples:

1 subgraph = explanation.details.subgraphs[0]
2 
3 for ex in subgraph.context_examples:
4     print(ex.entity_id, ex.label, ex.score, ex.seed_time)

Each ContextExample contains:

entity_id: Primary key of the context entity
label: Ground-truth target value for that in-context example
score: Importance score in [0, 1]; the example with the highest score is normalized to 1.0
seed_time: Anchor timestamp for that in-context example (None for static tasks)

Score Interpretation

Scores reflect how much perturbing that example’s representation would change the prediction.

1.00: Most influential example in the context window
0.10 – 0.50: Moderate influence
< 0.05: Negligible influence

Natural Language Summaries

For quick interpretation or stakeholder communication, we generate natural language explanations by default:

1 explanation = model.predict(     
2 		query="PREDICT COUNT(returns.*, 0, 30, days) > 0 FOR orders.order_id=333",     
3 		explain=ExplainConfig(skip_summary=False) # or explain=True
4 )
5 print(explanation.summary)

Example output:

The model predicts this order is very unlikely to be returned (probability near 0%). The primary driver is the order’s age—it was placed over 18 months ago, well beyond typical return windows. Historical patterns show that orders older than 2 months have a 0% return rate, while recent orders (0-2 months) have a 2% return rate. The order’s low price ($8.98) and sales channel also contribute moderately to the low return probability.

Summaries are powered by an LLM that interprets both cohort and subgraph data, making them ideal for reports, dashboards, or explaining decisions to non-technical audiences. But in any case, all the raw data which is summarized can be recovered as shown above and parsed into a desired format as needed.

To learn more about explainability see this example notebook