Skip to navigation

Understand Prediction Results

View as Markdown

client.predict() returns a PredictionResult, never a bare DataFrame. to_pandas() gives you the DataFrame.

from sdm_client import SDMClient, KumoRelational
with SDMClient(url="http://localhost:8000") as client:
result = client.predict(request, model=KumoRelational())
print(result.to_pandas().columns)
print(result.to_pandas().head())

Result fields

The fields that are present depend on the prediction task and the requested outputs.

FieldWhen presentDescription
row_indexAll resultsZero-based position of the row within the (possibly batched) request. Always present.
idWhen the model has an entity keyEntity identifier from the prediction input, in the schema’s declared dtype.
predictionBinary/multiclass classification, regression, ranking, and forecastingPredicted value. For regression, the predicted median unless prediction_statistic="mean" was requested. For ranking, the ranked candidates, in order.
probability_<class>Classification probabilitiesProbability for the named class, e.g. probability_True.
quantile_<level>Quantile outputsPredicted value at the specified quantile level.
scoresScore outputs, where the deployment produces themOrdered score values returned by the model, as a list column.
rankingsRanking resultsEach ranked candidate paired with its score, as a list column.
embeddingsWhen requested with return_embeddings, where the deployment produces themModel embedding values, as a list column.
forecast_stepForecast resultsOne-based sequence number of the forecast period.
explanationWhen outputs includes 'explanation', where the deployment produces oneRead result.explanations.gradients and .columns, not a prediction DataFrame column.

Do not assume that every task returns every field or one row per input entity. Binary and multiclass classification return one row per entity, forecasting returns one row per forecast timeframe, and a RANK TOP K query returns one row per entity with prediction/rankings holding the candidates as a list — three entities ranked TOP 3 come back as three rows, not nine. Inspect the DataFrame columns or client.capabilities(model) before processing optional outputs.

A ranking can hold fewer than k candidates

A ranking asks one yes-or-no question per candidate — will this entity link to it in the window? If no entity in a candidate’s sampled context linked to it, that question has only one answer in context, and the endpoint refuses it. The client leaves that candidate out rather than failing the whole ranking: it gets no score and appears in no row’s prediction or rankings.

with SDMClient(url="http://localhost:8000") as client:
result = client.predict(request, model=KumoRelational())
result.unscored_candidates # (13, 14), or () when every candidate was scored

A candidate’s context is shared by every entity in the ranking, so a candidate left out for one entity is left out for all, and result.unscored_candidates lists them in the order you passed them. The client warns once, naming the candidates it left out. If every candidate is left out, there is nothing to rank and predict() raises UnsupportedRequestError before sending anything.

A larger num_context_samples can reach entities that did link to a candidate. On sparse data — a catalogue where most products sell rarely — it often can’t.

When you can provide a table with one row per (entity, candidate) pair, ask one classification over the pairs instead of a RANK TOP K ranking. Each pair is scored from the features of its entity and its candidate, and the context pools every pair, so a candidate nobody linked to recently is still scored — with one classification request, batched if needed, rather than one call per candidate.

from sdm_client import Schema, Relationship, PredictiveQuery
schema = Schema.from_source(
source,
primary_keys={
"customers": "customer_id",
"products": "product_id",
"orders": "order_id",
"pairs": "pair_id",
},
time_columns={"orders": "ordered_at"},
relationships=[
Relationship("orders", "pair_id", "pairs", "pair_id"),
Relationship("pairs", "customer_id", "customers", "customer_id"),
Relationship("pairs", "product_id", "products", "product_id"),
],
)
# pair_id is '<customer_id>_<product_id>'; rank customer 42's candidates.
request = PredictiveQuery(
"PREDICT COUNT(orders.*, 0, 30, days) > 0 FOR EACH pairs.pair_id", schema,
).materialize(indices=[f"42_{product}" for product in candidates])
with SDMClient(url="http://localhost:8000") as client:
scores = client.predict(request, model=KumoRelational()).to_pandas()
ranking = scores.sort_values("probability_True", ascending=False)

The events table needs a pair_id column linking each event to its pair; the pairs table should hold the pairs you want scored, plus enough others — some that converted, some that didn’t — for the context to learn from.

What a regression reports

The relational model predicts a distribution for a regression; prediction is its median by default, matching what the engine returns. Pass materialize(..., prediction_statistic="mean") for the mean instead — the NIM averages the model’s predicted quantiles to compute it, which trims the tails.

Choose "mean" when predictions will be summed across entities, such as total demand across products: means add up and medians don’t, so on a right-skewed target the sum of the medians understates the total. Neither statistic is pinned to a floor of 0 on a zero-inflated target, because predicted quantiles aren’t clipped to the target’s range.

A regression on the tabular model always reports the mean and doesn’t take prediction_statistic.

Joining

Every row carries row_index. It also carries id when the model has an entity key.

Join on id when it is present, otherwise on row_index. Never on row order. The endpoint is not required to return rows in the order they were sent, and a request split across several calls will not preserve it either.

Checking whether a requested field arrived

with SDMClient(url="http://localhost:8000") as client:
result = client.predict(request, model=KumoRelational(), outputs=["prediction", "quantiles"])
missing = result.absent(["prediction", "quantiles"])

A field the client declares for a model can still be refused by a particular deployment — see Prediction Types. sdm-client warns when a field you asked for by name is absent from the answer.

Explanations

On a compatible client and deployment, EXPLAIN or the explanation output requests per-entity gradients and labelled-context cohorts in result.explanations. result.to_pandas() is unaffected. See Prediction Explainability.