> This page is for Structured Data Models Client.

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/sdgm/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/sdgm/_mcp/server.

# Improve Model Performance

> Evaluate and tune Kumo Relational with key-aligned historical backtests

Improve a Kumo Relational workflow by changing one factor at a time and evaluating each configuration on the same entities, labels, and historical anchor times.

## 1. Define the task and ground truth

This example predicts total sales for each `article_id` during the seven days after an anchor time. It includes only articles with sales during the preceding seven days.

```python
import pandas as pd

def total_price_next_7_days(transactions, anchor_time):
    data = transactions.copy()
    data["t_dat"] = pd.to_datetime(data["t_dat"])
    anchor = pd.Timestamp(anchor_time)

    prior = data["t_dat"].between(
        anchor - pd.Timedelta(days=7), anchor, inclusive="right"
    )
    eligible = data.loc[prior].groupby("article_id")["price"].sum()
    eligible_ids = eligible[eligible > 0].index

    future = data["t_dat"].between(
        anchor, anchor + pd.Timedelta(days=7), inclusive="right"
    )
    return (
        data.loc[future]
        .groupby("article_id")["price"]
        .sum()
        .reindex(eligible_ids, fill_value=0.0)
        .rename("actual_value")
        .reset_index()
    )
```

Verify that the interval boundaries match the business definition before comparing model configurations.

## 2. Align PQL with the evaluation population

```sql
PREDICT SUM(transactions.price, 0, 7, days)
FOR EACH article.article_id
WHERE SUM(transactions.price, -7, 0, days) > 0
```

The filter keeps the in-context population aligned with the entities used to construct labels.

## 3. Run a key-aligned backtest

```python
import pandas as pd
from sklearn.metrics import mean_absolute_error
from sdm_client import PredictiveQuery, SDMClient, KumoRelational

def score_predictions(client, schema, query_text, labels, **materialize_kwargs):
    query = PredictiveQuery(query_text, schema)
    request = query.materialize(
        indices=labels["article_id"].tolist(),
        anchor_time=pd.Timestamp("2020-09-07T00:00:00Z"),
        **materialize_kwargs,
    )
    result = client.predict(request, model=KumoRelational())
    predictions = result.to_pandas()
    scored = labels.merge(
        predictions[["id", "prediction"]],
        left_on="article_id",
        right_on="id",
        how="inner",
        validate="one_to_one",
    )
    if len(scored) != len(labels):
        raise ValueError("Some labeled articles did not receive a prediction")
    return mean_absolute_error(scored["actual_value"], scored["prediction"])

with SDMClient(url="http://localhost:8000") as client:
    baseline_mae = score_predictions(
        client, schema, query_text, labels, num_context_samples=1000
    )
```

Joining by `article_id`/`id` prevents a silent metric error when response order differs from label order. If predictions will be summed, score with `prediction_statistic="mean"` instead of the default median — see [Prediction Results](/sdgm/kumo-relational/user-guide/work-with-prediction-results/prediction-results#what-a-regression-reports).

## 4. Tune one factor at a time

Compare configurations while keeping the evaluation set fixed:

1. Align the PQL target and filters with the production population.
2. Compare different `num_context_samples` values.
3. Test task-relevant neighborhood settings (`num_neighbors`, `num_hops`) directly on `materialize()`.
4. Test `use_prediction_time` only when calendar time can contain a useful signal.
5. Remove noisy columns or add well-justified relational signals.

Do not assume that a larger context or neighborhood will always improve accuracy. Measure latency, request size, and the selected task metric on representative data.

## 5. Report reproducible results

Record the dataset revision, entity set, anchor times, label construction, SDK and NIM versions, query, materialize/predict settings, metric, and code revision.