> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/sdgm/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/sdgm/_mcp/server.

# Improve Model Performance

> Evaluate and tune Kumo Relational with key-aligned historical backtests

Use a historical backtest to improve a Kumo Relational workflow without changing the basis of comparison.
A backtest asks the model to predict from an earlier point in time, then compares those predictions with outcomes that are already known.
Change one factor at a time while keeping the entities, labels, and historical cutoff fixed so that changes in the score reflect the configuration you tested.

Before you begin, load the H\&M `transactions` table into a pandas DataFrame, build its relational `graph`, and configure access to a Kumo Relational endpoint.
The examples reuse those objects throughout the workflow.

## 1. Define the task and ground truth

This example predicts total sales for each `article_id` during the seven days after an anchor time.
The **anchor time** is the historical cutoff that separates the data available to the model from the future period being predicted.
The evaluation includes only articles with sales during the preceding seven days.

```python
import pandas as pd

def total_price_next_7_days(transactions, anchor_time):
    data = transactions.copy()
    data["t_dat"] = pd.to_datetime(data["t_dat"])
    anchor = pd.Timestamp(anchor_time)

    prior = data["t_dat"].between(
        anchor - pd.Timedelta(days=7), anchor, inclusive="right"
    )
    eligible = data.loc[prior].groupby("article_id")["price"].sum()
    eligible_ids = eligible[eligible > 0].index

    future = data["t_dat"].between(
        anchor, anchor + pd.Timedelta(days=7), inclusive="right"
    )
    return (
        data.loc[future]
        .groupby("article_id")["price"]
        .sum()
        .reindex(eligible_ids, fill_value=0.0)
        .rename("actual_value")
        .reset_index()
    )

BACKTEST_ANCHOR_TIME = pd.Timestamp("2020-09-07T00:00:00Z")
labels = total_price_next_7_days(transactions, BACKTEST_ANCHOR_TIME)
```

The resulting `labels` DataFrame contains one known outcome, `actual_value`, for each eligible `article_id`.
Verify that the interval boundaries match your business definition before comparing model configurations.

## 2. Align PQL with the evaluation population

Store the Predictive Query Language (PQL) query in `query` so the same prediction task is used for every configuration:

```python
query = """
PREDICT SUM(transactions.price, 0, 7, days)
FOR EACH article.article_id
WHERE SUM(transactions.price, -7, 0, days) > 0
"""
```

The `WHERE` clause applies the same eligibility rule used to construct `labels`, so the query and ground truth cover the same population.

## 3. Run a key-aligned backtest

This backtest uses mean absolute error (MAE), the average absolute difference between each prediction and its known value.
Lower MAE indicates better predictions for this task.

```python
import pandas as pd
from sklearn.metrics import mean_absolute_error
from kumo_relational_client import RelationalClient

def score_predictions(model, labels, query, run_mode="fast", **settings):
    predictions = model.predict(
        query,
        indices=labels["article_id"].tolist(),
        run_mode=run_mode,
        anchor_time=BACKTEST_ANCHOR_TIME,
        **settings,
    )
    scored = labels.merge(
        predictions[["ENTITY", "PREDICTION"]],
        left_on="article_id",
        right_on="ENTITY",
        how="inner",
        validate="one_to_one",
    )
    if len(scored) != len(labels):
        raise ValueError("Some labeled articles did not receive a prediction")
    return mean_absolute_error(scored["actual_value"], scored["PREDICTION"])

with RelationalClient(url="http://localhost:8000") as client:
    model = client.relational(graph)
    baseline_mae = score_predictions(
        model, labels, query, run_mode="fast"
    )
```

The merge joins predictions to labels by entity key instead of row position.
This prevents a silent scoring error if the service returns predictions in a different order from the labels.

## 4. Tune one factor at a time

Compare configurations in the following order while keeping the evaluation set fixed.
The order starts with the task definition, which often matters more than inference settings:

1. Align the PQL target and filters with the production population.
2. Compare the `fast`, `normal`, and `best` run modes, which are presets for the amount of prediction context and the default relational neighborhood size.
3. Test task-relevant neighborhood settings as direct `predict()` arguments to control how many related rows Kumo Relational samples through table relationships.
4. Test `use_prediction_time` only when the calendar timestamp itself could provide a useful signal, such as seasonality.
5. Remove noisy columns or add well-justified relational signals.

Do not assume that a larger run mode or neighborhood will always improve accuracy.
Measure latency, request size, and the selected task metric on representative data.

### Control the context examples

Use [`predict_task()`](/kumo-relational/user-guide/make-predictions/custom-context) when you need direct control over the labeled examples supplied to Kumo Relational.
These examples provide context that the model uses to make a prediction.
Controlling them lets you test how context selection affects prediction quality without changing the graph or prediction task.

When comparing configurations:

* Keep the context and prediction entities fixed.
* Use labels that would have been available at prediction time.
* Compare the custom-context result with the standard PQL-based workflow.
* Do not assume manually selected context will perform better.
  Evaluate it using the same metric and historical anchor times.

## 5. Report reproducible results

Record the information needed to reproduce and compare each result:

* Dataset and code revisions
* Entity set and anchor times
* Label construction
* SDK and NVIDIA NIM versions
* Query and prediction settings
* Evaluation metric