> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/sdgm/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/sdgm/_mcp/server.

# Batch Prediction

> Score Kumo Relational entities with built-in bounded batching and retries

Batch prediction lets you score a list of Kumo Relational entities, such as customers, with one `predict()` call.
Pass the target entity IDs through `indices`.
For lists above 1,000 IDs, set `batch_size` so the client splits the IDs into bounded batches and can retry failed batches.

| Method                                             |                 Scale | Use case                                    |
| -------------------------------------------------- | --------------------: | ------------------------------------------- |
| Explicit entity in Predictive Query Language (PQL) |                     1 | Query validation and debugging              |
| `indices` without batching                         |           Up to 1,000 | Small and medium scoring jobs               |
| `batch_size`                                       | More than 1,000 total | Larger scoring jobs with automatic batching |

“More than 1,000 total” does not mean that individual batches are unbounded.
Total duration, NVIDIA NIM inference capacity, connector sampling cost, client memory, and result storage still constrain the job.

The examples assume that `graph` is an initialized relational graph and `customer_ids` contains values from `customers.customer_id`.

## Submit one batch

```python
from kumo_relational_client import RelationalClient

with RelationalClient(url="http://localhost:8000", max_retries=3) as client:
    result = client.relational(graph).predict(
        "PREDICT COUNT(orders.*, 0, 30, days) "
        "FOR EACH customers.customer_id",
        indices=customer_ids[:1000],
        run_mode="fast",
    )
```

## Process a larger list

```python
with RelationalClient(url="http://localhost:8000") as client:
    result = client.relational(graph).predict(
        "PREDICT COUNT(orders.*, 0, 30, days) "
        "FOR EACH customers.customer_id",
        indices=customer_ids,
        run_mode="fast",
        batch_size="max",
        num_retries=3,
    )
```

Set `batch_size` to a positive integer to specify an explicit batch size.
The value `"max"` uses the maximum batch size for the task: 1,000 entities for standard prediction tasks and 200 entities for link-prediction tasks.
The `num_retries` parameter controls how many times `predict()` retries each failed batch.

For a multi-batch job, the SDK can upload the related data used for prediction once and reuse the resulting session instead of sending the same relational context with every batch.
Set `KUMO_RELATIONAL_DISABLE_SESSIONS` to `1`, `true`, `yes`, or `on` (case-insensitive) to force stateless per-batch uploads without changing prediction semantics.
Setting `random_seed=None` also disables session reuse because each unseeded batch resamples its neighborhood.

For production jobs, save the prediction settings in persistent storage so you can reproduce a run.
A retry after a timeout can run inference again, so downstream writes should be idempotent: writing the same prediction more than once must not create duplicates or change the result.
Key those writes by `ENTITY` plus `ANCHOR_TIMESTAMP` when that column is present.