> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/sdgm/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/sdgm/_mcp/server.

# Batch Prediction

> Score Kumo Relational entities with built-in bounded batching and retries

Use a single inference call for up to 1,000 entity IDs. For larger jobs, configure `predict()` to split the IDs into bounded batches and retry failed batches.

| Method                     |                 Scale | Use case                                    |
| -------------------------- | --------------------: | ------------------------------------------- |
| Explicit entity in PQL     |                     1 | Query validation and debugging              |
| `indices` without batching |           Up to 1,000 | Small and medium scoring jobs               |
| `batch_size`               | More than 1,000 total | Larger scoring jobs with automatic batching |

“More than 1,000 total” does not mean that individual batches are unbounded. Total duration, NIM capacity, connector sampling cost, client memory, and result storage still constrain the job.

## Submit one batch

```python
from kumo_relational_client import RelationalClient

with RelationalClient(url="http://localhost:8000", max_retries=3) as client:
    result = client.relational(graph).predict(
        "PREDICT COUNT(orders.*, 0, 30, days) "
        "FOR EACH customers.customer_id",
        indices=customer_ids[:1000],
        run_mode="fast",
    )
```

## Process a larger list

```python
with RelationalClient(url="http://localhost:8000") as client:
    result = client.relational(graph).predict(
        "PREDICT COUNT(orders.*, 0, 30, days) "
        "FOR EACH customers.customer_id",
        indices=customer_ids,
        run_mode="fast",
        batch_size="max",
        num_retries=3,
    )
```

Set `batch_size` to a positive integer to specify an explicit batch size. The value `"max"` uses the maximum batch size for the task: 1,000 entities for standard prediction tasks and 200 entities for link-prediction tasks. The `num_retries` parameter controls how many times `predict()` retries each failed batch.

For a multi-batch job, the SDK can upload relational context once and reuse the resulting session instead of sending the same context with every batch. Set `KUMO_RELATIONAL_DISABLE_SESSIONS` to `1`, `true`, `yes`, or `on` (case-insensitive) to force stateless per-batch uploads without changing prediction semantics. Setting `random_seed=None` also disables session reuse because each unseeded batch resamples its neighborhood.

For production jobs, record the prediction settings in durable state. A retry after a timeout can run inference again, so downstream writes should be idempotent and keyed by `ENTITY` plus `ANCHOR_TIMESTAMP` when that column is present.