Configuration | NVIDIA Kumo

Configure predictions by passing settings directly to PredictiveQuery.materialize(...) and to client.predict(...). The following sections describe the available settings, their defaults, and how to use them.

There is no run_mode in sdm-client. Set num_context_samples directly on materialize() — the client derives what the endpoint needs from that number instead of a preset.

Neighborhood sampling

Kumo Relational samples a relational neighborhood around each prediction entity. By default, it samples two hops.

num_neighbors controls both the number of hops and the maximum neighbor fan-out at each hop:

The number of entries determines the number of hops.
Each entry specifies the maximum number of neighbors sampled at that hop. For example, num_neighbors=[16, 8, 4] requests three hops, with fan-outs of 16, 8, and 4.
Kumo Relational supports at most six hops.

request = query.materialize(
    indices=[870],
    num_neighbors=[16, 8, 4],
)

num_hops accepts values from 1 through 6. When you provide num_neighbors, its length determines the sampling depth and takes precedence over num_hops.

Larger fan-outs and deeper neighborhoods can substantially increase sampling time, memory use, and request size.

Context size

num_context_samples sets the number of labeled context rows to gather. Omit it on a relational request and you get the smaller budget, up to 1,000 examples. Ask for more, up to 10,000, and the client uses the wider budget the endpoint offers.

request = query.materialize(
    indices=[870],
    num_context_samples=500,
)

The number is honored, not rounded up to a bucket. Ask for more than exists and you get what exists; ask for more than 10,000 and you’re told the ceiling.

When narrowing applies to a temporal question, it keeps the newest anchors, not a random sample across time. The engine gathers a block of entities per anchor date, newest first, and a narrowed count slices from the front: the result is full coverage of the most recent anchors and no coverage of older ones, not a thinned sample across the whole span. Entities within one anchor block are sampled, and that part does follow random_seed.

Materialize options

The following are keyword arguments to PredictiveQuery.materialize(...).

Setting	Meaning
`indices`	Entity primary keys for which to generate predictions. Up to 100,000 per call; gathered as several NIM calls automatically when there are more than one call scores.
`candidates`	The rows a ranking is over. Required for `RANK TOP K`; one call per candidate.
`task`	The kind of question being asked, for the ambiguous single-table case. Refused on a relational query, which the engine already resolves.
`num_context_samples`	Maximum labeled context rows to gather.
`model`	Which model to gather the request for. Defaults to the schema’s shape.
`num_neighbors`	Maximum neighbor fan-out for each hop.
`num_hops`	Sampling depth, 1 through 6. `num_neighbors` takes precedence.
`anchor_time`	Prediction reference time.
`context_anchor_time`	Maximum anchor time for context examples, independent of `anchor_time`.
`use_prediction_time`	Includes the prediction anchor timestamp as an input feature.
`lag_timesteps`	Number of previous target windows to include as lagged features.
`random_seed`	Seed used for pseudo-random sampling and request materialization.
`max_pq_iterations`	Maximum iterations used to collect valid labels for context examples.
`exclude_cols_dict`	Columns to leave out of the sampled context, per table.
`return_embeddings`	Includes the model embedding for each prediction example, where the deployment produces one.
`top_k`	Overrides the `RANK TOP K` written in the query, when the query does not set it.
`prediction_statistic`	What a regression’s `prediction` reports of its predicted distribution: `"median"` (default) or `"mean"`.
`rows_per_call`	How many entities one call scores. Defaults to the endpoint’s own cap (10,000, or 200 for temporal link prediction) and may only be lowered.
`inference_config`	Task-specific model settings, described below.

import pandas as pd
request = query.materialize(
    indices=[870],
    num_context_samples=500,
    num_neighbors=[16, 8, 4],
    anchor_time=pd.Timestamp("2025-06-01T00:00:00Z"),
    context_anchor_time=pd.Timestamp("2025-03-01T00:00:00Z"),
    use_prediction_time=False,
    lag_timesteps=3,
    random_seed=7,
)

Unknown materialize options are rejected during materialization, naming the accepted set.

Inference configuration

Pass inference_config as a dictionary. These fields are forwarded to the deployment, and acceptance is deployment dependent — a deployment may reject any value that differs from its own configured default, so check what a deployment accepts before setting any of them. The relational engine defaults to num_estimators=1.

request = query.materialize(indices=[870])

# Only set a field your deployment is known to accept:
request = query.materialize(
    indices=[870],
    inference_config={
        "num_estimators": 1,  # must match the deployment's own configured value
    },
)

Accepted fields

Field	Meaning
`num_estimators`	Number of estimators in the ensemble.
`column_shuffle`	Varies column order across estimators.
`category_shuffle`	Varies category order across estimators.
`hop_shuffle`	Varies subgraph depth across estimators.
`class_shuffle`	Varies class order across estimators, for classification.

Some deployments accept only their own configured num_estimators value and reject every shuffle flag. Check a deployment’s accepted values before setting these, or omit inference_config to use the deployment’s defaults.

Some deployments also do not accept target_transforms or output_type in inference_config, even though earlier clients exposed them for regression and forecasting quantile control. Ask for outputs=[..., "quantiles"] on client.predict() instead to receive quantile predictions as quantile_<level> columns — there is no client-side control over which quantile levels come back.

Output controls

Setting	Meaning
`outputs`	On `client.predict(request, model=..., outputs=[...])`. Which output fields to ask for. Defaults to a task-appropriate set.
`return_embeddings`	On `materialize()`. Includes the model embedding associated with each prediction example, where the deployment produces one.

result = client.predict(
    request,
    model=KumoRelational(),
    outputs=["prediction", "probabilities"],
)

Limits

The endpoint enforces several size limits. materialize() checks them before sending and refuses or samples while naming the limit.

Limit	Model	Value	Checked
Context rows	relational	1,000 by default, up to 10,000 with `num_context_samples`	At `materialize()`
Context rows	tabular	10,000	Sampled at `materialize()`; refused on a direct `Request`
Rows scored per call	relational	10,000, or 200 for temporal link prediction	More are gathered as several calls automatically
Entities one `materialize()` gathers	relational	100,000, in at most 100 calls	This client’s own bound — the endpoint is never asked for more than one call’s worth
Calls a question asked in parts makes	relational	200 — one per candidate (ranking) or step (forecast)	`sdm_client.decompose.MOST_PARTS`
Request body	relational	30 MiB per call	At `materialize()`
Request body, through Databricks Model Serving	relational	16 MB per call, the gateway’s own limit	At send time, on the wrapped envelope
Explanation	relational	Every scored entity within ordinary request limits, on a compatible client and deployment; see Explainability	Before sending
Hops	relational	6	At `materialize()`
Tables	relational	15	Against the graph the request carries

A request for more entities than one call scores is not refused: materialize() gathers it in batches of at most that many, predict() sends one call per batch, and the results come back as one PredictionResult. A ranking or a forecast is a different kind of “several calls” — one per candidate or step — and is bounded separately by MOST_PARTS (200); it cannot additionally be batched, so a ranking over 200 candidates and 50,000 entities is refused rather than silently sent as 200 × 50 calls.

Pass rows_per_call to ask in smaller batches than the default; it only ever goes down. Reach for it when a graph is wide enough that a full batch outruns the 30 MiB body, or your deployment’s cap is lower than this client’s default.

A call that fails ends the request, but the answers already paid for are not discarded: they’re carried on the error as details['answered'] and details['rows_answered'] (or details['parts_answered'] for a forecast), so you can ask again for only the rest.

Choose settings conservatively

Start with the defaults. Change one dimension at a time and measure the effect on latency, request size, and prediction quality with representative data.

If a request becomes too large, first reduce num_neighbors or num_context_samples, then rows_per_call. You can also remove unused columns or tables. See Batch Prediction for scoring jobs beyond what one materialize() call handles.