Skip to navigation

Limits

View as Markdown

The endpoint enforces a size on the context. Where the client can tell in advance, it refuses first and names the limit — a rejected request otherwise comes back as an opaque error that names neither the limit nor how to get under it.

Sizes

LimitValueChecked
Context rows10,000Sampled down at materialize(), with a report; refused outright on a directly-built Request that exceeds it

The cap was measured at the boundary rather than read from a document: 10,000 context rows are answered, and 10,001 come back as 422 UNSUPPORTED_SHAPE N_train 10001 exceeds limit 10000.

There is no run_mode

Earlier structured-data clients exposed a debug/fast/normal/best preset over the context size. sdm-client has no such preset — you set the number of context rows you want directly:

query.materialize(indices=[...], num_context_samples=500)

The number is honoured, not rounded up to a bucket. Ask for 500 and 500 labelled examples are gathered.

Omit num_context_samples on a single-table request and you get every labelled row, up to the 10,000-row cap; past that, a sample of 10,000 is taken and the client tells you so. Ask for more than exists and you get what exists — that is not an error. Ask for more than 10,000 and you are told the ceiling rather than silently given fewer. Zero, a negative number, a fraction, or a string are refused by name.

This cap applies to materialize(). A directly-built Request(context=..., predict=...) sends whatever context you pass, and the endpoint enforces the same 10,000-row cap on it — refused rather than trimmed, since the direct builder does not sample.

Which examples you get when the count narrows

The first n of what was gathered. Candidates are shuffled first, so narrowing to n is a sample of the labelled rows, not the first n rows of your table in their original order.

What a deployment serves

client.capabilities(model) describes this client — what it can build and send — and is not read from the endpoint. client.models() is read from the endpoint’s /v1/models, so it is the endpoint’s own answer for which model ids it serves. /v1/models is a NIM runtime route rather than part of the Universal TFM contract itself, so an endpoint that does not serve it gives a clear error rather than a wrong answer.