Skip to navigation

Prepare Your Data

View as Markdown

A Schema holds your table, what each column means, and — for Kumo Tabular — no relationships. One table is not a special case of a graph; a single-table model is the shape a Schema degenerates to, which is why sdm-client does not call it a graph.

You rarely build a Schema directly. Request(context=..., predict=..., target=..., task=...) and PredictiveQuery(text, {"table": df}) both infer one for you. Build one yourself only when you need to inspect or correct what was inferred.

Kumo Tabular reads a table already in memory. Schema.from_source(), which samples a warehouse or database in place, is for the relational model only — materialize() refuses it for a tabular question. Read your table into a DataFrame first with sdm_connectors.read().

Infer a schema

from sdm_client import Schema
schema = Schema.from_data({"users": users_df})
schema["users"].primary_key = "user_id"
schema["users"]["churn"].stype = "categorical"

Everything is inferred — dtype, semantic type, primary key, time column. You override only where inference is wrong.

Required characteristics

  • A stable identifier, when your rows have one. A column shaped like a key (*_id, *_key) whose values are nearly all distinct is inferred as the primary key and typed ID. Kumo Tabular does not require one: a request may score rows with no key at all, and results are then joined on row_index.
  • Consistent dtypes. Set a column’s pandas dtype before building the schema; correcting a physical dtype after inference is not supported the way correcting a semantic type is.
  • At least one labelled row. context must contain at least one row with a non-null value in the target column.

Semantic types

Semantic typeUse forCompatible dtypes
IDPrimary keys and other row identifiersInteger, floating-point, string
numericalQuantities and continuous or discrete measurementsInteger, floating-point
categoricalBoolean values or one label from a finite setbool, numerical types, string
textFree-form natural languagestring
timestampEvent times and validity boundariestimestamp[us]
multicategoricalZero or more labels associated with one rowstring; list-valued dtypes are declared but not yet encodable
sequenceFixed-length numerical vectors or embeddingsstring; list-valued dtypes are declared but not yet encodable

Kumo Tabular cannot read a text column. If a column is typed text, the adapter sends it as categorical instead and warns. If free-form text is the signal you want the model to use, this is a hard limit of the current NIM, not a client-side default you can change.

Inference reads values, so a code stored as a number reads as a number

A postcode, a store number, or a status code stored as an integer is inferred numerical, and the model treats its distances as meaningful. No amount of inference settles that — only you know the column is a label:

schema["users"]["zip_code"].stype = "categorical"

Above 64 distinct values, a string column is read as text

A string column is inferred categorical when it holds few enough distinct values and text otherwise. The threshold is a property of the column, not of how many rows happened to be read — but if your frame is a sample of a larger table, say what the column really is rather than letting a bigger frame change the inference later:

schema["users"]["segment"].stype = "categorical" # or "text"

This applies to your target column too. A classification target with more than 64 classes is inferred text, and Kumo Tabular cannot read it as one. Declare it categorical explicitly if that happens. A 0/1 integer target is read as a classification label, not a regression, as long as it holds only zeros, ones, and at most one missing value.

What Kumo Tabular refuses

A schema with more than one table, or with a relationship, is refused before anything is sent — Kumo Tabular reads no relationships. Drop the relationship, or use model=KumoRelational() to gather the same tables as a graph instead.

Next, make predictions with the schema you built, or let Request/PredictiveQuery infer one for you.