Best Practices for Kumo Tabular
Kumo Tabular is an in-context learning model. It does not train on your data. Instead, it reads a set of labeled context rows each time it predicts. Most of the gains come from choosing which rows it reads and from evaluating it fairly, not from tuning the model.
This guide explains how to prepare your data, evaluate predictions, and choose context and model settings for your task. If you’re new to Kumo Tabular, start with the Quickstart.
Best Practices
Keep Context and Evaluation Data Separate
Kumo Tabular does not train, so instead of a training set, split your labeled rows three ways:
- Context pool: the rows the model draws its context from.
- Validation set: the rows you score to compare settings.
- Test set: the rows you set aside until you have chosen your settings. Score it once, at the end, to estimate performance on new data.
Never put validation or test rows in a context. Validation and test rows should also differ from the context pool the way your future rows will, so the scores reflect what happens in production. How should I evaluate Kumo foundation models? covers metrics and calibration.
Pick the split by what you will predict:
- Future rows: split by time. Put the earliest period in the context pool, the next period in validation, and the latest in test. The model is then always scored on rows that come after its context.
- New customers, products, or stores: split by group. Put all rows of each customer in one split. Otherwise the model can recognize a validation customer from that customer’s own context rows, and it scores better than it will on customers it has never seen. If you only predict for existing customers, split by time instead.
- Rows with no time order or groups: split at random. Keep the real label mix in each split. Balancing the classes, covered later, changes only the context, not the validation or test sets.
When the label looks into the future, leave a gap between splits. For a label such as “churns in the next 30 days”, a context row from June 25 gets its label from activity up to July 25. If validation starts on July 1, that label overlaps the validation period. A gap at least as long as the label period, 30 days here, prevents this. A label about the row’s own moment, such as whether a transaction was fraud, needs no gap.
The code on this page assumes a binary classification task, with the label in a column named target that holds 0 and 1. It calls the three sets context_pool_df, validation_df, and test_df.
This helper draws a context from the pool, with the pool’s label mix by default, from a recent window, or with a chosen share of positive rows:
With time_column and days, draw_context keeps only the pool’s most recent days. With positive_share, it samples each class separately, and warns you if the pool has too few rows of a class to reach that share. The same seed draws the same rows. Improve Model Performance covers when to use each option.
Ensure No Leakage
Leakage is a feature that holds information you would not have at the moment you predict. Two common examples:
- An outcome field. In a churn table, a cancellation date is filled in only for customers who already churned, so it gives away the label.
- An aggregate that looks past the prediction. A customer’s total orders for the month, computed at month end, includes orders placed after a prediction made on the 15th. On the 15th, you would only know the orders up to that day.
A simple check is to see how well each feature predicts the label on its own. Compute each feature’s AUROC on the validation set. If a single feature scores above 0.95 by itself, find out why before you keep it. Often its value was recorded after the outcome. Leave ID columns out of the check, because their numbers carry no meaning.
Set the Right Column Types
Kumo Tabular processes each column according to its semantic type, using a recipe. A recipe is the set of steps that turns your columns into the inputs the model expects, and turns the model’s outputs back into predictions on your original scale. The default recipe processes numerical and categorical columns and leaves every other type out. SDM guesses each column’s type from its data type: numbers become numerical, strings become categorical, and dates become datetime. Table Semantics in the SDM guide explains the types.
The guesses most often go wrong in four places:
- A label stored as 0 and 1 becomes
numerical, and the task turns into a regression. Mark itcategorical. - An integer identifier, such as
customer_id, becomes a number, and the model reads meaning into its size. Mark itid. - A code stored as a number, such as a region or product code, gets an order it does not have. Mark it
categorical. - Free text becomes
categorical, with almost every value distinct. Mark ittext.
Missing values need no placeholder, because Kumo Tabular has a dedicated input for missing cells. Fill in a value only when you know what its absence means, such as a missing activity count that means zero.
Here, text="infer" marks string columns with many distinct, multi-word values as text, and id="infer" marks columns whose name contains the word id, such as customer_id, as id. The overrides fix types that SDM cannot guess from the values alone. In this example, they mark the 0 and 1 label as categorical, so the task is classification, and region_code as categorical, because it is a code rather than an amount. Print the result and check every column you know.
The default recipe does not use date and text columns. To use them, add them to the recipe.
Dates. AddCalendarFields turns a date into numerical fields, such as the weekday and month. Add it to the recipe, then pass recipe=recipe when you call the model:
You can also build date features yourself, such as days since the last order, computed as of the moment you predict. With a split by time, use only fields that repeat, such as the weekday. A field that only marks the period, such as the month when your history covers less than a year, takes values in new rows that the context never held.
Text. Add TF-IDF features or sentence embeddings to the recipe in the same way, as Data Processing in the SDM guide describes.
Reduce Variance Between Runs
Two runs of the same setting can score slightly differently. The context is drawn at random, and each estimator reads it with its own random column order, class order, and numerical transform. An estimator is one pass of the model over the context.
This matters most when you compare settings. If the same setting moves by a point or two between runs, a one-point gain from a new setting may be chance. The steps below make your scores repeatable, so a difference you measure comes from the change you made.
- Fix the seeds for the context draw and the model, so the same setting gives the same score on every run.
- Average more estimators, ideally each with its own context draw, as Choose the Number of Estimators describes.
- Score every setting on the same validation rows, with enough positive rows for a stable metric.
- Confirm a difference across three to five seeds. For each seed, compute a bootstrap interval for the difference from your baseline. Keep the setting only when every seed agrees on the sign and each interval excludes zero.
Improve Model Performance
The settings below can help improve your model’s performance. Their effect depends on your table, so test each one against your baseline across seeds, as Reduce Variance Between Runs describes. The first three shape the context. The context is the only information Kumo Tabular has about your table, so it usually matters most.
Balance the Classes
When the positive class is rare, as with fraud or churn, a context with the natural mix holds few positive rows. A higher share of positive rows often helps. Try a few shares on validation, such as 10%, 25%, and 50%, and choose again when you change the context size.
Favor Recent Context
If the patterns in your data change over time, for example after a pricing change or a new product launch, old rows can teach the model behavior that no longer holds. Draw the context from only the most recent rows, using the helper’s recent window, and compare a few window lengths on validation, such as the last three, six, and twelve months. A short window holds fewer rows, so check that it still has enough rows of the rarer class.
Test Larger Contexts
More context rows usually help, until the gain levels off. Test a few sizes on validation, such as 10,000, 30,000, 60,000, and 100,000 rows, and weigh any gain against GPU memory and time.
When choosing a context size, consider both rows and columns:
- Rows: Kumo Tabular was pretrained on contexts of up to 60,000 rows. This is not a maximum context size; use validation to check whether larger contexts improve your results.
- Columns: It was pretrained on up to 100 feature columns, and the default recipe uses at most 500 columns per estimator. Calendar fields and text features add columns, so count them too.
See What are the context-size limits for Kumo Tabular and Kumo Relational? and the GPU Sizing Guide.
Choose the Number of Estimators
Kumo Tabular averages its estimators, and the default is one. Try two. If the metric improves, try four, six, and eight. Gains beyond eight are usually small, while compute grows with every estimator.
Most of the gain comes from estimators that read different rows. By default, every estimator reads the same context. When your pool is larger than one context, give each estimator its own draw. SDM accepts one context per estimator, stacked into a single input, as its model reference describes. Convert all the draws to one table together before you split and stack them. Tables converted separately can number their categories differently and change predictions without any error.
When to Consider Fine-Tuning
Fine-tuning updates the model weights on your table. Fine-tuning gains vary by dataset. Compare the fine-tuned model with the pretrained baseline on validation data before deciding whether to use it. If you try it, do so after the steps above, and keep it only if it beats the model without fine-tuning on the same validation rows across seeds. The fine-tuning guide walks through it.