Kumo Tabular Infrastructure Benchmarks
This page shows how long Kumo Tabular Large takes to fit a context and to predict, and how much GPU memory it uses, on four NVIDIA GPUs.
What the Benchmarks Measure
Unlike a model such as XGBoost, Kumo Tabular has no separate training step. Instead, it takes your labeled rows, called the context, as input when it predicts. SDM splits this work into two calls:
- Fitting the context (
model.fit) prepares the context rows, runs them through the model once, and keeps the result in memory. It is the slower call, but you run it once per context. Fit again when you update the context, for example when your data has drifted or when you want to replace older rows with more recent ones. - Predicting (
model.predict) scores the rows of each request against the prepared context. It runs on every request, so its time adds to every response from a production service.
To serve predictions, fit once and reuse the prepared context for every request, as Reuse Context for Many Queries shows.
The benchmarks record five values for these two calls:
A fit or a prediction fails when its peak memory exceeds the GPU’s memory. Size your GPU by the larger of the two peaks: the fit usually needs more, but large requests can need more than the fit, especially on the T4.
Benchmark Setup
All measurements use Kumo Tabular Large with weights v1.0.1, SDM 0.2.1, PyTorch 2.10, and CUDA 12.8. Each GPU ran on its own AWS instance with 16 CPU cores:
- Data: Synthetic regression tables with numerical features only. They have no categorical or text columns and no missing values. Contexts have 5,000 to 50,000 rows and 20 to 500 feature columns, or 20 to 200 on the T4. Requests have 1 to 1,000 rows.
- Model settings: 1, 4, or 8 estimators. Estimators ran one at a time, as they do by default.
- Runs: Each setting ran three times, each time in a fresh process. Each run fit the context once, made five warmup predictions, and then timed 50 predictions. Each host used one GPU and sent one request at a time.
- Reported values: Fit time is the median of the three runs. Prediction time is the median of the three runs’ median times. Peak memory is the highest value across the three runs, and held memory is the median.
- What the times include: Fit time covers the fit call and its preprocessing, but not loading the model. Prediction time covers the model call on request rows already on the GPU. Neither includes reading files, converting your data, or network time in a service.
- What the memory includes: Memory is the GPU memory that PyTorch allocates. Peak memory during fit includes loading the model and the input data. The CUDA driver uses some GPU memory on top of these values.
These are single-request times, not throughput under concurrent load. The benchmarks do not cover categorical or text columns, so datasets with them can need different time and memory.
Benchmark Results
Choose your GPU and the number of feature columns to see the numbers for that setup. Each table has one column per estimator count: 1, 4, and 8.
Fitting the Context
Show
GPU
Feature columns
| Context rows | 1 estimator (s) | 4 estimators (s) | 8 estimators (s) |
|---|---|---|---|
| 5,000 | 1.3 | 3.3 | 5.8 |
| 10,000 | 1.9 | 5.2 | 9.5 |
| 20,000 | 3.2 | 11 | 20 |
| 50,000 | 8.9 | 34 | 67 |
Time to fit the context once, in seconds. Kumo Tabular Large on A10G (24 GB, BF16), 100 feature columns.
Predicting
Show
GPU
Feature columns
Context rows
| Rows per request | 1 estimator (ms) | 4 estimators (ms) | 8 estimators (ms) |
|---|---|---|---|
| 1 | 39 | 155 | 299 |
| 10 | 40 | 150 | 301 |
| 100 | 40 | 154 | 286 |
| 500 | 54 | 231 | 437 |
| 1,000 | 99 | 410 | 795 |
Median time of one prediction, in milliseconds. Kumo Tabular Large on A10G (24 GB, BF16), 100 feature columns, 10,000 context rows.
What Drives Time and Memory
The benchmarks vary four settings: context rows, feature columns, request size, and the number of estimators. This section shows how each one changes time and memory, so you can judge which of your settings matter most.
Each example uses the A10G and changes one setting from this starting setup: 10,000 context rows, 100 feature columns, 8 estimators, and requests of 100 rows. In this setup, a fit takes 9.5 s and a prediction takes 286 ms. The L40S and L4 follow the same pattern at different speeds.
- Context rows mostly add fit time, a cost you pay once per context. Going from 10,000 to 50,000 rows raises fit time from 9.5 s to 67 s, but prediction time only from 286 ms to 494 ms.
- Feature columns add fit time and memory. Going from 100 to 500 columns raises fit time from 9.5 s to 36.7 s and peak fit memory from 2.4 GiB to 5.0 GiB. Predictions take twice as long, 593 ms.
- Request size matters little up to 100 rows, so send rows together. A 100-row request usually takes no more than 10% longer than a single row. At 1,000 rows, the GPUs differ more: 383 ms on the L40S, 795 ms on the A10G, and 1.3 s on the L4.
- Estimators raise prediction time nearly in step with their number, and fit time by less. One estimator takes 1.9 s to fit and 40 ms to predict. Eight take 9.5 s and 286 ms.
- On the T4, prediction time also grows with the context and the request size. Doubling the context raises it from 1.2 s to 2.3 s, and a 1,000-row request takes 8.1 s.
Your data, GPU, and settings can change these numbers. Before you choose production hardware, it’s worth timing a fit and a prediction on a sample of your own data. Check Your GPU and Measure a Workload shows how.