> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/sdgm/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/sdgm/_mcp/server.

# How should I evaluate Kumo foundation models?

> Evaluate prediction quality and inference requirements on representative held-out data

Predict on held-out examples whose labels are excluded from the context. For temporal tasks, use only records and labels available at each prediction time.

Compare with your existing model or another baseline on the same examples. Choose metrics that reflect the decisions the predictions support. Check probability calibration if those decisions depend on probability values, and measure latency and memory use at the expected context size and scoring volume.

## Public Benchmark Datasets

Choose tasks supported by your model and runtime, and use the benchmark's prescribed splits and evaluator.

For **Kumo Relational**, [RelBench v1](https://huggingface.co/datasets/stanford-star/relbench-v1) and [RelBench v2 additional datasets](https://huggingface.co/datasets/stanford-star/relbench-v2-extra) provide multi-table databases and prediction tasks. Follow the [RelBench evaluation protocol](https://star-project.stanford.edu/relbench/) for splits and scoring.

For **Kumo Tabular**, use:

* [TabArena](https://tabarena.ai/): benchmark tasks, results, and links to supporting resources.
* [TALENT datasets](https://drive.google.com/drive/folders/1j1zt3zQIo8dO6vkO-K-WE6pSrl71bf0z): classification and regression datasets, with setup instructions in the [repository](https://github.com/LAMDA-Tabular/TALENT).
* [BeyondArena](https://huggingface.co/datasets/TabArena/BeyondArena): datasets with random, temporal, and grouped splits.