> For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.nvidia.com/sdgm/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/sdgm/_mcp/server. # KumoRFM: A Relational Foundation Model > Why KumoRFM was needed, how its relational foundation-model architecture works, and what the RelBench results show KumoRFM is a foundation model for structured relational data. It is designed to generalize across databases and prediction tasks without retraining, preserving the multi-table, temporal, and graph-structured signals that conventional feature engineering loses. **Originally published:** May 2025\ **Authors:** Matthias Fey, Vid Kocijan, Federico Lopez, Jan Eric Lenssen, and Jure Leskovec\ **Paper:** [KumoRFM: A Relational Foundation Model](https://arxiv.org/abs/2604.12596) ## The problem Foundation models have transformed unstructured data. Large language models generalize across text tasks, diffusion models generate images from descriptions, and code models complete functions. Each uses pre-training to generalize without a task-specific model. Structured relational data stored across interconnected database tables has lacked an equivalent. For tasks such as churn, fraud, recommendations, and demand forecasting, teams typically engineer features, train a task-specific model, and maintain a separate pipeline. A typical enterprise database contains 10 to 50 interconnected tables, such as `users` to `orders` to `products` to `categories`. The data is relational and temporal, but conventional ML requires it to be flattened into one feature table. For a single prediction task, the paper measured 12.3 hours and 878 plus or minus 77 lines of code for a data scientist with a Stanford CS Master's degree and five years of experience. The process starts again for each new question. ### What gets lost Feature engineering is not just time-consuming. It asks people to select and encode relational patterns from a combinatorially large space. * **Multi-hop relationships:** Customer behavior can depend on products, categories, and similar customers several relationships away. * **Temporal sequences:** A count of three orders in 30 days loses whether the orders were accelerating, decelerating, or clustered. * **Graph topology:** Fraud rings, supply-chain cascades, and social communities are patterns in the shape of connections. * **Entity-level context:** Aggregates obscure details such as returns, support contacts, or out-of-stock products that live in connected tables. **Example:** In the H\&M retail dataset, a churn prediction can depend on the interaction of low order count, no fashion-news subscription, and inactive club membership. It can also depend on the temporal pattern of purchases rather than a single recency aggregate. The paper also evaluates Formula 1 race outcomes across nine tables and 67 columns. On a top-three prediction task, KumoRFM scores 91.07 AUROC zero-shot, compared with 82.40 for a human data scientist baseline. The limitation of feature engineering is coverage, not only speed. Many relevant multi-table, multi-hop, and time-windowed combinations exist in the relational structure but are never manually encoded. ## Why not XGBoost? Gradient-boosted trees such as XGBoost, LightGBM, and CatBoost are strong tools for flat, pre-engineered tables. Their structural limitation is that they require one row per entity and one column per feature. To apply them to a relational database, teams join tables, compute aggregates over time windows, and collapse the results into a feature table. That destroys multi-hop relationships, event ordering, graph topology, and detailed context from connected entities. **Analogy:** A relational database is like a social network. A flat feature table describes each person with numbers such as age, friend count, and post count, but loses who they know, the nature of those connections, and their community structure. Every new task also needs a separate feature-engineering process. Churn, fraud, recommendation, and lead scoring demand different aggregations and join paths, with no transfer between tasks. ## Why not GNNs? Graph neural networks (GNNs) represent rows as nodes and foreign-key relationships as edges. They preserve relational structure, temporal dependencies, and graph topology through message passing. Methods such as GraphSAGE, GCN, and graph transformers have shown the value of learning directly on the relational graph. The [Relational Deep Learning framework](https://arxiv.org/abs/2312.04615) made this approach reproducible. GNNs nevertheless remain task- and schema-specific: 1. A GNN trained for churn cannot be directly reused for fraud or another database schema. 2. Even an RDL pipeline needs about 30 minutes of training and 56 lines of setup code per task. 3. Standard GNNs do not adapt to a new task at inference time through in-context examples. 4. They offer limited insight into why a particular prediction was made. GNNs solve the representation problem: relational data should be modeled as a graph. KumoRFM addresses the remaining generalization problem: applying learned relational patterns to new databases and tasks without retraining. ## Why not build your own? Serializing tables as JSON, CSV, or natural language for an LLM does not resolve the structural problem. The paper's Llama 3.2 3B baseline averages 68.06 AUROC on classification tasks, ahead of LightGBM at 62.44 but below KumoRFM zero-shot at 76.71. LLMs are trained for next-token prediction, struggle with numerical patterns and large relational subgraphs, and have a mismatched objective for forecasting or ranking. Tabular foundation models learn relationships within single flat tables, but still require multiple tables to be joined and flattened. They also face feature-count and context-length constraints. Building a relational foundation model from scratch requires a schema-agnostic encoder, an architecture that handles varied graph topologies, training across diverse databases, and an in-context learning mechanism for structured data. KumoRFM was pre-trained on publicly available real-world databases and synthetic relational data across e-commerce, social, medical, financial, and other domains. ## The foundation-model insight Pre-training lets language models learn patterns that recur across domains, then use in-context examples to understand a specific task. The paper proposes that relational databases have analogous universal patterns: * **Recency, frequency, and monetary patterns:** Recent and frequent interactions predict future behavior across domains. * **Temporal dynamics:** Seasonality, trends, and decay appear in timestamps and event sequences. * **Graph topology:** Hubs, clusters, and bridges carry signal in fraud, social, and supply-chain data. * **Cross-table propagation:** An entity is influenced by connected entities and their connections. A model pre-trained on a diverse set of relational databases can learn these patterns and use labeled historical examples from a new database to make predictions without retraining. ## What is KumoRFM? ![Comparison of XGBoost and LightGBM, GNN-based relational deep learning, and KumoRFM across their data representation, transfer, and task-adaptation capabilities.](https://fdr-prod-docs-files-public.s3.us-east-1.amazonaws.com/nvidia-sdgm.docs.buildwithfern.com/410fcb96f8482fec7bbe8dc60ca51090f1badb5236b62b5f7c74aaeda1f71e51/img/research/kumorfm-approach-comparison.svg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIA6KXJSKKNFOCF7G4B%2F20260920%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260920T185215Z&X-Amz-Expires=604800&X-Amz-Signature=db527e2470376c0b403b722247ff633ad688d19391537d2d257708798f846d26&X-Amz-SignedHeaders=host&x-amz-checksum-mode=ENABLED&x-id=GetObject) KumoRFM, the Kumo Relational Foundation Model, is pre-trained on publicly available real-world databases and synthetic relational data. No private enterprise data was used during pre-training. It combines a schema-agnostic row encoder, a relational graph transformer, and an in-context learning module. 1. **Schema-agnostic:** Works across arbitrary tables, columns, and relationship types. 2. **Multi-modal:** Handles numerical, categorical, timestamp, text, embedding, and hashed or anonymized identifier columns. 3. **In-context:** Uses labeled examples from the target data at prediction time rather than retraining. 4. **Multi-task:** Supports classification, multi-label prediction, regression, and link prediction from the same model weights. 5. **Explainable:** Provides global and entity-level explanations. 6. **Fine-tunable:** Supports zero-shot prediction and task-specific fine-tuning. ## Predictive Query Language ![Four PQL examples mapping a target aggregation and semantic type to multi-class classification, binary classification, regression, and link prediction.](https://fdr-prod-docs-files-public.s3.us-east-1.amazonaws.com/nvidia-sdgm.docs.buildwithfern.com/b7983a7fe5efb9fdae12cc7ee6d24c9f7d3eff7ad9302f9030c5ddd750465488/img/research/pql-task-mapping.svg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIA6KXJSKKNFOCF7G4B%2F20260920%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260920T185215Z&X-Amz-Expires=604800&X-Amz-Signature=f8517fcd0a336333ca5ccdaf28db692107908b6e303a13c0e45d340583d94a5e&X-Amz-SignedHeaders=host&x-amz-checksum-mode=ENABLED&x-id=GetObject) Predictive Query Language (PQL) declaratively specifies what to predict from relational data. Where SQL retrieves existing data, PQL defines a future prediction target. KumoRFM handles subgraph sampling, context generation, feature extraction, and inference. PQL has three main clauses: * **`PREDICT`** specifies a target or aggregation over a future time window. * **`FOR EACH`** specifies the entities to score. * **`WHERE`** optionally filters those entities. Supported aggregations include `SUM`, `COUNT`, `AVG`, `MAX`, `MIN`, `FIRST`, and `LIST_DISTINCT`. Time windows can be expressed in hours, days, or months. The language supports label independence, temporal safety, and automatic task typing. It lets KumoRFM generate additional historical labels for context, prevent time leakage, and select the appropriate model head from the query specification. PQL separates what to predict from how to predict it. The user specifies the business question while the model handles the ML pipeline. ## Architecture ![KumoRFM architecture pipeline: row encoder, relational graph transformer, in-context learning, and prediction.](https://fdr-prod-docs-files-public.s3.us-east-1.amazonaws.com/nvidia-sdgm.docs.buildwithfern.com/8ac0e0a1523e27730165a9bef52d4c9bd2b8027d1f18c7143b8ee1b4d4cf983a/img/research/kumorfm-architecture.svg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIA6KXJSKKNFOCF7G4B%2F20260920%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260920T185215Z&X-Amz-Expires=604800&X-Amz-Signature=920cfb8c12c9352ed46e8563aa55fbc4c23f9a6af921c387c6d2b00ad4a30981&X-Amz-SignedHeaders=host&x-amz-checksum-mode=ENABLED&x-id=GetObject) KumoRFM transforms a relational database and PQL query into a prediction in four stages. ### Stage 1: Table-invariant row encoder Each row is encoded into a fixed-dimensional vector based on the semantic type of every column. Numerical values are normalized and projected, categorical and multi-categorical values are embedded, timestamps receive temporal encodings, text is encoded with a language model, and embeddings are projected directly. The encoder uses a transformer over each table's two-dimensional cell grid, making it agnostic to both table width and table size. ### Stage 2: Relational graph transformer The database becomes a temporal heterogeneous graph: rows are nodes and primary-foreign key relationships are edges. A relational graph transformer performs attention over the graph, using node-type, hop, time, and subgraph positional encodings. Unlike fixed GNN message-passing rules, attention adapts to node type, hop distance, temporal context, and local topology. These encodings let the same transformer weights operate across schemas. ### Stage 3: In-context learning At prediction time, KumoRFM samples historical entities, retrieves their ground-truth labels for the PQL task, and processes these context-label pairs alongside test entities. This is analogous to few-shot prompting: examples with known outcomes calibrate predictions for new entities. The context is generated online by constructing a training table from historical labels and attaching it to the entity table through relational keys. The paper reports generating approximately two million in-context labels in under one second on average. ### Link prediction For recommendation tasks, KumoRFM produces fully inductive pair-wise representations from a user-centric subgraph. It samples item representations to a fixed context size so the model can learn signals from repeated purchases, collaborative patterns, and diverse hop distances. ## Benchmark results The paper evaluates KumoRFM on [RelBench](https://relbench.stanford.edu), a benchmark covering seven relational databases, 30 tasks, more than 103 million rows, and 51 tables. KumoRFM was not trained on any RelBench dataset, so its reported results are zero-shot. | Dataset | Domain | Tasks | Tables | Rows | Columns | | ---------- | ---------- | ----: | -----: | ----------: | ------: | | rel-amazon | E-commerce | 7 | 3 | 15,000,713 | 15 | | rel-avito | E-commerce | 4 | 8 | 20,679,117 | 42 | | rel-event | Social | 3 | 5 | 41,328,337 | 128 | | rel-f1 | Sports | 3 | 9 | 74,063 | 67 | | rel-hm | E-commerce | 3 | 3 | 16,664,809 | 37 | | rel-stack | Social | 5 | 7 | 4,247,264 | 52 | | rel-trial | Medical | 5 | 15 | 5,434,924 | 140 | | Total | - | 30 | 51 | 103,466,370 | 489 | ### Entity classification Across 12 entity-classification tasks, KumoRFM zero-shot averages 76.71 AUROC. The paper reports 62.44 for LightGBM, 68.06 for the LLM baseline, and 75.83 for supervised RDL. Fine-tuning increases the KumoRFM average to 81.14. | Dataset | Task | LightGBM | Data scientist | RDL | LLM | KumoRFM zero-shot | KumoRFM tuned | | ---------- | ----------- | -------: | -------------: | ----: | ----: | ----------------: | ------------: | | rel-amazon | user-churn | 52.22 | 67.60 | 70.42 | 62.55 | 67.29 | 70.47 | | rel-amazon | item-churn | 62.54 | 81.80 | 82.81 | 73.41 | 79.93 | 82.83 | | rel-f1 | driver-dnf | 68.86 | 69.80 | 72.62 | 80.03 | 82.41 | 82.63 | | rel-f1 | driver-top3 | 73.93 | 82.40 | 75.54 | 87.11 | 91.07 | 99.62 | | rel-hm | user-churn | 55.21 | 69.00 | 69.88 | 63.81 | 67.71 | 71.23 | | Average | - | 62.44 | - | 75.83 | 68.06 | 76.71 | 81.14 | ### Recommendations and regression On nine recommendation tasks, fine-tuned KumoRFM achieves state-of-the-art results in the paper's evaluation. For regression, the results are more nuanced: the data-scientist baseline leads three of nine tasks, while fine-tuned KumoRFM leads five. The paper reports that zero-shot KumoRFM is strongest on classification and recommendation, and that fine-tuning closes the gap on regression. ### Time to first prediction * **KumoRFM zero-shot:** About one second and one line of PQL. * **RDL GNN pipeline:** About 30 minutes and 56 lines of code. * **Manual data-scientist workflow:** About 12.3 hours and 878 lines of code. ## Explainability KumoRFM offers global and local explanations. At the dataset level, it groups column-level context into cohorts and relates distributions to ground-truth labels. At the entity level, gradient-based saliency assigns importance scores to individual cells in an input subgraph. For temporal predictions, it also evaluates accuracy on recent historical snapshots with known labels. Reported metrics include AUROC, AP, MAE, MAPE, MAP\@k, diversity, and popularity bias. For example, a churn prediction can surface order count, order-date recency, and club-membership status as important factors. A recommendation may identify recent browsing history and previous purchasing patterns as the strongest drivers. ## From zero-shot to production KumoRFM has two operating modes. ### In-context, or zero-shot Users provide a database and PQL query, then receive predictions without task-specific training. This mode supports rapid hypothesis testing, data exploration, and assessing whether a prediction task is viable. ### Fine-tuned For production deployment, fine-tuning specializes the model for one dataset and task. It retains pre-trained relational representations while replacing the in-context head with a task-specific head and training on a generated table. The paper reports 10 to 30 percent additional accuracy over zero-shot and efficient scaling to billions of predictions through a single forward pass. ## Further reading * [KumoRFM paper](https://arxiv.org/abs/2604.12596) * [RelBench](https://relbench.stanford.edu) * [Relational Deep Learning](https://arxiv.org/abs/2312.04615) * [KumoRFM documentation](/rfm/overview)