Skip to navigation

Data Requirements

View as Markdown

Prepare entity and event tables before constructing a graph with the client. Entity tables contain customers or products; event tables contain orders or other interactions. Keys define joins, and timestamps determine which records the client can sample at each prediction anchor time.

Required table characteristics

  • Stable entity keys: Each entity table must have a non-null, unique primary key.
  • Consistent foreign keys: Relationship columns must use compatible values and types across tables.
  • Usable timestamps: Event tables must identify when each event occurred. Convert source values to a supported timestamp type before constructing the graph.
  • Task-relevant history: Include events that occur before the prediction anchor time, which is the point in time when Kumo makes each prediction. Include enough labeled history to represent the task.
  • Clear column meaning: Assign semantic types, which tell Kumo how to interpret each column, to distinguish identifiers, numerical measures, categories, text, timestamps, and sequences.

Graph validation can detect structural problems, but it cannot determine whether a relationship, timestamp, or historical window is appropriate for your prediction task. Review the metadata that Kumo infers from your tables before generating predictions.

Choose a data location

Choose a location based on where your prepared data already lives and how you want Kumo to access it.

Data locationBest for
pandasPrepared data that fits in application memory
SQLiteFile-based prototypes
DuckDBLocal analytical data
SnowflakeWarehouse-backed sampling
DatabricksUnity Catalog and SQL warehouse data

For databases and warehouses, refer to the corresponding page under Connect Data Sources for information about permissions, connection parameters, and source-specific preparation. Connector-backed graphs query and sample data from the source instead of loading every source table into a DataFrame.

Prepare example tables

The following example creates two entity tables, customers and products, and one event table, orders. The key columns use matching physical types so that Kumo can connect each order to its customer and product.

import pandas as pd
customers = pd.DataFrame({
"customer_id": pd.Series([1, 2, 3], dtype="int64"),
"segment": pd.Series(["small", "enterprise", "small"], dtype="string"),
"notes": pd.Series(["prefers email", "", "new account"], dtype="string"),
"signup_time": pd.to_datetime(["2024-01-01", "2024-02-15", "2024-03-20"]),
})
orders = pd.DataFrame({
"order_id": pd.Series([101, 102, 103], dtype="int64"),
"customer_id": pd.Series([1, 1, 2], dtype="int64"),
"product_id": pd.Series([501, 502, 501], dtype="int64"),
"amount": pd.Series([20.0, 35.0, 50.0], dtype="float64"),
"order_time": pd.to_datetime(["2024-04-01", "2024-04-08", "2024-04-10"]),
})
products = pd.DataFrame({
"product_id": pd.Series([501, 502], dtype="int64"),
"product_name": pd.Series(["Basic", "Premium"], dtype="string"),
})

Before continuing, confirm that primary-key candidates are non-null and unique, and that related key columns use compatible physical types.

assert customers["customer_id"].notna().all()
assert customers["customer_id"].is_unique
assert orders["order_id"].notna().all()
assert orders["order_id"].is_unique
assert products["product_id"].notna().all()
assert products["product_id"].is_unique
assert orders["customer_id"].dtype == customers["customer_id"].dtype
assert orders["product_id"].dtype == products["product_id"].dtype

Kumo also accepts unsigned integer columns. During request preparation, nullable integer keys and identifiers larger than JavaScript’s safe-integer range retain their integer identity. Timezone-aware timestamps are converted to UTC rather than having their offsets discarded.

Next, review Data Types and Semantic Types, choose the appropriate Time and End-Time Columns, and follow the Data Preparation Best Practices. After the source data is ready, create a graph.