Table Definitions

LocalTable objects wrap a pandas.DataFrame with metadata about columns, primary keys, and time columns. The semantic types are required metadata, while the primary key and time column are optional. Each table can have at most one primary key and at most one time column, but it can contain many foreign keys (primary keys of other tables).

Dtype and Metadata Inference

When creating a LocalTable, column dtypes and stypes are automatically inferred from the underlying data based on the pandas data type and heuristics. The key metadata that needs to be properly set includes:

• Stypes: Semantic types that determine model processing behavior
• Primary key: Unique identifier for the table (optional but recommended)
• Time column: Temporal column for time-based operations (optional)

The LocalTable.infer_metadata() method automates much of this process:

• Primary key detection: Uses heuristics to suggest potential primary keys based on column names, uniqueness, and data patterns
• Time column detection: Identifies columns with temporal data types or time-related naming patterns

1
import kumoai.rfm as rfm
2
3
# Automatic inference (recommended for initial setup)
4
table = rfm.LocalTable(df, "users").infer_metadata()
5
6
# Manual override of inferred metadata when needed
7
table['user_id'].stype = Stype.ID  # Override inferred stype
8
table.primary_key = "user_id"      # Override inferred primary key

Basic Table Creation

1
import pandas as pd
2
import kumoai.rfm as rfm
3
4
# Create table with automatic metadata inference
5
users_table = rfm.LocalTable(
6
    df=df_users,
7
    name="users"
8
).infer_metadata()
9
10
# Create table with explicit metadata
11
transactions_table = rfm.LocalTable(
12
    df=df_transactions,
13
    name="transactions",
14
    primary_key="transaction_id",
15
    time_column="timestamp"
16
)

Inspecting Table Metadata

1
# Access table metadata
2
print(f"Primary key: {users_table.primary_key}")
3
print(f"Time column: {users_table.time_column}")
4
print(f"Columns: {[col.name for col in users_table.columns]}")
5
6
# View metadata summary
7
metadata_df = users_table.metadata
8
print(metadata_df)
9
10
# Check column information
11
print(f"Column: {users_table['age'].name}")
12
print(f"Dtype: {users_table['age'].dtype}")
13
print(f"Stype: {users_table['age'].stype}")

What Makes a Good Table

A good LocalTable should have:

• Clean dtypes: Set proper pandas dtypes at DataFrame level before table creation
• Meaningful stypes: ID columns use Stype.ID, categorical data uses Stype.categorical, text uses Stype.text, etc
• Unique primary key: Non-null, no duplicates, uniquely identifies each row, preferably stored as integer
• Consistent naming: Foreign keys match their referenced primary key names
• Single time column: One temporal column when temporal data is available