Best Practices

This section outlines the best practices for preparing high-quality datasets for KumoRFM, incorporating key insights from Table and Graph design patterns.

Table Structure and Entity Design

1. One entity or event per table:

   1
   # Good: Separate tables for different entities
   2
   users = df[['user_id', 'user_name', 'user_email']]
   3
   transactions = df[['transaction_id', 'user_id', 'amount', 'timestamp']]
   4
   5
   # Avoid: Mixing entities in one table
   6
   mixed_data = df[['user_id', 'user_name', 'transaction_id', 'amount']]

2. Single time column per table:

   1
   # Good: Split tables with multiple timestamps
   2
   policies = df[['policy_id', 'user_id', 'start_date', 'policy_type']]
   3
   claims = df[['claim_id', 'policy_id', 'claim_date', 'claim_amount']]
   4
   5
   # Avoid: Multiple time columns in one table
   6
   mixed_times = df[['policy_id', 'start_date', 'claim_date', 'end_date']]

3. Handle many-to-many relationships with junction tables:

   1
   # Good: Junction table pattern
   2
   users = df[['user_id', 'user_name']]
   3
   skills = df[['skill_id', 'skill_name']]
   4
   user_skills = df[['user_skill_id', 'user_id', 'skill_id', 'proficiency']]

Data Preparation

1. Modify dtypes at `pandas.DataFrame` level before creating tables:

   1
   # Good: Set proper pandas dtypes first
   2
   df['user_id'] = df['user_id'].astype('int64')
   3
   df['category'] = df['category'].astype('string')
   4
   df['timestamp'] = pd.to_datetime(df['timestamp'])
   5
   table = rfm.LocalTable(df, "my_table")

2. Use consistent naming conventions:

   1
   # Good: Consistent foreign key naming
   2
   users.user_id, transactions.user_id, profiles.user_id
   3
   4
   # Avoid: Inconsistent naming
   5
   users.id, transactions.uid, profiles.customer_id

3. Ensure unique primary keys:

   1
   # Validate uniqueness
   2
   assert df['user_id'].nunique() == len(df)
   3
   assert df['user_id'].notna().all()
   4
   # Otherwise, Kumo will automatically drop duplicates internally!

Semantic Type Assignment

1. Choose meaningful semantic types:

   1
   # IDs should use ID stype
   2
   table['user_id'].stype = 'ID'
   3
   4
   # Text descriptions should use text stype
   5
   table['description'].stype = 'text'
   6
   7
   # Limited categories should use categorical stype
   8
   table['status'].stype = 'categorical'
   9
   10
   # Numerical measurements should use numerical stype
   11
   table['amount'].stype = 'numerical'
   12
   13
   # Important: Validate metadata before proceeding:
   14
   print(table.metadata)

Graph Construction

1. Design for meaningful relationships:

   1
   # Good: Meaningful entity relationships
   2
   graph = rfm.Graph.from_data({
   3
       'users': users_df,           # Entity table
   4
       'transactions': transactions_df,  # Event table linked to users
   5
       'products': products_df      # Entity table linked via transactions
   6
   })

2. Ensure prediction-ready structure:

   1
   # Consider what PQL queries you want to run
   2
   # Example: "PREDICT COUNT(transactions.*, 0, 30, days) FOR users.user_id=1"
   3
   # Requires: users -> transactions relationship via user_id
   4
   # Requires: Timestamp at the transaction table

Common Data Modeling Patterns

Entity-Event Pattern

1
# Users (entity) with transactions (events)
2
users = pd.DataFrame({'user_id': [1, 2], 'name': ['Alice', 'Bob']})
3
transactions = pd.DataFrame({
4
    'transaction_id': [1, 2, 3],
5
    'user_id': [1, 1, 2],
6
    'amount': [100, 50, 200],
7
    'timestamp': ['2023-01-01', '2023-01-02', '2023-01-03']
8
})

Hierarchical Entities

1
# Multiple levels: company -> department -> employee
2
companies = pd.DataFrame({'company_id': [1, 2], 'company_name': ['ACME', 'TechCorp']})
3
departments = pd.DataFrame({'dept_id': [1, 2], 'company_id': [1, 1], 'dept_name': ['Engineering', 'Sales']})
4
employees = pd.DataFrame({'emp_id': [1, 2, 3], 'dept_id': [1, 1, 2], 'emp_name': ['Alice', 'Bob', 'Charlie']})

Junction Table for Many-to-Many

1
# Products and categories with many-to-many relationship
2
products = pd.DataFrame({'product_id': [1, 2], 'product_name': ['Laptop', 'Mouse']})
3
categories = pd.DataFrame({'category_id': [1, 2], 'category_name': ['Electronics', 'Accessories']})
4
product_categories = pd.DataFrame({
5
    'product_category_id': [1, 2, 3],
6
    'product_id': [1, 1, 2],
7
    'category_id': [1, 2, 2]
8
})

Summary

Following these best practices will help ensure your KumoRFM datasets are well-structured, validated, and optimized for performance:

Table Design:

• One entity or event per table
• Single time column per table
• Unique primary keys with consistent naming
• Junction tables for many-to-many relationships

Data Preparation:

• Set proper pandas dtypes before creating tables
• Use meaningful semantic types (ID, categorical, text, numerical)
• Validate metadata and semantic types before proceeding

Graph Structure:

• Design meaningful entity relationships
• Consider PQL query requirements in your structure
• Ensure single connected component
• Test with validation workflow

These patterns will help you create robust, queryable datasets that work effectively with KumoRFM’s predictive capabilities.