Data Types

View as Markdown

KumoRFM uses two complementary type systems: Data Types (Dtype) for physical storage and Semantic Types (Stype) for semantic meaning.

Data Types (Dtype)

Data types represent how data is physically stored and processed:

1from kumoai import Dtype
2
3# Numerical types
4Dtype.bool # Boolean values
5Dtype.int # Integer values
6Dtype.float # Floating point values
7
8# String types
9Dtype.string # Text data
10Dtype.binary # Binary data
11
12# Temporal types
13Dtype.date # Date/timestamp
14Dtype.time # Time values
15Dtype.timedelta # Time differences
16
17# List types
18Dtype.floatlist # Lists of floats (embeddings/sequences)
19Dtype.intlist # Lists of integers
20Dtype.stringlist # Lists of strings

Dtype Mapping

When constructing a LocalTable, each pandas dtype is automatically mapped to a corresponding Kumo dtype. While you can access the dtype of individual columns in a LocalTable, you cannot modify it. For data type modifications, modify the underlying pandas.DataFrame instead before creating the table:

1import pandas as pd
2import kumoai.rfm as rfm
3
4df = pd.DataFrame({'user_id': [1, 2, 3]})
5table = rfm.LocalTable(df, name="users")
6
7print(table["user_id"].dtype)

Semantic Types (Stype)

Semantic types define the meaning of data and determine column-level data processing within the model:

1from kumoai import Stype
2
3# Core semantic types
4Stype.numerical # Numerical values for mathematical operations
5Stype.categorical # Discrete categories with limited cardinality
6Stype.multicategorical # Multiple categories in single field
7Stype.ID # Unique identifiers
8Stype.text # Natural language text
9Stype.timestamp # Date/time information
10Stype.sequence # Embeddings or sequential data

Dtype-Stype Compatibility

Not all combinations are valid. Check compatibility using:

1# Check if a semantic type supports a data type
2stype = Stype.categorical
3dtype = Dtype.string
4
5is_compatible = stype.supports_dtype(dtype)
6print(f"{stype} supports {dtype}: {is_compatible}")
7
8# Get default semantic type for a data type
9default_stype = dtype.default_stype
10print(f"Default stype for {dtype}: {default_stype}")

Stype Assignment and Encoding

Semantic types can be modified after table creation, provided they are compatible with the underlying data type. The semantic type determines how values are encoded and processed by the foundation model:

1# Valid stype modifications (compatible with underlying dtype)
2table['user_id'].stype = 'ID' # Dtype.int -> Stype.ID
3table['category'].stype = 'categorical' # Dtype.string -> Stype.categorical
4table['description'].stype = 'text' # Dtype.string -> Stype.text
5
6# Invalid: integers do not support text semantic types:
7table['user_id'].stype = 'text'

For detailed information about how different semantic types affect column preprocessing and encoding, please refer to the Column Preprocessing Guide.