> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/sdgm/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/sdgm/_mcp/server.

# Column Preprocessing

## Data Type and Semantic Type

Kumo automatically detects column types for preprocessing, but you can manually adjust them.
If a mismatch is detected, Kumo provides recommendations or alerts you with an invalid data type error.

Ensure that the **semantic type** (`Type`) aligns with the **data type** (`Data Type`) to avoid inconsistencies.

### Supported Column Types

Kumo supports **preprocessing** for the following data types:

* **Numerical** – Integers and floats where numerical ordering is meaningful (for example, product price or discount percentage).
* **Categorical** – Single-token strings or booleans with a limited set of unique values (up to 4,000 by default), such as product type or subscription status.
* **Multi-Categorical** – Comma-separated lists of categorical values (for example, restaurant tags: `"vegetarian, italian, pickup_only"`).
* **ID** – Unique identifiers with no numerical meaning, such as customer IDs or product group numbers.
* **Text** – Multi-token strings where semantic meaning is important (for example, product descriptions and reviews).
* **Timestamp** – Date/time values in a valid format (preferably **ISO 8601** or epoch time). For Parquet data, ensure timestamps are correctly cast to a `DATE/TIME/TIMESTAMP` type.
* **Embedding** – Lists of equal-length floats, typically representations from AI models.

![Column configuration for the DimCustomer table, with the Semantic Type dropdown open showing the supported column types](https://fdr-prod-docs-files-public.s3.us-east-1.amazonaws.com/nvidia-sdgm.docs.buildwithfern.com/85203c26eaf6541bc446c0c1268dd55a3a797be152eef9e56743216c0ec01519/img/kumo3/column-types-adventure-works.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIA6KXJSKKNFOCF7G4B%2F20260912%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260912T200844Z&X-Amz-Expires=604800&X-Amz-Signature=c438dc724ff21f4c63f9d35a5f494b3148c97200b1eb9d80f4c347316301d75c&X-Amz-SignedHeaders=host&x-amz-checksum-mode=ENABLED&x-id=GetObject)

### Unsupported Column Types

The following column types **are not supported** for preprocessing in Kumo.
Transform them before ingestion:

* **Full URLs** – Extract meaningful components (such as domain and path) and treat them as categorical values.
* **Lat/Long Coordinates** – Convert to categorical geographic areas.
* **IP Addresses** – Remove PII, extract high-level details (such as subnet), and treat as categorical elements.
* **Phone Numbers** – Remove PII, extract relevant components (such as area code), and treat as categorical values.

### Handling Nested or Complex Data

Kumo does not support **nested schemas, arrays, or maps**. To use such data, transform it into a string format:

**Example: Converting an array to a string**

**Before:** `["TV", "electronics", "promotion"]`

**After:** `"TV, electronics, promotion"`

## Column Properties

### **Primary Key Column**

Each row should have a **unique primary key** (for example, `user_id`).
If duplicate rows share the same key, Kumo retains only one and drops the rest.

### **Create Date Column**

The **Create date column** represents when a row was created or when the data became valid.
This helps define training timelines and ensures predictions use the correct time-based data.

### **End Date Column**

The **end date column** restricts training and predictions to a specific timeframe.

* For **temporal tasks**, training includes only data valid within this timeframe.
* For **batch predictions**, Kumo uses only rows where the **Create date** is on or before the prediction time and the **End date** is before the prediction time.

**Example: End Date for product availability**

If a product goes out of stock on a particular date, set **End date** to the column tracking this date.

**End\_Date-aware sampling is currently supported only in the following contexts:**

* The **entity table** (the table referenced after `FOR EACH` in a predictive query).
* The **RHS table** in **link prediction** tasks (the table referenced after `LIST_DISTINCT`).