Connect Data Sources

Kumo Tabular reads one table, already in memory. Use sdm_connectors.read() to load that table into a pandas DataFrame, then build a Request or Schema from it. This is a flat read: read() returns a DataFrame, it does not build a graph or infer relationships, and it does not sample a warehouse in place — that capability (Schema.from_source) is for the relational model.

Installation

Install the connector extra for your source:

pip install "structured-data-models-client[sqlite]"     # also duckdb, postgres, lakebase, snowflake, databricks, s3
pip install "structured-data-models-client[all]"        # every backend, plus the relational model

Read a table

from sdm_connectors import read
# A local file
orders = read("local", path="orders.parquet")
# Amazon S3
orders = read("s3", path="s3://my-bucket/orders.parquet")
# A SQL backend: exactly one of table= or query=
orders = read("sqlite", database="data.db", table="orders")
orders = read("snowflake", account="...", user="...", password="...", warehouse="...", query="SELECT * FROM orders WHERE region = 'US'")

read(source: str, **kwargs) -> pandas.DataFrame

source is one of 'local', 's3', 'sqlite', 'duckdb', 'postgres', 'snowflake', or 'databricks'. A SQL source needs exactly one of table= or query=, plus the connection arguments for that backend. A connection opened by read() is closed again before it returns.

Local files and Amazon S3

from sdm_connectors import read
local = read("local", data=my_dataframe)          # from an in-memory DataFrame or dict
local = read("local", path="customers.csv")        # CSV or Parquet, optionally compressed
s3 = read(
    "s3",
    path="s3://my-bucket/customers.parquet",
    storage_options={"key": "<access-key-id>", "secret": "<secret-access-key>"},
)

Do not store credentials in source code. Prefer the ambient AWS credential chain, an instance or workload role, or values from your platform’s secret manager; omit storage_options when the runtime is already configured to authenticate to S3.

SQLite and DuckDB

orders = read("sqlite", database="data.db", table="orders")
orders = read("duckdb", database="warehouse.duckdb", query="SELECT * FROM orders")

database and uri are aliases for the same argument; pass exactly one. Omitting it for DuckDB opens an in-memory database.

PostgreSQL and Databricks Lakebase

orders = read("postgres", host="db.example.com", dbname="analytics", user="reader", password="...", table="orders")

Standard PG* environment variables (PGHOST, PGPORT, PGDATABASE, PGUSER, PGPASSWORD, PGSSLMODE) fill in any of host, port, dbname, user, password, sslmode you omit. read() has no Lakebase-specific form — for Lakebase’s short-lived token authentication, build a connection with sdm_connectors.databricks.lakebase() and read from it with read_table() instead (see Kumo Relational’s Lakebase connector for the pattern; read_table() works the same for a single table).

Snowflake

orders = read("snowflake", account="my_account", user="my_user", password="...", warehouse="my_warehouse", table="orders")

In a Snowflake notebook with an active Snowpark session, omit every authentication argument and the connector reuses that session.

Databricks

import os
orders = read(
    "databricks",
    server_hostname="<workspace>.cloud.databricks.com",
    http_path="/sql/1.0/warehouses/<warehouse_id>",
    access_token=os.environ["DATABRICKS_TOKEN"],
    table="orders",
)

When connection arguments are omitted, the connector reads DATABRICKS_SERVER_HOSTNAME, DATABRICKS_HTTP_PATH, DATABRICKS_TOKEN, DATABRICKS_CATALOG, and DATABRICKS_SCHEMA.

Build a request from the result

from sdm_connectors import read
from sdm_client import Request
labelled = read("snowflake", account="...", user="...", password="...", warehouse="...", query="SELECT * FROM orders WHERE churn IS NOT NULL")
to_score = read("snowflake", account="...", user="...", password="...", warehouse="...", query="SELECT * FROM orders WHERE churn IS NULL")
request = Request(context=labelled, predict=to_score.drop(columns=["churn"]), target="churn", task="classification")

Review the inferred schema before predicting — see Prepare Your Data.

Errors

read() raises ConnectorError with a stable code: UNKNOWN_CONNECTOR, INVALID_CONNECTOR_ARGS, CONNECT_FAILED, QUERY_FAILED, READ_FAILED, or NOT_FOUND. A missing optional driver raises MissingBackendError instead, naming the extra to install.