Connector Reference | Structured Data and Graph Models

The Kumo SDK exposes connector classes for object storage, warehouses, uploaded files, AWS Glue, and administrator-managed group connectors. Use a connector to obtain SourceTable objects, then convert those source tables into Kumo Table objects for your graph.

1 import kumoai as kumo
2 
3 kumo.init(url="https://<customer_id>.kumoai.cloud/api", api_key="<api_key>")
4 
5 connector = kumo.S3Connector(root_dir="s3://my-bucket/kumo-input/")
6 orders_src = connector["orders"]        # same as connector.table("orders")
7 print(connector.table_names())
8 print(orders_src.head())

Connector classes

Connector	Use for	Required setup
`S3Connector`	CSV or Parquet files in Amazon S3	Kumo deployment role can read the bucket/prefix
`GCSConnector`	CSV or Parquet files in Google Cloud Storage	Kumo workload identity can read the bucket/prefix
`ADLSConnector`	CSV or Parquet files in Azure Data Lake Storage	Kumo workload identity can read `abfs://` or `abfss://` paths
`SnowflakeConnector`	Tables and views in a Snowflake schema	Snowflake account, warehouse, database, schema, and credentials unless using native SPCS auth
`DatabricksConnector`	Tables in a Databricks catalog	Databricks host plus warehouse/cluster/catalog and OAuth or PAT credentials
`BigQueryConnector`	Tables in a BigQuery dataset	Project, dataset, and service-account credentials
`GlueConnector`	Partitioned Parquet tables in AWS Glue Catalog	Glue catalog metadata and IAM access to the backing S3 data
`FileUploadConnector`	Local or remote CSV/Parquet files uploaded into Kumo	Use `upload()` before reading the table
`GroupConnector`	Admin-managed connectors in RBAC-enabled workspaces	Select a group, then call `kumo.list_group_connectors()`

If you use Kumo in Snowpark Container Services, SnowflakeConnector is the supported connector type.

Object storage connectors

Object storage connectors share the same access pattern. Create the connector with a root directory and read tables by relative table name. If you omit root_dir, pass the full table path to table().

1 s3 = kumo.S3Connector(root_dir="s3://my-bucket/kumo/")
2 gcs = kumo.GCSConnector(root_dir="gs://my-bucket/kumo/")
3 adls = kumo.ADLSConnector(root_dir="abfss://container@account.dfs.core.windows.net/kumo/")
4 
5 orders = s3.table("orders")
6 customers = gcs["customers"]
7 transactions = adls.table("transactions")
8 
9 # Full-path access is also supported when no root_dir is configured.
10 events = kumo.S3Connector().table("s3://my-bucket/other/events/")

Warehouse connectors

Warehouse connectors create or attach to named Kumo connector resources. You can supply credentials directly or set the documented environment variables before constructing the connector.

Snowflake

1 snowflake = kumo.SnowflakeConnector(
2     name="prod_snowflake",
3     account="<account>",
4     warehouse="COMPUTE_WH",
5     database="ANALYTICS",
6     schema_name="PUBLIC",
7     credentials={"user": "<user>", "password": "<password>"},
8 )
9 
10 orders = snowflake.table("ORDERS")

SnowflakeConnector can also use SNOWFLAKE_USER, SNOWFLAKE_PASSWORD, SNOWFLAKE_PRIVATE_KEY, and SNOWFLAKE_PRIVATE_KEY_PASSPHRASE environment variables. In SPCS environments initialized with snowflake_credentials, those credentials are reused when connector credentials are omitted; when initialized from a Snowflake notebook with snowflake_application, the SDK can create a native Snowflake connector without explicit connector credentials.

Databricks

1 databricks = kumo.DatabricksConnector(
2     name="prod_databricks",
3     host="https://<workspace>.cloud.databricks.com",
4     cluster_id="<cluster_id>",
5     warehouse_id="<warehouse_id>",
6     catalog="main",
7     credentials={"token": "<personal_access_token>"},
8 )
9 
10 orders = databricks.table("sales.orders")

Use either a personal access token (token or DATABRICKS_TOKEN) or OAuth client credentials (client_id/client_secret or DATABRICKS_CLIENT_ID/DATABRICKS_CLIENT_SECRET), not both.

BigQuery

1 bigquery = kumo.BigQueryConnector(
2     name="prod_bigquery",
3     project_id="my-gcp-project",
4     dataset_id="analytics",
5     credentials={
6         "private_key_id": "<private_key_id>",
7         "private_key": "<private_key>",
8         "client_id": "<client_id>",
9         "client_email": "<client_email>",
10         "token_uri": "https://oauth2.googleapis.com/token",
11         "auth_uri": "https://accounts.google.com/o/oauth2/auth",
12     },
13 )
14 
15 orders = bigquery.table("orders")

Equivalent environment variables are BIGQUERY_PRIVATE_KEY_ID, BIGQUERY_PRIVATE_KEY, BIGQUERY_CLIENT_ID, BIGQUERY_CLIENT_EMAIL, BIGQUERY_TOKEN_URI, and BIGQUERY_AUTH_URI.

AWS Glue

1 glue = kumo.GlueConnector(
2     name="prod_glue",
3     account="123456789012",
4     region="us-west-2",
5     database="analytics",
6 )
7 
8 orders = glue.table("orders")

GlueConnector currently targets partitioned Parquet tables registered in the Glue Catalog.

Reusing existing connectors

For connectors that are stored in Kumo, retrieve a connector created earlier or created in the UI with get_by_name().

1 snowflake = kumo.SnowflakeConnector.get_by_name("prod_snowflake")
2 databricks = kumo.DatabricksConnector.get_by_name("prod_databricks")
3 bigquery = kumo.BigQueryConnector.get_by_name("prod_bigquery")
4 glue = kumo.GlueConnector.get_by_name("prod_glue")

Uploaded files

Use FileUploadConnector when you have a local file or a remote file/directory that should be uploaded into Kumo-managed storage before it is used as a source table.

1 parquet_uploads = kumo.FileUploadConnector(file_type="parquet")
2 parquet_uploads.upload(name="customers", path="/data/customers.parquet")
3 
4 csv_uploads = kumo.FileUploadConnector(file_type="csv")
5 csv_uploads.upload(name="events", path="gs://my-bucket/staging/events_csv/")
6 
7 customers = parquet_uploads["customers"]
8 events = csv_uploads.table("events")

Local files larger than about 1 GB are automatically partitioned by default. Remote single files must be about 1 GB or smaller; for larger remote data, upload a directory of same-format shards.

Group connectors

In RBAC-enabled workspaces, administrators can expose shared connectors at the group level. Users list and select those connectors instead of creating connectors with raw credentials.

1 kumo.init(
2     url="https://<customer_id>.kumoai.cloud/api",
3     api_key="<api_key>",
4     group="marketing",
5     project="churn-modeling",
6 )
7 
8 for connector in kumo.list_group_connectors():
9     print(connector.display_name, connector.source_type, connector.scope)
10 
11 connector = kumo.list_group_connectors()[0]
12 orders = connector.table("sales.orders")

If your workspace uses project scoping, initialize the SDK with the appropriate group and project before listing group connectors.

After selecting source tables

After a connector returns SourceTable objects, continue with table and graph construction:

1 orders = kumo.Table.from_source_table(
2     source_table=orders_src,
3     time_column="created_at",
4 ).infer_metadata()
5 
6 customers = kumo.Table.from_source_table(
7     source_table=customers_src,
8     primary_key="customer_id",
9 ).infer_metadata()

Next, see Tables and Graphs.