> For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.nvidia.com/sdgm/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/sdgm/_mcp/server. # Kumo Engine Prerequisites > Prepare Python, compute, local storage, and data access for Kumo Engine Prepare your Python environment, local storage, and data access before following an environment-specific setup guide. These prerequisites apply to Kumo running in your notebook or Linux Python process. ## 1. Python and Package Access Use Python 3.12 for the documented Linux and notebook setup paths, with `pip` available in the environment that will run Kumo. You need: * **Package access:** Access to the Python package index that supplies Kumo Engine and its dependencies, including any required index credentials. * **Installation permissions:** Permission to install packages into the interpreter used by your script or notebook, and to restart the notebook's Python process when required. On Linux, use a dedicated virtual environment to isolate dependencies. Choose one installation command for your environment and source: | Use | Installation command | | -------------------------------- | ------------------------------------------ | | Base Kumo Engine for local files | `pip install kumo-engine` | | Amazon S3 integration | `pip install "kumo-engine[s3]"` | | Google Cloud Storage integration | `pip install "kumo-engine[gcs]"` | | BigQuery integration | `pip install "kumo-engine[bigquery]"` | | Snowflake integration | `pip install "kumo-engine[snowflake]"` | | Databricks integration | `pip install "kumo-engine[databricks]"` | | Databricks with Amazon S3 | `pip install "kumo-engine[databricks,s3]"` | Each integration option installs Kumo Engine with the dependencies for that connector. The Python import name is `kumo.engine` for all installation choices. Use the Kumo Engine build supplied or approved for your selected environment, Python version, and PyTorch/CUDA stack. These guides use [Kumo Engine 1.0.0 on public PyPI](https://pypi.org/project/kumo-engine/1.0.0/). The published package targets Linux x86-64 and requires Python 3.11 or later; the documented setup paths below use Python 3.12. The matching native dependencies must also be available for your environment and PyTorch/CUDA stack. Follow your environment's setup guide for notebook command syntax, storage, and identity. ## 2. Documented Environment Targets Use the environment version listed for your deployment with the matching Kumo build. The environment version alone does not identify a compatible Kumo artifact. | Setup path | Environment | Python | Runtime notes | | -------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------ | ---------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Snowflake notebook | [Snowflake GPU Container Runtime 2.9](https://docs.snowflake.com/en/developer-guide/snowflake-ml/container-runtime/releases/gpu/2_9) | 3.12 | PyTorch `2.9.1+cu129`; CUDA toolkit 12.9 | | Databricks notebook | [Serverless GPU environment version 6](https://docs.databricks.com/aws/en/release-notes/serverless/environment-version/six-gpu) | 3.12 | Select the version 6 AI base environment and a GPU accelerator. Use `/tmp/kumoai-local` for scratch. | | Databricks notebook | [Databricks Runtime 17.3 LTS ML GPU](https://learn.microsoft.com/en-us/azure/databricks/release-notes/runtime/17.3lts-ml) | 3.12 | Retain the supported PyTorch `2.7.0`. PyTorch uses its CUDA PyPI dependencies; the built-in CUDA libraries do not establish its effective target. Verify `torch.version.cuda` in the notebook. | | Databricks notebook | [Databricks Runtime 16.4 LTS ML GPU](https://learn.microsoft.com/en-us/azure/databricks/release-notes/runtime/16.4lts-ml) | 3.12 | Preinstalled PyTorch `2.6.0+cu124` requires a manual upgrade to PyTorch `2.9.0` with CUDA 12.6 before installing Kumo. Supported only after the upgrade in [Databricks Notebook Setup](/kumo-engine/1.0.0/get-started-with-kumo-engine/installation/databricks-notebook#install-kumo). | | Linux machine | [Ubuntu 24.04 x86_64](https://releases.ubuntu.com/noble/) | 3.12 | PyTorch and CUDA are selected by the Kumo Engine build | | [Google Colab](/kumo-engine/1.0.0/user-code-guide/code-examples/google-colab-notebook) | Runtime 2026.07 | Runtime supplied | Use local files as the data source. | | SageMaker | SageMaker Distribution 4.5 (GPU image) | Runtime supplied | Use Amazon S3 as the data source. | Google Colab and SageMaker are suitable for small or introductory graphs and models. Python compatibility alone is not sufficient for GPU execution. Use PyTorch and `torch-geometric` versions compatible with your Kumo build and CUDA target. Kumo no longer requires the separate `pyg-lib` or `torch-scatter` packages. Follow the runtime-specific installation steps in your setup guide, including the required PyTorch upgrade for Databricks 16.4. Otherwise, do not replace runtime packages independently unless Kumo release instructions provide a tested stack for your environment. Run the setup guide's verification cell after installation to check imports, effective versions, and GPU execution. ## 3. Compute The GPU must be accessible to the Python process running Kumo. Use the shared [GPU Sizing](/kumo-engine/1.0.0/get-started-with-kumo-engine/gpu-sizing) guide to choose CPU, RAM, and scratch capacity from the total source Parquet size, then check GPU memory for your model. ## 4. Local Storage **Kumo jobs require local storage for execution.** Place job working files and scratch data on that storage. Use local SSD storage, such as NVMe or SATA SSDs. Verify that the storage is local to the compute machine rather than relying on its device or mount name. A writable directory alone does not meet this requirement. For the S3 workflow on AWS, use EC2 compute with local instance storage. **Amazon EBS volumes are not supported as execution storage for these jobs.** Source data can reside in S3; job execution still requires local storage on the compute machine. For an AWS Databricks cluster that runs Kumo jobs, disable elastic disk and EBS volumes in the cluster configuration: ```json { "enable_elastic_disk": false, "aws_attributes": { "ebs_volume_count": 0 } } ``` After the cluster starts, inspect its effective configuration and confirm that both values remain set. A cluster policy can override requested settings. This configuration is specific to AWS Databricks and does not apply to Azure compute. Verify the storage backing your configured directories. An `/dev/nvme*` device name is not sufficient evidence: AWS also exposes EBS volumes through an NVMe interface. Refer to [Amazon EBS volumes and NVMe](https://docs.aws.amazon.com/ebs/latest/userguide/nvme-ebs-volumes.html). Provide a writable scratch directory with enough space for staged source data, graph indexes, intermediate files, and prediction outputs. Configure the directory for your environment: * **Linux:** Choose a directory on your mounted local SSD and set it as Kumo's scratch directory. See [Linux working storage setup](/kumo-engine/1.0.0/get-started-with-kumo-engine/installation/linux-machine-deployment#configure-working-storage). * **Snowflake notebook:** Choose a directory on the compute pool's local storage. The setup example uses `/tmp/kumoai-local`; verify that this path is on the intended local disk before using it. See [Snowflake working storage setup](/kumo-engine/1.0.0/get-started-with-kumo-engine/installation/snowflake-notebook#configure-working-storage). * **Databricks notebook:** Use `/tmp/kumoai-local` on Serverless GPU or `/local_disk0/kumo-scratch` on a classic dedicated GPU cluster. Confirm that the path is backed by local storage on your selected compute. Prepare a Unity Catalog volume for model and prediction files you need to retain. See [Databricks working storage setup](/kumo-engine/1.0.0/get-started-with-kumo-engine/installation/databricks-notebook#configure-working-storage). Confirm how long that storage survives notebook restarts or machine shutdowns. An artifact directory does not automatically provide durable storage. Follow [Publish and Retrieve Models](/kumo-engine/1.0.0/user-code-guide/train-a-model/publish-and-retrieve-models) to retain an independent model copy and identify the other artifacts that need separate retention. ## 5. Credentials and Data Permissions Choose a deployment and connector below. Follow its setup and connector instructions to configure credentials and data permissions. | Deployment | Connector | | ------------------- | ---------------------------------------------------------------------------------------------------------------- | | Google Colab | [Local files](/kumo-engine/1.0.0/user-code-guide/connect-your-data-linux#local-files) | | SageMaker | [Amazon S3](/kumo-engine/1.0.0/user-code-guide/connect-your-data-linux#amazon-s3) | | Linux | [Local files](/kumo-engine/1.0.0/user-code-guide/connect-your-data-linux#local-files) | | Linux | [Amazon S3](/kumo-engine/1.0.0/user-code-guide/connect-your-data-linux#amazon-s3) | | Linux | [Google Cloud Storage](/kumo-engine/1.0.0/user-code-guide/connect-your-data-linux#google-cloud-storage) | | Linux | [BigQuery](/kumo-engine/1.0.0/user-code-guide/connect-your-data-linux#bigquery) | | Linux | [Snowflake](/kumo-engine/1.0.0/user-code-guide/connect-your-data-linux#snowflake) | | Linux | [Databricks](/kumo-engine/1.0.0/user-code-guide/connect-your-data-linux#linux-to-databricks) | | Snowflake notebook | [Snowflake warehouse](/kumo-engine/1.0.0/user-code-guide/connect-your-data-snowflake#snowflake-tables-and-views) | | Databricks notebook | [Amazon S3](/kumo-engine/1.0.0/user-code-guide/connect-your-data-databricks#databricks-s3) | | Databricks notebook | [Azure Data Lake Storage](/kumo-engine/1.0.0/user-code-guide/connect-your-data-databricks#databricks-adls) | | Databricks notebook | [Unity Catalog](/kumo-engine/1.0.0/user-code-guide/connect-your-data-databricks#databricks-unity-catalog) | ## Continue to Setup Choose the guide for where your Python code will run: * [Google Colab Notebook Example](/kumo-engine/1.0.0/user-code-guide/code-examples/google-colab-notebook) for a small or introductory local-file workflow * [Snowflake Notebook Setup](/kumo-engine/1.0.0/get-started-with-kumo-engine/installation/snowflake-notebook) * [Databricks Notebook Setup](/kumo-engine/1.0.0/get-started-with-kumo-engine/installation/databricks-notebook) * [Linux Machine Deployment Setup](/kumo-engine/1.0.0/get-started-with-kumo-engine/installation/linux-machine-deployment) for local files, S3, GCS, BigQuery, Snowflake, or Databricks These guides configure execution, storage, and identity. Connector creation follows setup.