> For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.nvidia.com/sdgm/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/sdgm/_mcp/server. # GPU Sizing for Docker > How to size the GPU host for a Kumo Docker Compose deployment. This page covers GPU host sizing for a Kumo Docker Compose deployment. GPU support is optional for evaluation; training and prediction jobs fall back to CPU when no GPU is present. For production GPU workloads, use the sizing tables below to select the right host before deploying. Two numbers drive every decision: | Input | What it means | How to find it | | ------- | --------------------------------------------------- | ----------------------------------------------------------------- | | `G_max` | The source size of your largest single graph, in GB | Sum the source table sizes of the largest graph you plan to build | | `C` | Your peak concurrent training and prediction jobs | How many models will realistically train at the same moment | `G_max` is the size Kumo reports, not the size your warehouse reports. Kumo measures the exported Parquet it writes to blob storage, which is typically larger than the figure shown in a warehouse console - a 2-3x difference is common. Budget 3x your warehouse estimate when planning a first deployment, and verify against the graph size shown in the Kumo UI. ## How Docker deployments use GPUs There is no pool in a Docker deployment. Kumo runs one worker on one host, with no autoscaling, no overflow pool, and no warm-cache routing. Sizing means sizing a single machine. Concurrency is a fixed setting rather than an elastic one. The `POOL_GPU_CONCURRENCY` environment variable controls how many training and prediction jobs run at once. It ships set to `1`. Every job beyond that count queues. Leaving `POOL_GPU_CONCURRENCY` at `1` on a multi-GPU host is the most common and most costly Docker misconfiguration. In Kumo's test data, jobs that queued behind another job took a median of 148 minutes against 32.5 minutes for jobs that ran immediately - roughly a 4.5x penalty from serialization. The GPU Compose overlay (`docker-compose.singlehost.gpu.yml`) exposes all available NVIDIA devices to the pool worker container using `count: all`. `POOL_GPU_CONCURRENCY` is the only throttle; set it to your physical GPU count. ## Choosing a size | | S | M | L | XL | | ----------------------------------- | -------- | -------- | -------- | -------- | | Largest graph `G_max` | ≤ 10 GB | ≤ 50 GB | ≤ 100 GB | ≤ 250 GB | | GPUs on host | 1 | 1 | 2 | 4 | | `POOL_GPU_CONCURRENCY` | `1` | `1` | `2` | `4` | | VRAM per GPU | 16-24 GB | 24-48 GB | 48 GB | 48-80 GB | | vCPU | 16 | 32 | 64 | 96+ | | RAM | 64 GB | 128 GB | 256 GB | 512 GB | | Local NVMe at `DISKGRAPH_DATA_ROOT` | 250 GB | 500 GB | 1 TB | 2 TB | | `shm_size` for the pool worker | 8g | 8g | 16g | 32g | | Concurrent jobs | 1 | 1 | 2 | 4 | Match the GPU count to your chosen tier first. vCPU and RAM vary between instance families, so you may need a larger variant than the minimum GPU count implies. Reference instances: | | AWS | Azure | Google Cloud | | ---------- | -------------- | --------------------------- | ---------------- | | S (1 GPU) | `g4dn.4xlarge` | `Standard_NC16as_T4_v3` | `g2-standard-16` | | M (1 GPU) | `g6e.8xlarge` | `Standard_NC40ads_H100_v5` | `g2-standard-32` | | L (2 GPU) | `g6e.12xlarge` | `Standard_NC80adis_H100_v5` | `g2-standard-24` | | XL (4 GPU) | `g6e.24xlarge` | - | `g2-standard-48` | On Azure, there is no 48 GB VRAM option. Azure steps from 24 GB to 80 GB or above. Treat the choice as binary: use 24 GB unless link prediction over large target tables forces you higher. ## Configuration **`POOL_GPU_CONCURRENCY`** - set this to your physical GPU count. This is the highest-value change available on a multi-GPU host. It is hardcoded as `"1"` in `docker-compose.singlehost.yml` under the `kumo-pool-worker` service; edit that value directly in the compose file (or supply a compose override file). Setting it in `.env` alone has no effect because the compose file does not use variable substitution for this key. * Never set it above your GPU count. This value is the only safeguard preventing two training processes from contending for a single device. **`DISKGRAPH_DATA_ROOT`** - the host path where DiskGraph stores graph indexes. Default: `/mnt/localssd`. Point this at a fast local SSD for real workloads. Size local disk at roughly 2x `G_max` (Kumo requires approximately 1.2x for the index, plus room for staging tables and temporary files). **`shm_size`** - set in `docker-compose.yml` on the `kumo-pool-worker` service, not in `.env`. The blueprint ships with `shm_size: 8g`. For L and XL hosts, increase it to the value shown in the sizing table above by editing the Compose file directly. Keep `ipc: host` and `shm_size` as shipped. Both are required for the GPU subprocess flow. Removing either causes training failures that are difficult to diagnose. Add memory limits to your Compose file for production use. The shipped file does not define them, so a runaway training process can exhaust the host. Set per-service limits appropriate to your available RAM. ## When to move to Helm Move to a Kubernetes deployment when any of the following becomes true: * You need more than approximately four concurrent training jobs. * Your graphs exceed approximately 250 GB. * You need elasticity - a Docker host incurs full cost whether idle or busy. * You need warm-cache reuse across many graphs. There is no in-place migration path between the two. Plan your deployment path before Docker becomes production-critical. ## Quick reference | Question | Answer | | ------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | How many GPUs do I need? | One per concurrent job. Set `POOL_GPU_CONCURRENCY` to match. | | How much VRAM per GPU? | 16-24 GB for S, 24-48 GB for M, 48 GB for L, 48-80 GB for XL. | | How much local SSD? | Roughly 2x `G_max`. | | What if I have 4 GPUs but `POOL_GPU_CONCURRENCY` is 1? | Only 1 job runs at a time - the other 3 GPUs are idle and jobs queue, at a \~4.5x time penalty. | | Is 48 GB VRAM available on Azure? | No. Azure steps from 24 GB to 80 GB or more. | | What are the most common mistakes? | Leaving `POOL_GPU_CONCURRENCY` at 1 on a multi-GPU host (it must be edited in the compose file, not `.env`); not pointing `DISKGRAPH_DATA_ROOT` at a fast SSD; removing `ipc: host` or `shm_size` from the Compose file. |