> For clean Markdown content of this page, append .md to this URL. For the complete documentation index, see https://docs.nvidia.com/dynamo/llms.txt. For full content including API reference and SDK examples, see https://docs.nvidia.com/dynamo/llms-full.txt.

# Size a Kubernetes Deployment with AIConfigurator

This page is a detour from the **Determine topology and parallelism** step of [Deploy with DGD](/dynamo/dev/kubernetes/model-deployment/deploy-with-dgd). Rather than hand-pick tensor- and pipeline-parallel sizes, run AIConfigurator to search layouts against a latency target, then copy the recommended parallelism into your worker. It covers parallelism sizing only. For the full workflow — aggregated versus disaggregated comparison, generating complete manifests, and validating with AIPerf — see the [AIConfigurator](/dynamo/dev/additional-resources/ai-configurator-reference) reference.

AIConfigurator itself is not Kubernetes-specific. You can run the same optimizer for a local or
bare-metal Dynamo deployment and translate the recommended TP, PP, and replica counts into worker
commands. This tutorial is Kubernetes-specific only because its final step applies the result to a
DGD. For that workflow, see [Sizing a Local Deployment with AIConfigurator](/dynamo/dev/cli/model-deployment/sizing-with-ai-configurator).

AIConfigurator uses analytical and profiled performance models to estimate candidate configurations.
It is sometimes described as performance simulation, but it does not run the request-by-request
Dynamo scheduler, router, or KV-cache lifecycle. Mocker provides the simulated engine behavior, and
DynoSim supplies the replay and sweep harness; DynoSim can use AIConfigurator estimates as its
forward-pass timing model.

## Prerequisites

* A DGD in progress from [Deploy with DGD](/dynamo/dev/kubernetes/model-deployment/deploy-with-dgd), with the Frontend and worker defined and only parallelism left to set.
* AIConfigurator installed: `pip3 install aiconfigurator`.
* Your checkpoint's HuggingFace ID, GPU system (for example `h200_sxm`), backend (`vllm`, `sglang`, or `trtllm`), the total GPUs you can allocate, and your SLA: Time To First Token (TTFT), Time Per Output Token (TPOT), and typical input/output sequence lengths (ISL/OSL).

AIConfigurator only models supported checkpoint + system + backend combinations. Confirm yours before sizing with `aiconfigurator cli support --model-path <model> --system <sku> --backend <backend>`. See the [support matrix](https://ai-dynamo.github.io/aiconfigurator/support-matrix/).

#### Generate a configuration

Run `aiconfigurator cli default` with your GPU budget, system, backend, and SLA:

```bash
aiconfigurator cli default \
  --model-path Qwen/Qwen3-32B-FP8 \
  --total-gpus 8 \
  --system h200_sxm \
  --backend vllm \
  --backend-version 0.12.0 \
  --isl 4000 --osl 500 \
  --ttft 600 --tpot 16.67
```

* `--total-gpus` — GPUs available for the deployment; the search stays within this budget.
* `--system` — GPU SKU (`h200_sxm`, `h100_sxm`, `a100_sxm`).
* `--backend` / `--backend-version` — inference engine and version.
* `--isl` / `--osl` — input and output sequence lengths, in tokens.
* `--ttft` / `--tpot` — latency targets, in milliseconds; candidates that miss them are filtered out.

#### Understand the GPU and node model

`--total-gpus` is a budget, not a flat list with no topology. The selected `--system` definition
includes GPUs per node, intra-node bandwidth, inter-node bandwidth, PCIe bandwidth, and communication
latency. AIConfigurator uses that system model when evaluating parallel configurations and can
produce workers that span multiple nodes.

AIConfigurator does not inspect your live cluster. It assumes a homogeneous pool described by the
system definition and GPU budget; it does not account for current node availability, GPU
fragmentation, scheduler placement, or cluster-specific network differences.

The manual DGD example below assumes `gpus/worker` does not exceed the system's GPUs per node. If a
recommended worker spans nodes, use the generated multi-node artifacts from the full
[AIConfigurator workflow](/dynamo/dev/additional-resources/ai-configurator-reference) and make sure the
cluster supports the required LeaderWorkerSet deployment.

#### Read the recommended parallelism

AIConfigurator ranks the layouts that meet your SLA. The `parallel` column is the layout to copy, and `gpus/worker` is the GPUs it needs (abridged):

```text
agg Top Configurations: (Sorted by tokens/s/gpu)
+------+--------------+--------+----------+--------------+-------------+----------+
| Rank | tokens/s/gpu |  TTFT  | replicas | gpus/replica | gpus/worker | parallel |
+------+--------------+--------+----------+--------------+-------------+----------+
|  1   |    322.69    | 546.92 |    2     |      4       | 4 (=4x1x1)  |  tp4pp1  |
|  2   |    293.94    | 593.10 |    4     |      2       | 2 (=2x1x1)  |  tp2pp1  |
+------+--------------+--------+----------+--------------+-------------+----------+
```

Read the top-ranked row:

* **`parallel`** — `tp4pp1` means tensor-parallel 4, pipeline-parallel 1.
* **`gpus/worker`** — GPUs one worker needs; equals TP × PP.
* **`replicas`** — how many copies of the worker to run.

#### Apply it to your worker

Copy those three numbers into the worker you built in the DGD guide:

| AIConfigurator     | DGD field                                                                 |
| ------------------ | ------------------------------------------------------------------------- |
| `parallel: tp4pp1` | `--tensor-parallel-size 4` (add `--pipeline-parallel-size N` when PP > 1) |
| `gpus/worker: 4`   | `nvidia.com/gpu: "4"` for this single-node worker                         |
| `replicas: 2`      | the worker's `replicas`                                                   |

```yaml
  - name: VllmWorker
    type: worker
    replicas: 2                         # recommended replicas
    podTemplate:
      spec:
        containers:
        - name: main
          command:
          - /bin/bash
          - -c
          - exec python3 -m dynamo.vllm --model $MODEL --tensor-parallel-size 4
          resources:
            limits:
              nvidia.com/gpu: "4"       # must equal TP × PP
```

Disaggregating? The `disagg` table sizes prefill and decode separately, as `(p)parallel` / `(p)gpus/worker` and `(d)parallel` / `(d)gpus/worker`. Apply each to the matching prefill and decode worker from the DGD guide's disaggregated step.

## Next steps

* Return to [Deploy with DGD](/dynamo/dev/kubernetes/model-deployment/deploy-with-dgd) to apply the deployment and send a request.
* For aggregated versus disaggregated comparison, generating complete Kubernetes manifests with `--deployment-target dynamo-j2`, and validating predictions with AIPerf, see the [AIConfigurator](/dynamo/dev/additional-resources/ai-configurator-reference) reference.