> For clean Markdown content of this page, append .md to this URL. For the complete documentation index, see https://docs.nvidia.com/dynamo/llms.txt. For full content including API reference and SDK examples, see https://docs.nvidia.com/dynamo/llms-full.txt.

# Size a Local Deployment with AIConfigurator

AIConfigurator estimates inference performance across candidate parallelism and deployment layouts.
Use it before launching a local Dynamo deployment to choose tensor parallelism (TP), pipeline
parallelism (PP), and the number of worker replicas to validate.

AIConfigurator is an analytical performance model and configuration optimizer. It does not start a
Dynamo frontend, simulate request-by-request scheduler or KV-cache behavior, or benchmark a live
endpoint. Use [Mocker Live Simulation](/dynamo/dev/cli/operations/dynosim/mocker-live-simulation) for a live GPU-free deployment,
[DynoSim Replay](/dynamo/dev/kubernetes/operations/dynosim/simulation-runs) for request-level offline simulation, and
[Benchmarking with AIPerf](/dynamo/dev/cli/operations/benchmarking-with-ai-perf) for measurements against the running local endpoint.

## Prerequisites

Install AIConfigurator in the environment where you run the sizing command:

```bash
pip install aiconfigurator
```

Collect:

* the model ID or local checkpoint path
* the backend and backend version
* the GPU system
* the total GPU budget
* typical input and output sequence lengths
* Time to First Token (TTFT) and Time Per Output Token (TPOT) targets

Confirm that AIConfigurator supports the combination:

```bash
aiconfigurator cli support \
  --model-path Qwen/Qwen3-32B-FP8 \
  --system h200_sxm \
  --backend vllm
```

#### Generate candidate configurations

Run the optimizer with the local GPU budget and workload targets:

```bash
aiconfigurator cli default \
  --model-path Qwen/Qwen3-32B-FP8 \
  --total-gpus 8 \
  --system h200_sxm \
  --backend vllm \
  --backend-version 0.12.0 \
  --isl 4000 \
  --osl 500 \
  --ttft 600 \
  --tpot 16.67
```

`--total-gpus` is the budget available to the local deployment. The selected system definition
provides the assumed GPUs per node and communication characteristics. AIConfigurator does not inspect
which GPUs or nodes are currently free.

#### Apply a recommendation

Choose a highly ranked row that meets the SLA. Translate its output into the local worker launch:

| AIConfigurator output   | Local deployment setting                        |
| ----------------------- | ----------------------------------------------- |
| `parallel`              | Backend TP and PP arguments                     |
| `gpus/worker`           | GPUs exposed to each worker process             |
| `replicas`              | Number of worker processes to launch            |
| Prefill and decode rows | Separate worker pools for disaggregated serving |

For example, apply a `tp4pp1` recommendation to a vLLM worker:

```bash
CUDA_VISIBLE_DEVICES=0,1,2,3 python -m dynamo.vllm \
  --model Qwen/Qwen3-32B-FP8 \
  --tensor-parallel-size 4
```

Start additional replicas on separate GPU sets when the recommendation calls for them. All replicas
register with the same frontend. For backend-specific arguments, see the local deployment examples
for [vLLM](/dynamo/dev/recipes/cli-templates/v-llm),
[SGLang](/dynamo/dev/recipes/cli-templates/sg-lang), and
[TensorRT-LLM](/dynamo/dev/recipes/cli-templates/tensor-rt-llm).

If one worker requires more GPUs than a node provides, follow
[Multi-Node Deployment](/dynamo/dev/cli/model-deployment/multi-node-deployment) rather than treating the GPU budget as one flat
local device list.

#### Validate the recommendation

AIConfigurator narrows the configurations to test; it does not replace measurement. Launch the
recommended local deployment and use [Benchmarking with AIPerf](/dynamo/dev/cli/operations/benchmarking-with-ai-perf) to validate latency and
throughput on the target hardware.

For aggregated versus disaggregated analysis, generated artifacts, output fields, and advanced
options, see the [AIConfigurator Reference](/dynamo/dev/additional-resources/ai-configurator-reference).