> This page is for version v1.4.0.
> For other versions, use one of these documentation indexes:
> - Latest (v1.5.1) (default): https://docs.nvidia.com/dynamo/latest/llms.txt
> - dev: https://docs.nvidia.com/dynamo/dev/llms.txt
> - v1.5.1: https://docs.nvidia.com/dynamo/v1.5.1/llms.txt
> - v1.5.0: https://docs.nvidia.com/dynamo/v1.5.0/llms.txt
> - v1.4.2: https://docs.nvidia.com/dynamo/v1.4.2/llms.txt
> - v1.4.1: https://docs.nvidia.com/dynamo/v1.4.1/llms.txt
> - v1.4.0: https://docs.nvidia.com/dynamo/v1.4.0/llms.txt
> - v1.3.0: https://docs.nvidia.com/dynamo/v1.3.0/llms.txt
> - v1.2.1: https://docs.nvidia.com/dynamo/v1.2.1/llms.txt
> - v1.2.0: https://docs.nvidia.com/dynamo/v1.2.0/llms.txt
> - v1.1.1: https://docs.nvidia.com/dynamo/v1.1.1/llms.txt
> - v1.1.0: https://docs.nvidia.com/dynamo/v1.1.0/llms.txt
> - v1.0.2: https://docs.nvidia.com/dynamo/v1.0.2/llms.txt
> - v1.0.1: https://docs.nvidia.com/dynamo/v1.0.1/llms.txt
> - v1.0.0: https://docs.nvidia.com/dynamo/v1.0.0/llms.txt
> - v0.9.1: https://docs.nvidia.com/dynamo/v-0-9-1/llms.txt
> - v0.9.0: https://docs.nvidia.com/dynamo/v-0-9-0/llms.txt
> - v0.8.1: https://docs.nvidia.com/dynamo/v-0-8-1/llms.txt
> - v0.8.0: https://docs.nvidia.com/dynamo/v-0-8-0/llms.txt
> - v0.7.1: https://docs.nvidia.com/dynamo/v-0-7-1/llms.txt
> - v0.7.0: https://docs.nvidia.com/dynamo/v-0-7-0/llms.txt

> For clean Markdown content of this page, append .md to this URL. For the complete documentation index, see https://docs.nvidia.com/dynamo/llms.txt. For full content including API reference and SDK examples, see https://docs.nvidia.com/dynamo/llms-full.txt.

# Benchmark a Kubernetes Deployment with AIPerf

AIPerf sends load to an OpenAI-compatible endpoint and measures latency and throughput. Use this
tutorial after deploying a model with a DynamoGraphDeployment (DGD).

AIPerf measures the live deployment. It does not predict a configuration like AIConfigurator or run
a GPU-free simulation like Mocker and DynoSim.

## Prerequisites

Install AIPerf on the workstation or benchmark host:

```bash
pip install aiperf
```

Set the namespace and forward the DGD frontend service to port 8000:

```bash
export NAMESPACE=dynamo-system
kubectl port-forward -n "$NAMESPACE" svc/<deployment-name>-frontend 8000:8000
```

The service name is the DGD `metadata.name` followed by `-frontend`.

#### Run a baseline benchmark

In another terminal, set `--model` to the model name served by the deployment:

```bash
aiperf profile \
  --model Qwen/Qwen3-32B-FP8 \
  --url http://localhost:8000 \
  --endpoint-type chat \
  --streaming \
  --concurrency 10 \
  --request-count 100 \
  --synthetic-input-tokens-mean 2000 \
  --output-tokens-mean 256
```

AIPerf writes artifacts to `artifacts/` and prints a metrics summary. Record Time to First Token
(TTFT), Inter-Token Latency (ITL), end-to-end latency, and output throughput.

#### Compare Kubernetes configurations

Change one deployment setting at a time, wait for the DGD to become ready, and rerun the same AIPerf
command. Common comparisons include:

* aggregated versus disaggregated serving
* KV-aware versus round-robin routing
* different worker replica counts
* different TP or PP sizes recommended by AIConfigurator
* backend or engine-argument changes

Keep the request shape, concurrency, and request count fixed so the results remain comparable. Then
sweep concurrency to find the saturation point of the selected deployment.

## Next steps

* Use [Sizing with AIConfigurator](/dynamo/v1.4.0/kubernetes/disaggregated-serving/size-with-ai-configurator) to select additional DGD
  configurations.
* Use the full [Dynamo Benchmarking guide](/dynamo/v1.4.0/recipes/benchmarks/benchmarking) for in-cluster load
  generation, concurrency sweeps, arrival patterns, trace replay, visualization, and GPU telemetry.
* Start from a [Dynamo Recipe](https://github.com/ai-dynamo/dynamo/tree/v1.4.0/recipes) when a supported
  model and hardware combination already has a known-good baseline.