Kubernetes Templates

DGDR Templates

Ready-to-apply DynamoGraphDeploymentRequest manifests for profiling and generating DynamoGraphDeployments.

View as Markdown

A DynamoGraphDeploymentRequest (DGDR) describes a model, workload, hardware, and optional latency targets. Dynamo profiles that intent and generates the DynamoGraphDeployment (DGD) that serves traffic. Use these templates when you want Dynamo to choose a deployment configuration instead of writing the complete DGD yourself.

Each manifest is embedded from examples/deployments/dgdr/. Open an example, use the copy button, and adjust its model, backend, hardware, and namespace before applying it.

Prerequisites

  • Install the Dynamo Kubernetes platform and operator.
  • Create hf-token-secret in the target namespace when the model requires authentication.
  • For namespace-restricted operator installations, set hardware.gpuSku, hardware.vramMb, and hardware.numGpusPerNode explicitly.

Release installations select the matching profiler image when spec.image is omitted. For a local operator build without a known release version, set spec.image to a compatible dynamo-planner image.

Apply a template with:

$export NAMESPACE=dynamo-cloud
$kubectl apply -f rapid.yaml -n ${NAMESPACE}

Generate a DGD

Uses simulated performance estimates to generate a DGD without benchmarking candidate engines on real GPUs.

rapid.yaml
1# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
2# SPDX-License-Identifier: Apache-2.0
3#
4# Licensed under the Apache License, Version 2.0 (the "License");
5# you may not use this file except in compliance with the License.
6# You may obtain a copy of the License at
7#
8# http://www.apache.org/licenses/LICENSE-2.0
9#
10# Unless required by applicable law or agreed to in writing, software
11# distributed under the License is distributed on an "AS IS" BASIS,
12# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13# See the License for the specific language governing permissions and
14# limitations under the License.
15
16apiVersion: nvidia.com/v1beta1
17kind: DynamoGraphDeploymentRequest
18metadata:
19  name: qwen-rapid
20spec:
21  model: Qwen/Qwen3-0.6B
22  backend: vllm
23  searchStrategy: rapid
24  autoApply: true

Deploys and benchmarks candidate configurations before selecting and generating the DGD.

thorough.yaml
1# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
2# SPDX-License-Identifier: Apache-2.0
3#
4# Licensed under the Apache License, Version 2.0 (the "License");
5# you may not use this file except in compliance with the License.
6# You may obtain a copy of the License at
7#
8# http://www.apache.org/licenses/LICENSE-2.0
9#
10# Unless required by applicable law or agreed to in writing, software
11# distributed under the License is distributed on an "AS IS" BASIS,
12# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13# See the License for the specific language governing permissions and
14# limitations under the License.
15
16apiVersion: nvidia.com/v1beta1
17kind: DynamoGraphDeploymentRequest
18metadata:
19  name: qwen-thorough
20spec:
21  model: Qwen/Qwen3-0.6B
22  backend: vllm
23  searchStrategy: thorough
24  autoApply: true

Adds a Planner configuration so the generated disaggregated deployment can adjust prefill and decode replicas at runtime.

planner.yaml
1# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
2# SPDX-License-Identifier: Apache-2.0
3#
4# Licensed under the Apache License, Version 2.0 (the "License");
5# you may not use this file except in compliance with the License.
6# You may obtain a copy of the License at
7#
8# http://www.apache.org/licenses/LICENSE-2.0
9#
10# Unless required by applicable law or agreed to in writing, software
11# distributed under the License is distributed on an "AS IS" BASIS,
12# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13# See the License for the specific language governing permissions and
14# limitations under the License.
15
16apiVersion: nvidia.com/v1beta1
17kind: DynamoGraphDeploymentRequest
18metadata:
19  name: qwen-planner
20spec:
21  model: Qwen/Qwen3-0.6B
22  backend: vllm
23  searchStrategy: rapid
24  workload:
25    isl: 4000
26    osl: 1000
27    requestRate: 10
28  sla:
29    ttft: 500
30    itl: 50
31  features:
32    planner:
33      optimization_target: sla
34      enable_throughput_scaling: true
35      enable_load_scaling: false
36      pre_deployment_sweeping_mode: rapid
37      mode: disagg
38      backend: vllm
39  autoApply: true

Uses SGLang and a 32-GPU budget for a large Mixture-of-Experts model. Adapt its hardware, scheduling, model-cache, and SLA settings before applying it.

moe-sglang.yaml
1# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
2# SPDX-License-Identifier: Apache-2.0
3#
4# Licensed under the Apache License, Version 2.0 (the "License");
5# you may not use this file except in compliance with the License.
6# You may obtain a copy of the License at
7#
8# http://www.apache.org/licenses/LICENSE-2.0
9#
10# Unless required by applicable law or agreed to in writing, software
11# distributed under the License is distributed on an "AS IS" BASIS,
12# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13# See the License for the specific language governing permissions and
14# limitations under the License.
15
16apiVersion: nvidia.com/v1beta1
17kind: DynamoGraphDeploymentRequest
18metadata:
19  name: deepseek-r1
20spec:
21  model: deepseek-ai/DeepSeek-R1
22  backend: sglang
23  searchStrategy: thorough
24  hardware:
25    totalGpus: 32
26    numGpusPerNode: 8
27  workload:
28    isl: 4000
29    osl: 1000
30  sla:
31    ttft: 2000
32    itl: 100
33  autoApply: false

Review and Customize the Generated DGD

Sets autoApply: false so the request stops after generation instead of creating the DGD.

review-before-deploy.yaml
1# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
2# SPDX-License-Identifier: Apache-2.0
3#
4# Licensed under the Apache License, Version 2.0 (the "License");
5# you may not use this file except in compliance with the License.
6# You may obtain a copy of the License at
7#
8# http://www.apache.org/licenses/LICENSE-2.0
9#
10# Unless required by applicable law or agreed to in writing, software
11# distributed under the License is distributed on an "AS IS" BASIS,
12# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13# See the License for the specific language governing permissions and
14# limitations under the License.
15
16apiVersion: nvidia.com/v1beta1
17kind: DynamoGraphDeploymentRequest
18metadata:
19  name: qwen-review
20spec:
21  model: Qwen/Qwen3-0.6B
22  backend: vllm
23  searchStrategy: rapid
24  autoApply: false

Merges an environment-variable override into the generated frontend component. Overrides can modify generated components but cannot add a new topology component.

generated-dgd-override.yaml
1# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
2# SPDX-License-Identifier: Apache-2.0
3#
4# Licensed under the Apache License, Version 2.0 (the "License");
5# you may not use this file except in compliance with the License.
6# You may obtain a copy of the License at
7#
8# http://www.apache.org/licenses/LICENSE-2.0
9#
10# Unless required by applicable law or agreed to in writing, software
11# distributed under the License is distributed on an "AS IS" BASIS,
12# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13# See the License for the specific language governing permissions and
14# limitations under the License.
15
16apiVersion: nvidia.com/v1beta1
17kind: DynamoGraphDeploymentRequest
18metadata:
19  name: qwen-router-override
20spec:
21  model: Qwen/Qwen3-0.6B
22  backend: vllm
23  searchStrategy: rapid
24  overrides:
25    dgd:
26      apiVersion: nvidia.com/v1beta1
27      kind: DynamoGraphDeployment
28      spec:
29        components:
30          - name: Frontend
31            podTemplate:
32              spec:
33                containers:
34                  - name: main
35                    env:
36                      - name: DYN_ROUTER_MODE
37                        value: kv
38  autoApply: false

Mounts a pre-populated model cache into the profiling job and generated workers. Replace the snapshot placeholder and create the PVC before applying the request.

model-cache.yaml
1# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
2# SPDX-License-Identifier: Apache-2.0
3#
4# Licensed under the Apache License, Version 2.0 (the "License");
5# you may not use this file except in compliance with the License.
6# You may obtain a copy of the License at
7#
8# http://www.apache.org/licenses/LICENSE-2.0
9#
10# Unless required by applicable law or agreed to in writing, software
11# distributed under the License is distributed on an "AS IS" BASIS,
12# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13# See the License for the specific language governing permissions and
14# limitations under the License.
15
16apiVersion: nvidia.com/v1beta1
17kind: DynamoGraphDeploymentRequest
18metadata:
19  name: qwen-model-cache
20spec:
21  model: Qwen/Qwen3-0.6B
22  backend: vllm
23  searchStrategy: rapid
24  modelCache:
25    pvcName: model-cache
26    pvcMountPath: /home/dynamo/.cache/huggingface
27    pvcModelPath: hub/models--Qwen--Qwen3-0.6B/snapshots/<commit-hash>
28  autoApply: true

Test and Inspect Profiling

Replaces model-serving workers with mock workers for testing routing, profiling, and Planner behavior without serving the model on GPUs.

mocker.yaml
1# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
2# SPDX-License-Identifier: Apache-2.0
3#
4# Licensed under the Apache License, Version 2.0 (the "License");
5# you may not use this file except in compliance with the License.
6# You may obtain a copy of the License at
7#
8# http://www.apache.org/licenses/LICENSE-2.0
9#
10# Unless required by applicable law or agreed to in writing, software
11# distributed under the License is distributed on an "AS IS" BASIS,
12# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13# See the License for the specific language governing permissions and
14# limitations under the License.
15
16apiVersion: nvidia.com/v1beta1
17kind: DynamoGraphDeploymentRequest
18metadata:
19  name: qwen-mocker
20spec:
21  model: Qwen/Qwen3-0.6B
22  backend: vllm
23  searchStrategy: rapid
24  features:
25    mocker:
26      enabled: true
27  autoApply: true

Replaces the profiling output volume with the existing dynamo-pvc claim so plots, logs, configurations, and raw profiling data remain available after the Job completes.

profiling-artifacts.yaml
1# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
2# SPDX-License-Identifier: Apache-2.0
3#
4# Licensed under the Apache License, Version 2.0 (the "License");
5# you may not use this file except in compliance with the License.
6# You may obtain a copy of the License at
7#
8# http://www.apache.org/licenses/LICENSE-2.0
9#
10# Unless required by applicable law or agreed to in writing, software
11# distributed under the License is distributed on an "AS IS" BASIS,
12# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13# See the License for the specific language governing permissions and
14# limitations under the License.
15
16apiVersion: nvidia.com/v1beta1
17kind: DynamoGraphDeploymentRequest
18metadata:
19  name: qwen-profiling-artifacts
20spec:
21  model: Qwen/Qwen3-0.6B
22  backend: vllm
23  searchStrategy: thorough
24  overrides:
25    profilingJob:
26      template:
27        spec:
28          containers: []
29          volumes:
30            - name: profiling-output
31              persistentVolumeClaim:
32                claimName: dynamo-pvc
33  autoApply: false

Inspect the Result

Watch the request until it reaches Deployed or Ready:

$kubectl get dgdr <request-name> -n ${NAMESPACE} -w

For a request with autoApply: false, extract the generated DGD:

$kubectl get dgdr <request-name> -n ${NAMESPACE} \
>  -o jsonpath='{.status.profilingResults.selectedConfig}' > generated-dgd.yaml

See Auto Deploy with DGDR for the task-oriented workflow and the DGDR Reference for field definitions, lifecycle phases, and validation behavior.