DGDR Reference

A DynamoGraphDeploymentRequest (DGDR) is Dynamo’s deploy-by-intent generator for DynamoGraphDeployment (DGD) resources. You describe what you want to run and your performance targets; the profiler determines a configuration and produces the DGD that serves traffic.

For the full deployment mental model — including DGD, DCD, DGDR, recipes, strategy selection, model caching, planner setup, and common pitfalls — see the Deployment Overview.

DGDR, DGD, and Recipes

Dynamo provides two Custom Resources for deploying inference graphs:

Use DGD directly when you have a hand-crafted configuration for a specific model/hardware combination. Most recipes are tuned DGD manifests. Use DGDR when you want Dynamo to generate the DGD for you.

For DGD deployment details, see Creating Deployments.

Spec Reference

Minimal Example

1
apiVersion: nvidia.com/v1beta1
2
kind: DynamoGraphDeploymentRequest
3
metadata:
4
  name: my-model
5
spec:
6
  model: Qwen/Qwen3-0.6B
7
  image: "nvcr.io/nvidia/ai-dynamo/dynamo-planner:1.2.1"  # dynamo-frontend for Dynamo < 1.1.0

Field Reference

For the complete CRD spec, see the API Reference.

Planner

DGDR supports Planner through spec.features.planner. Set this field to a PlannerConfig object to have DGDR pass that configuration to the profiler and generate Planner support in the final DGD. DGDR passes this PlannerConfig through without field-level validation; the Planner service validates it when it starts.

When Planner is enabled, the generated output may include a Planner service in the DGD plus supporting Planner configuration resources, such as a planner-config-* ConfigMap. Depending on profiling mode and Planner settings, DGDR may also generate profiling-data resources for Planner bootstrap data.

Minimal Planner-enabled DGDR:

1
apiVersion: nvidia.com/v1beta1
2
kind: DynamoGraphDeploymentRequest
3
metadata:
4
  name: qwen3-planner
5
spec:
6
  model: Qwen/Qwen3-0.6B
7
  backend: vllm
8
  image: "nvcr.io/nvidia/ai-dynamo/dynamo-planner:1.2.1"  # dynamo-frontend for Dynamo < 1.1.0
9
  features:
10
    planner:
11
      mode: disagg
12
      backend: vllm

To evaluate Planner recommendations without applying scaling changes, enable advisory mode in the same features.planner object:

1
spec:
2
  features:
3
    planner:
4
      mode: disagg
5
      backend: vllm
6
      advisory: true

For Planner behavior, scaling modes, and the full PlannerConfig field reference, see the Planner overview and Planner Guide. For additional generated-deployment examples, see DGDR Examples.

spec.overrides.dgd is not required to enable Planner. Use spec.features.planner for Planner enablement and configuration. Use spec.overrides.dgd only when you need to customize the generated DGD after DGDR has assembled it.

Generated DGD Overrides

Use spec.overrides.dgd when the generated DynamoGraphDeployment needs a field that DGDR does not expose directly. Provide a partial, versioned DGD with kind: DynamoGraphDeployment and either apiVersion: nvidia.com/v1beta1 or apiVersion: nvidia.com/v1alpha1. Use v1beta1 for new DGDRs.

The override is applied to each complete DGD that the profiler materializes:

1. The profiler generates a complete DGD blueprint.
2. If the blueprint and override use different API versions, Dynamo converts the complete blueprint to the override’s version.
3. Dynamo merges the override according to that version’s structural schema.
4. Dynamo converts the complete merged result back to the blueprint’s version.
5. The DGDR controller records .status.profilingResults.selectedConfig as a nvidia.com/v1beta1 DGD and creates a v1beta1 DGD when autoApply is enabled.

Dynamo never converts the partial override by itself. This avoids filling omitted fields with conversion defaults before the merge.

For example, this v1beta1 override sets a graph-level environment variable that is prepended to every generated component. The spec.env list is atomic, so include any generated graph-level entries that must be retained:

1
apiVersion: nvidia.com/v1beta1
2
kind: DynamoGraphDeploymentRequest
3
metadata:
4
  name: qwen3-sglang
5
spec:
6
  model: Qwen/Qwen3-30B-A3B
7
  backend: sglang
8
  image: "nvcr.io/nvidia/ai-dynamo/dynamo-planner:1.2.1"  # dynamo-frontend for Dynamo < 1.1.0
9
  overrides:
10
    dgd:
11
      apiVersion: nvidia.com/v1beta1
12
      kind: DynamoGraphDeployment
13
      spec:
14
        env:
15
          - name: TRITON_PTXAS_PATH
16
            value: /usr/local/cuda/bin/ptxas

Use spec.env for variables that apply to all generated components. To target one component, identify it by name. Nested map lists, including containers and env, also merge by name:

1
spec:
2
  overrides:
3
    dgd:
4
      apiVersion: nvidia.com/v1beta1
5
      kind: DynamoGraphDeployment
6
      spec:
7
        components:
8
          - name: VllmDecodeWorker  # replace with a generated component name
9
            podTemplate:
10
              spec:
11
                containers:
12
                  - name: main
13
                    env:
14
                      - name: CUSTOM_WORKER_ENV
15
                        value: "enabled"

Inspect .status.profilingResults.selectedConfig with autoApply: false when you need the generated component names.

Merge behavior depends on the override’s declared API version:

Both versions follow these rules:

• When overriding topology entries, an override can update only services or components present in the generated blueprint. Dynamo ignores unknown names and reports a warning; an override cannot add deployment topology.
• metadata.labels and metadata.annotations merge into the generated DGD. metadata.name selects the final DGD name. Other identity and runtime metadata, such as namespace and finalizers, is ignored.
• status overrides and null values for typed fields are rejected. Field deletion is not supported.

The operator injects the override helper into Kubernetes profiling jobs. For direct profiler runs, install the matching helper as described in Local Runs with DGD Overrides.

Routing

DGDR-generated deployments include a standalone Frontend component. That frontend runs Dynamo’s embedded router and defaults to round-robin routing, which is often not optimal. Because DGDR does not yet expose a first-class router feature, configure the generated frontend with spec.overrides.dgd.

For the full router mode and environment variable reference, see Router Guide and Router Configuration.

For example, enable KV-aware routing on the generated frontend:

1
apiVersion: nvidia.com/v1beta1
2
kind: DynamoGraphDeploymentRequest
3
metadata:
4
  name: qwen3-kv-router
5
spec:
6
  model: Qwen/Qwen3-0.6B
7
  backend: vllm
8
  overrides:
9
    dgd:
10
      apiVersion: nvidia.com/v1beta1
11
      kind: DynamoGraphDeployment
12
      spec:
13
        components:
14
          - name: Frontend
15
            podTemplate:
16
              spec:
17
                containers:
18
                  - name: main
19
                    env:
20
                      - name: DYN_ROUTER_MODE
21
                        value: kv

Use the same Frontend override for other frontend router modes, such as random, least-loaded, or device-aware-weighted. For normal DGDR deployments, use kv when you want prefix-cache-aware routing and round-robin or least-loaded when you only want load balancing. Use direct only when an external router supplies explicit worker IDs in the request routing hints. For detailed mode definitions, see Router Guide.

KV-aware routing can use event-driven prefix-cache state or approximate prefix matching. The frontend still runs in kv mode in both cases. If you do not configure worker KV-event publication, set DYN_ROUTER_USE_KV_EVENTS=false to use approximate KV mode:

1
spec:
2
  overrides:
3
    dgd:
4
      apiVersion: nvidia.com/v1beta1
5
      kind: DynamoGraphDeployment
6
      spec:
7
        components:
8
          - name: Frontend
9
            podTemplate:
10
              spec:
11
                containers:
12
                  - name: main
13
                    env:
14
                      - name: DYN_ROUTER_MODE
15
                        value: kv
16
                      - name: DYN_ROUTER_USE_KV_EVENTS
17
                        value: "false"

For event-driven prefix-cache state, enable worker event publication only where prefill happens: the single worker in aggregated serving, or prefill workers in disaggregated serving. Decode workers are scored by load (dyn-decode-scorer), not prefix overlap (dyn-prefill-scorer), so vLLM decode workers omit both --enable-prefix-caching and --kv-events-config. Component names depend on the selected backend and topology, so inspect the generated DGD first, especially when autoApply: false.

For example, a generated vLLM disaggregated deployment may contain a VllmPrefillWorker component. This override appends the vLLM KV-event publishing arguments to that component while enabling the frontend KV router:

1
spec:
2
  overrides:
3
    dgd:
4
      apiVersion: nvidia.com/v1alpha1
5
      kind: DynamoGraphDeployment
6
      spec:
7
        services:
8
          Frontend:
9
            envs:
10
              - name: DYN_ROUTER_MODE
11
                value: kv
12
          VllmPrefillWorker:
13
            extraPodSpec:
14
              mainContainer:
15
                args:
16
                  - --enable-prefix-caching
17
                  - --kv-events-config
18
                  - '{"publisher":"zmq","topic":"kv-events","endpoint":"tcp://*:20080","enable_kv_cache_events":true}'

Worker KV-event flags are backend-specific. For cross-backend behavior, see Router Operations.

In Kubernetes deployments the Dynamo runtime normally uses Kubernetes discovery and the NATS event plane. Some backends, such as vLLM and SGLang, emit raw KV events over ZMQ; the Dynamo worker consumes those backend events and republishes router events through the Dynamo event plane. For the event plane model, see Event Plane.

EPP and Gateway Routing

EPP/Gateway routing is a different topology from the standalone frontend that DGDR generates:

client -> Gateway -> EPP selects worker -> worker frontend sidecar -> engine

In this mode the EPP owns worker selection. The worker-local frontend sidecar must run with --router-mode direct so it honors the worker IDs selected by EPP. In the normal Gateway path, the selected endpoint and the frontend sidecar are the same worker pod; if they differ, direct mode can still forward to the worker ID supplied by EPP.

DGDR does not currently generate EPP components or frontend sidecars. Also, overrides.dgd only patches services or components that already exist in the generated DGD, so it cannot add a missing Epp component to a DGDR-generated deployment. Use a direct DGD manifest or a GAIE recipe for EPP deployments. For manifests, frontendSidecar configuration, direct routing, EPP routing variables such as DYN_USE_KV_EVENTS, and route setup, see Gateway API Inference Extension.

SKU Format

When providing hardware configuration manually, use lowercase underscore format:

All supported values: gb200_sxm, b200_sxm, h200_sxm, h100_sxm, h100_pcie, a100_sxm, a100_pcie, a30, l40s, l40, l4, v100_sxm, v100_pcie, t4, mi200, mi300.

Lifecycle

When you create a DGDR, it progresses through these phases:

Conditions

The operator maintains these conditions on the DGDR status:

Monitoring

$
# Watch phase transitions
$
kubectl get dgdr my-model -n $NAMESPACE -w
$
$
# Detailed status, conditions, and events
$
kubectl describe dgdr my-model -n $NAMESPACE
$
$
# Profiling sub-phase
$
kubectl get dgdr my-model -n $NAMESPACE -o jsonpath='{.status.profilingPhase}'
$
$
# Profiling job logs
$
kubectl get pods -n $NAMESPACE -l nvidia.com/dgdr-name=my-model
$
kubectl logs -f <profiling-pod-name> -n $NAMESPACE
$
$
# View generated DGD spec (when autoApply: false)
$
kubectl get dgdr my-model -n $NAMESPACE \
>
  -o jsonpath='{.status.profilingResults.selectedConfig}' | python3 -m json.tool

Resource Ownership

• The DGDR does not set an owner reference on the DGD it creates. Deleting a DGDR does not delete the DGD — it persists independently so it can continue serving traffic.
• The relationship is tracked via labels: dgdr.nvidia.com/name and dgdr.nvidia.com/namespace.
• Additional resources (planner ConfigMaps) are created in the same namespace and labeled with dgdr.nvidia.com/name.

Known Issues

• pareto_analysis.py produces NaN for some configurations. Tracked as an engineering follow-up. Workaround: re-run with a narrower sweep; narrow sweeps bypass the NaN path in practice.
• PCIe profiler data not yet available. See the PCIe callout under SKU Format.

Further Reading

• Deployment Overview — DGD, DCD, DGDR, recipes, strategy selection, and common pitfalls
• Profiler Guide — Profiling algorithms, picking modes, gate checks
• Profiler Examples — Ready-to-use YAML for SLA targets, private models, MoE, overrides
• Planner Guide — Scaling modes, PlannerConfig reference
• API Reference — Complete CRD field specifications
• Creating Deployments — DGD spec for full manual control