DGDR Examples

Practical examples for deploying with DynamoGraphDeploymentRequest (DGDR). The DGDR workflow can use native AIC estimates, optional bootstrap profiling data, or live FPM warmup depending on the model/backend combination. For DGDR concepts, see the DGDR Reference. For profiling concepts, see the Profiler Guide.

DGDR Examples

Minimal DGDR with AIC (Fastest)

The simplest way to generate a deployment from native AIC estimates. Uses AI Configurator for offline profiling (20-30 seconds instead of hours):

1
apiVersion: nvidia.com/v1beta1
2
kind: DynamoGraphDeploymentRequest
3
metadata:
4
  name: sla-aic
5
spec:
6
  model: Qwen/Qwen3-32B
7
  backend: vllm
8
  image: "nvcr.io/nvidia/ai-dynamo/dynamo-planner:1.2.1"  # dynamo-frontend for Dynamo < 1.1.0

Deploy:

$
export NAMESPACE=your-namespace
$
# Save the manifest above as sla-aic.yaml first.
$
kubectl apply -f sla-aic.yaml -n $NAMESPACE

Online Profiling (Real Measurements)

Standard online profiling runs real GPU measurements for more accurate results. Takes 2-4 hours:

1
apiVersion: nvidia.com/v1beta1
2
kind: DynamoGraphDeploymentRequest
3
metadata:
4
  name: sla-online
5
spec:
6
  model: meta-llama/Llama-3.3-70B-Instruct
7
  backend: vllm
8
  image: "nvcr.io/nvidia/ai-dynamo/dynamo-planner:1.2.1"  # dynamo-frontend for Dynamo < 1.1.0
9
  searchStrategy: thorough

Deploy:

$
# Save the manifest above as sla-online.yaml first.
$
kubectl apply -f sla-online.yaml -n $NAMESPACE

Note: Starting with Dynamo 1.0.0 (DGDR API version v1beta1), DGDR fields use structured spec fields (e.g., spec.workload, spec.sla, spec.hardware) instead of the nested profilingConfig.config blob used in v1alpha1.

Planner-Enabled DGDR

Set spec.features.planner to enable Planner generation in the final DGD. DGDR passes this object as PlannerConfig to the Planner service; see the Planner Guide for available fields.

1
apiVersion: nvidia.com/v1beta1
2
kind: DynamoGraphDeploymentRequest
3
metadata:
4
  name: qwen3-planner
5
spec:
6
  model: Qwen/Qwen3-0.6B
7
  backend: vllm
8
  image: "nvcr.io/nvidia/ai-dynamo/dynamo-planner:1.2.1"  # dynamo-frontend for Dynamo < 1.1.0
9
  features:
10
    planner:
11
      mode: disagg
12
      backend: vllm

spec.overrides.dgd is not required to enable Planner; use it only when the generated DGD needs additional customization.

Additional DGDR Patterns

MoE Models (SGLang)

For Mixture-of-Experts models like DeepSeek-R1, use SGLang backend:

1
apiVersion: nvidia.com/v1beta1
2
kind: DynamoGraphDeploymentRequest
3
metadata:
4
  name: sla-moe
5
spec:
6
  model: deepseek-ai/DeepSeek-R1
7
  backend: sglang
8
  image: "nvcr.io/nvidia/ai-dynamo/dynamo-planner:1.2.1"  # dynamo-frontend for Dynamo < 1.1.0

Deploy:

$
# Save the manifest above as sla-moe.yaml first.
$
kubectl apply -f sla-moe.yaml -n $NAMESPACE

Customizing the Generated DGD

Use spec.overrides.dgd to provide a partial DynamoGraphDeployment that is merged into the profiler-generated deployment. Use a v1beta1 override for new DGDRs:

1
apiVersion: nvidia.com/v1beta1
2
kind: DynamoGraphDeploymentRequest
3
metadata:
4
  name: deepseek-r1
5
spec:
6
  model: deepseek-ai/DeepSeek-R1
7
  backend: sglang
8
  image: "nvcr.io/nvidia/ai-dynamo/dynamo-planner:1.2.1"  # dynamo-frontend for Dynamo < 1.1.0
9
  overrides:
10
    dgd:
11
      apiVersion: nvidia.com/v1beta1
12
      kind: DynamoGraphDeployment
13
      spec:
14
        env:
15
          - name: CUSTOM_WORKER_ENV
16
            value: "enabled"

DGDR merges the override into the generated DGD after profiling selects a configuration. Both .status.profilingResults.selectedConfig and the DGD created when autoApply: true use nvidia.com/v1beta1.

Existing v1alpha1 overrides remain supported. Their field shape follows the v1alpha1 DGD schema:

1
spec:
2
  overrides:
3
    dgd:
4
      apiVersion: nvidia.com/v1alpha1
5
      kind: DynamoGraphDeployment
6
      spec:
7
        envs:
8
          - name: CUSTOM_WORKER_ENV
9
            value: "enabled"

The override’s API version controls its merge semantics. In particular, the v1beta1 graph-level spec.env list and container args replace their generated lists, while nested container environment variables merge by name. v1alpha1 worker arguments append for compatibility. See Generated DGD Overrides for the complete behavior and direct-profiler requirements.

Inline Configuration (Simple Use Cases)

For simple use cases without a custom DGD config, provide the configuration directly in the v1beta1 DGDR spec fields. The profiler auto-generates a basic DGD configuration:

1
spec:
2
  workload:
3
    isl: 8000
4
    osl: 200
5
6
  sla:
7
    ttft: 200.0
8
    itl: 10.0
9
10
  hardware:
11
    gpuSku: h200_sxm
12
13
  searchStrategy: rapid

Simulation with Mocker

Deploy a mocker backend that simulates GPU timing behavior without real GPUs. Useful for:

• Large-scale experiments without GPU resources
• Testing profiling behavior and infrastructure
• Validating deployment configurations

1
spec:
2
  model: <model-name>
3
  backend: trtllm  # Real backend for profiling
4
  features:
5
    mocker:
6
      enabled: true  # Deploy mocker instead of real backend
7
8
  image: "nvcr.io/nvidia/ai-dynamo/dynamo-planner:1.2.1"  # dynamo-frontend for Dynamo < 1.1.0

Profiling runs against the real backend (via GPUs or AIC). The mocker deployment then uses profiling data to simulate realistic timing.

Model Cache PVC (0.8.1+)

For large models, use a pre-populated PVC instead of downloading from HuggingFace:

See SLA-Driven Profiling for configuration details.

Advanced DGDR Patterns

Review Before Deploy (autoApply: false)

Disable auto-deployment to inspect the generated DGD:

1
spec:
2
  autoApply: false

After profiling completes:

$
# Extract and review generated DGD
$
kubectl get dgdr sla-aic -n $NAMESPACE \
>
  -o jsonpath='{.status.profilingResults.selectedConfig}' > my-dgd.yaml
$
$
# Review and modify as needed
$
vi my-dgd.yaml
$
$
# Deploy manually
$
kubectl apply -f my-dgd.yaml -n $NAMESPACE

Profiling Artifacts with PVC

Save detailed profiling artifacts (plots, logs, raw data) to a PVC:

1
spec:
2
  workload:
3
    isl: 3000
4
    osl: 150
5
6
  sla:
7
    ttft: 200
8
    itl: 20

Setup:

$
export NAMESPACE=your-namespace
$
deploy/utils/setup_benchmarking_resources.sh

Access results:

$
kubectl apply -f deploy/utils/manifests/pvc-access-pod.yaml -n $NAMESPACE
$
kubectl wait --for=condition=Ready pod/pvc-access-pod -n $NAMESPACE --timeout=60s
$
kubectl cp $NAMESPACE/pvc-access-pod:/data ./profiling-results
$
kubectl delete pod pvc-access-pod -n $NAMESPACE

Related Documentation

• DGDR Reference — DGDR field reference and lifecycle
• Profiler Guide — Profiling workflow