> For clean Markdown content of this page, append .md to this URL. For the complete documentation index, see https://docs.nvidia.com/dynamo/llms.txt. For full content including API reference and SDK examples, see https://docs.nvidia.com/dynamo/llms-full.txt.

# vLLM Deployment Templates

Copy-paste `DynamoGraphDeployment` (`nvidia.com/v1alpha1`) manifests for the vLLM backend, grouped by
topology. Each manifest is embedded from
[`examples/backends/vllm/deploy/`](https://github.com/ai-dynamo/dynamo/tree/main/examples/backends/vllm/deploy)
— open an entry, use the copy button, then set your image tag and `hf-token-secret` before applying.

Apply any template with:

```bash
kubectl apply -f agg.yaml
```

A few templates bundle a second resource (a `ResourceClaimTemplate` for Dynamic Resource Allocation).
Apply the whole file — both documents are part of the example.

## Aggregated

#### agg.yaml · Baseline aggregated serving

```yaml title={"agg.yaml"} maxLines={0}
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

apiVersion: nvidia.com/v1alpha1
kind: DynamoGraphDeployment
metadata:
  name: vllm-agg
spec:
  services:
    Frontend:
      envFromSecret: hf-token-secret
      componentType: frontend
      replicas: 1
      extraPodSpec:
        mainContainer:
          image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:my-tag
    VllmDecodeWorker:
      envFromSecret: hf-token-secret
      componentType: worker
      replicas: 1
      resources:
        limits:
          gpu: "1"
        requests:
          custom:
            # Increase this value for larger models
            ephemeral-storage: "2Gi"
      extraPodSpec:
        mainContainer:
          image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:my-tag
          workingDir: /workspace/examples/backends/vllm
          command:
            - python3
            - -m
            - dynamo.vllm
          args:
            - --model
            - Qwen/Qwen3-0.6B

```

#### agg\_router.yaml · Aggregated with KV-aware routing

```yaml title={"agg_router.yaml"} maxLines={0}
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

apiVersion: nvidia.com/v1alpha1
kind: DynamoGraphDeployment
metadata:
  name: vllm-agg-router
spec:
  services:
    Frontend:
      componentType: frontend
      replicas: 1
      extraPodSpec:
        mainContainer:
          image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:my-tag
      envs:
        - name: DYN_ROUTER_MODE
          value: kv
    VllmDecodeWorker:
      envFromSecret: hf-token-secret
      componentType: worker
      replicas: 2
      resources:
        limits:
          gpu: "1"
        requests:
          custom:
            # Increase this value for larger models
            ephemeral-storage: "2Gi"
      extraPodSpec:
        mainContainer:
          image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:my-tag
          workingDir: /workspace/examples/backends/vllm
          command:
          - python3
          - -m
          - dynamo.vllm
          args:
            - --model
            - Qwen/Qwen3-0.6B
            - --kv-events-config
            - '{"publisher":"zmq","topic":"kv-events","endpoint":"tcp://*:20080","enable_kv_cache_events":true}'

```

#### agg\_router\_kv\_approx.yaml · KV routing with approximate cache tracking (no NATS/KV events)

```yaml title={"agg_router_kv_approx.yaml"} maxLines={0}
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

## This example demonstrates KV-aware routing with the --no-kv-events flag.
## Instead of receiving KV events from workers, the router predicts cache state
## locally based on routing decisions with TTL-based expiration and pruning.
## Note: This mode does not require NATS during Dynamo platform deployment.
apiVersion: nvidia.com/v1alpha1
kind: DynamoGraphDeployment
metadata:
  name: vllm-agg-router-kv-approx
spec:
  services:
    Frontend:
      componentType: frontend
      replicas: 1
      extraPodSpec:
        mainContainer:
          image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:my-tag
          command:
            - python3
          args:
            - -m
            - dynamo.frontend
            - --router-mode
            - kv
            - --no-kv-events
      envs:
        - name: DYN_ROUTER_MODE
          value: kv
    VllmDecodeWorker:
      envFromSecret: hf-token-secret
      componentType: worker
      replicas: 2
      resources:
        limits:
          gpu: "1"
      extraPodSpec:
        mainContainer:
          image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:my-tag
          workingDir: /workspace/examples/backends/vllm
          command:
          - python3
          - -m
          - dynamo.vllm
          args:
            - --model
            - Qwen/Qwen3-0.6B
            - --kv-events-config
            - '{"enable_kv_cache_events": false}'

```

#### agg\_kvbm.yaml · Aggregated with KV Block Manager offloading

```yaml title={"agg_kvbm.yaml"} maxLines={0}
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

apiVersion: nvidia.com/v1alpha1
kind: DynamoGraphDeployment
metadata:
  name: vllm-agg-kvbm
spec:
  services:
    Frontend:
      componentType: frontend
      replicas: 1
      extraPodSpec:
        mainContainer:
          image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:my-tag
    VllmDecodeWorker:
      envFromSecret: hf-token-secret
      componentType: worker
      replicas: 1
      resources:
        requests:
          gpu: "1"
          memory: "200Gi"
        limits:
          gpu: "1"
          memory: "250Gi"
      envs:
        - name: DYN_KVBM_CPU_CACHE_GB
          value: "100"
      extraPodSpec:
        mainContainer:
          image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:my-tag
          workingDir: /workspace/examples/backends/vllm
          command:
          - python3
          - -m
          - dynamo.vllm
          args:
            - --model
            - Qwen/Qwen3-8B
            - --max-model-len
            - "32000"
            - --enforce-eager
            - --kv-transfer-config
            - '{"kv_connector":"DynamoConnector","kv_connector_module_path":"kvbm.vllm_integration.connector","kv_role":"kv_both"}'

```

#### agg\_gms.yaml · Aggregated with GPU Memory Service sidecar

```yaml title={"agg_gms.yaml"} maxLines={0}
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
#
# GPU Memory Service (GMS) sidecar example.
#
# The operator injects a GMS sidecar init container that provides shared GPU
# memory access via DRA (Dynamic Resource Allocation). The sidecar runs two GMS
# processes per GPU (weights + kv_cache) and communicates with the main container
# over UDS sockets on a shared emptyDir volume.
#
# Requires Kubernetes 1.34+ with DRA v1 enabled and the NVIDIA GPU DRA driver installed.

apiVersion: nvidia.com/v1alpha1
kind: DynamoGraphDeployment
metadata:
  name: vllm-agg-gms
spec:
  services:
    Frontend:
      envFromSecret: hf-token-secret
      componentType: frontend
      replicas: 1
      extraPodSpec:
        mainContainer:
          image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:my-tag
    VllmWorker:
      envFromSecret: hf-token-secret
      componentType: worker
      replicas: 1
      resources:
        limits:
          gpu: "1"
        requests:
          custom:
            ephemeral-storage: "2Gi"
      gpuMemoryService:
        enabled: true
      extraPodSpec:
        mainContainer:
          image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:my-tag
          workingDir: /workspace/examples/backends/vllm
          command:
            - python3
            - -m
            - dynamo.vllm
          args:
            - --model
            - Qwen/Qwen3-0.6B
            - --load-format
            - gms

```

#### agg\_failover.yaml · Active-passive GPU failover

```yaml title={"agg_failover.yaml"} maxLines={0}
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
#
# Active-passive GPU failover example.
#
# The operator clones the worker's main container into two engine containers
# (engine-0 active, engine-1 standby) sharing GPUs via DRA and the GMS sidecar.
# When the active engine fails, the standby acquires the flock and takes over.
#
# Requires:
#   - gpuMemoryService.enabled: true (GMS sidecar + DRA)
#   - nvidia.com/dynamo-kube-discovery-mode: container (per-container K8s discovery)
#   - Kubernetes 1.34+ with DRA v1 enabled and the NVIDIA GPU DRA driver installed

apiVersion: nvidia.com/v1alpha1
kind: DynamoGraphDeployment
metadata:
  name: vllm-agg-failover
  annotations:
    nvidia.com/dynamo-kube-discovery-mode: container
spec:
  services:
    Frontend:
      envFromSecret: hf-token-secret
      componentType: frontend
      replicas: 1
      extraPodSpec:
        mainContainer:
          image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:my-tag
    VllmWorker:
      envFromSecret: hf-token-secret
      componentType: worker
      replicas: 1
      resources:
        limits:
          gpu: "2"
        requests:
          custom:
            ephemeral-storage: "2Gi"
      gpuMemoryService:
        enabled: true
      failover:
        enabled: true
      extraPodSpec:
        mainContainer:
          image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:my-tag
          workingDir: /workspace/examples/backends/vllm
          command:
            - python3
            - -m
            - dynamo.vllm
          args:
            - --model
            - Qwen/Qwen3-0.6B
            - --tensor-parallel-size
            - "2"
            - --load-format
            - gms

```

#### agg\_tracing.yaml · Aggregated with OpenTelemetry tracing

```yaml title={"agg_tracing.yaml"} maxLines={0}
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

# Aggregated vLLM deployment with OpenTelemetry tracing enabled.
# Base deployment: agg.yaml
# See docs/observability/tracing.md for setup instructions.

apiVersion: nvidia.com/v1alpha1
kind: DynamoGraphDeployment
metadata:
  name: vllm-agg-tracing
spec:
  envs:
    - name: DYN_LOGGING_JSONL
      value: "true"
    - name: OTEL_EXPORT_ENABLED
      value: "true"
    - name: OTEL_EXPORTER_OTLP_TRACES_ENDPOINT
      value: "http://tempo.observability.svc.cluster.local:4317"
  services:
    Frontend:
      envFromSecret: hf-token-secret
      componentType: frontend
      replicas: 1
      extraPodSpec:
        mainContainer:
          image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:my-tag
          env:
            - name: OTEL_SERVICE_NAME
              value: "dynamo-frontend"
    VllmDecodeWorker:
      envFromSecret: hf-token-secret
      componentType: worker
      replicas: 1
      resources:
        limits:
          gpu: "1"
        requests:
          custom:
            # Increase this value for larger models
            ephemeral-storage: "2Gi"
      extraPodSpec:
        mainContainer:
          image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:my-tag
          workingDir: /workspace/examples/backends/vllm
          command:
            - python3
            - -m
            - dynamo.vllm
          args:
            - --model
            - Qwen/Qwen3-0.6B
          env:
            - name: OTEL_SERVICE_NAME
              value: "dynamo-worker-vllm"

```

#### agg\_xpu\_dra.yaml · Aggregated with Dynamic Resource Allocation (bundles a ResourceClaimTemplate)

```yaml title={"agg_xpu_dra.yaml"} maxLines={0}
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

apiVersion: resource.k8s.io/v1
kind: ResourceClaimTemplate
metadata:
  name: gpu-template
spec:
  spec:
    devices:
      requests:
        - name: gpu
          exactly:
            deviceClassName: gpu.intel.com
            count: 1
---
apiVersion: nvidia.com/v1alpha1
kind: DynamoGraphDeployment
metadata:
  name: vllm-agg-xpu-dra
spec:
  services:
    Frontend:
      envFromSecret: hf-token-secret
      componentType: frontend
      replicas: 1
      extraPodSpec:
        mainContainer:
          image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:my-tag
    VllmDecodeWorker:
      envFromSecret: hf-token-secret
      componentType: worker
      replicas: 1
      resources:
        requests:
          custom:
            # Increase this value for larger models
            ephemeral-storage: "2Gi"
      extraPodSpec:
        resourceClaims:
          - name: gpu
            resourceClaimTemplateName: gpu-template
        # NOTE: Uncomment if your environment requires specific group access
        # securityContext:
        #   runAsUser: 1000
        #   runAsGroup: 1000
        #   supplementalGroups:
        #     - 44   # render group
        #     - 991  # video group
        mainContainer:
          image: nvcr.io/nvidia/ai-dynamo/vllm-runtime-xpu:my-tag
          resources:
            claims:
              - name: gpu
          env:
            - name: VLLM_TARGET_DEVICE
              value: xpu
          workingDir: /workspace/examples/backends/vllm
          command:
            - python3
            - -m
            - dynamo.vllm
          args:
            - --model
            - Qwen/Qwen3-0.6B
            - --block-size
            - "64"

```

## Disaggregated

#### disagg.yaml · Baseline disaggregated prefill/decode

```yaml title={"disagg.yaml"} maxLines={0}
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

apiVersion: nvidia.com/v1alpha1
kind: DynamoGraphDeployment
metadata:
  name: vllm-disagg
spec:
  services:
    Frontend:
      componentType: frontend
      replicas: 1
      extraPodSpec:
        mainContainer:
          image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:my-tag
    VllmDecodeWorker:
      envFromSecret: hf-token-secret
      componentType: worker
      subComponentType: decode
      replicas: 1
      resources:
        limits:
          gpu: "1"
        requests:
          custom:
            # Increase this value for larger models
            ephemeral-storage: "2Gi"
      extraPodSpec:
        mainContainer:
          image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:my-tag
          workingDir: /workspace/examples/backends/vllm
          command:
          - python3
          - -m
          - dynamo.vllm
          args:
            - --model
            - Qwen/Qwen3-0.6B
            - --disaggregation-mode
            - decode
    VllmPrefillWorker:
      envFromSecret: hf-token-secret
      componentType: worker
      subComponentType: prefill
      replicas: 1
      resources:
        limits:
          gpu: "1"
        requests:
          custom:
            # Increase this value for larger models
            ephemeral-storage: "2Gi"
      extraPodSpec:
        mainContainer:
          image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:my-tag
          workingDir: /workspace/examples/backends/vllm
          command:
          - python3
          - -m
          - dynamo.vllm
          args:
            - --model
            - Qwen/Qwen3-0.6B
            - --disaggregation-mode
            - prefill
            - --kv-transfer-config
            - '{"kv_connector":"NixlConnector","kv_role":"kv_both"}'

```

#### disagg\_router.yaml · Disaggregated with KV-aware routing

```yaml title={"disagg_router.yaml"} maxLines={0}
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

apiVersion: nvidia.com/v1alpha1
kind: DynamoGraphDeployment
metadata:
  name: vllm-v1-disagg-router
spec:
  services:
    Frontend:
      componentType: frontend
      replicas: 1
      extraPodSpec:
        mainContainer:
          image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:my-tag
      envs:
        - name: DYN_ROUTER_MODE
          value: kv
    VllmDecodeWorker:
      envFromSecret: hf-token-secret
      componentType: worker
      replicas: 2
      resources:
        limits:
          gpu: "1"
        requests:
          custom:
            # Increase this value for larger models
            ephemeral-storage: "2Gi"
      extraPodSpec:
        mainContainer:
          image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:my-tag
          workingDir: /workspace/examples/backends/vllm
          command:
          - python3
          - -m
          - dynamo.vllm
          args:
            - --model
            - Qwen/Qwen3-0.6B
            - --disaggregation-mode
            - decode
    VllmPrefillWorker:
      envFromSecret: hf-token-secret
      componentType: worker
      replicas: 2
      resources:
        limits:
          gpu: "1"
        requests:
          custom:
            # Increase this value for larger models
            ephemeral-storage: "2Gi"
      extraPodSpec:
        mainContainer:
          image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:my-tag
          workingDir: /workspace/examples/backends/vllm
          command:
          - python3
          - -m
          - dynamo.vllm
          args:
            - --model
            - Qwen/Qwen3-0.6B
            - --disaggregation-mode
            - prefill
            - --kv-transfer-config
            - '{"kv_connector":"NixlConnector","kv_role":"kv_both"}'
            - --kv-events-config
            - '{"publisher":"zmq","topic":"kv-events","endpoint":"tcp://*:20080","enable_kv_cache_events":true}'

```

#### disagg\_planner.yaml · Disaggregated with Dynamo Planner autoscaling

```yaml title={"disagg_planner.yaml"} maxLines={0}
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

apiVersion: nvidia.com/v1alpha1
kind: DynamoGraphDeployment
metadata:
  name: vllm-disagg-planner
spec:
  services:
    Frontend:
      componentType: frontend
      replicas: 1
      extraPodSpec:
        mainContainer:
          image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:my-tag
    Planner:
      componentType: planner
      replicas: 1
      extraPodSpec:
        mainContainer:
          # Planner image selection:
          #   Dynamo >= 1.1.0: use the dedicated planner image
          #     nvcr.io/nvidia/ai-dynamo/dynamo-planner:<version>
          #     (backend runtime images no longer ship planner runtime deps
          #     such as kubernetes_asyncio, pmdarima, prophet, aiconfigurator).
          #   Dynamo <  1.1.0: use the backend runtime image
          #     nvcr.io/nvidia/ai-dynamo/vllm-runtime:<version>.
          image: nvcr.io/nvidia/ai-dynamo/dynamo-planner:my-tag
          command:
          - python3
          - -m
          - dynamo.planner
          args:
            - --config
            - '{"environment": "kubernetes", "backend": "vllm", "optimization_target": "sla", "enable_throughput_scaling": true, "enable_load_scaling": true, "pre_deployment_sweeping_mode": "none", "throughput_adjustment_interval_seconds": 60, "load_adjustment_interval_seconds": 5}'
    VllmDecodeWorker:
      envFromSecret: hf-token-secret
      componentType: worker
      subComponentType: decode
      replicas: 1
      resources:
        limits:
          gpu: "1"
      extraPodSpec:
        mainContainer:
          image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:my-tag
          workingDir: /workspace/examples/backends/vllm
          command:
            - python3
          args:
            - -m
            - dynamo.vllm
            - --model
            - Qwen/Qwen3-0.6B
    VllmPrefillWorker:
      envFromSecret: hf-token-secret
      componentType: worker
      subComponentType: prefill
      replicas: 1
      resources:
        limits:
          gpu: "1"
      extraPodSpec:
        mainContainer:
          image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:my-tag
          workingDir: /workspace/examples/backends/vllm
          command:
            - python3
          args:
            - -m
            - dynamo.vllm
            - --model
            - Qwen/Qwen3-0.6B
            - --disaggregation-mode
            - prefill
            - --kv-transfer-config
            - '{"kv_connector":"NixlConnector","kv_role":"kv_both"}'

```

#### disagg\_kvbm.yaml · Disaggregated with KV Block Manager offloading

```yaml title={"disagg_kvbm.yaml"} maxLines={0}
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

apiVersion: nvidia.com/v1alpha1
kind: DynamoGraphDeployment
metadata:
  name: vllm-disagg-kvbm
spec:
  services:
    Frontend:
      componentType: frontend
      replicas: 1
      extraPodSpec:
        mainContainer:
          image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:my-tag
    VllmDecodeWorker:
      envFromSecret: hf-token-secret
      componentType: worker
      replicas: 1
      resources:
        limits:
          gpu: "1"
      extraPodSpec:
        mainContainer:
          image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:my-tag
          workingDir: /workspace/examples/backends/vllm
          command:
          - python3
          - -m
          - dynamo.vllm
          args:
            - --model
            - Qwen/Qwen3-8B
            - --disaggregation-mode
            - decode
            - --kv-transfer-config
            - '{"kv_connector":"NixlConnector","kv_role":"kv_both"}'
            - --max-model-len
            - "32000"
            - --enforce-eager
    VllmPrefillWorker:
      envFromSecret: hf-token-secret
      componentType: worker
      replicas: 1
      resources:
        requests:
          gpu: "1"
          memory: "200Gi"
        limits:
          gpu: "1"
          memory: "250Gi"
      envs:
        - name: DYN_KVBM_CPU_CACHE_GB
          value: "100"
      extraPodSpec:
        mainContainer:
          image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:my-tag
          workingDir: /workspace/examples/backends/vllm
          command:
          - python3
          - -m
          - dynamo.vllm
          args:
            - --model
            - Qwen/Qwen3-8B
            - --disaggregation-mode
            - prefill
            - --max-model-len
            - "32000"
            - --enforce-eager
            - --kv-transfer-config
            - '{"kv_connector":"PdConnector","kv_role":"kv_both","kv_connector_extra_config":{"connectors":[{"kv_connector":"DynamoConnector","kv_connector_module_path":"kvbm.vllm_integration.connector","kv_role":"kv_both"},{"kv_connector":"NixlConnector","kv_role":"kv_both"}]},"kv_connector_module_path":"kvbm.vllm_integration.connector"}'

```

#### disagg\_kvbm\_tp2.yaml · Disaggregated KVBM, tensor-parallel 2

```yaml title={"disagg_kvbm_tp2.yaml"} maxLines={0}
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

apiVersion: nvidia.com/v1alpha1
kind: DynamoGraphDeployment
metadata:
  name: vllm-disagg-kvbm-tp2
spec:
  services:
    Frontend:
      componentType: frontend
      replicas: 1
      extraPodSpec:
        mainContainer:
          image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:my-tag
    VllmDecodeWorker:
      envFromSecret: hf-token-secret
      componentType: worker
      replicas: 1
      resources:
        requests:
          gpu: "2"
        limits:
          gpu: "2"
      extraPodSpec:
        mainContainer:
          image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:my-tag
          workingDir: /workspace/examples/backends/vllm
          command:
          - python3
          - -m
          - dynamo.vllm
          args:
            - --model
            - Qwen/Qwen3-8B
            - --disaggregation-mode
            - decode
            - --kv-transfer-config
            - '{"kv_connector":"NixlConnector","kv_role":"kv_both"}'
            - --gpu-memory-utilization
            - "0.23"
            - --max-model-len
            - "32000"
            - --enforce-eager
            - --tensor-parallel-size
            - "2"
    VllmPrefillWorker:
      envFromSecret: hf-token-secret
      componentType: worker
      replicas: 1
      resources:
        requests:
          gpu: "2"
          memory: "200Gi"
        limits:
          gpu: "2"
          memory: "250Gi"
      envs:
        - name: DYN_KVBM_CPU_CACHE_GB
          value: "100"
      extraPodSpec:
        mainContainer:
          image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:my-tag
          workingDir: /workspace/examples/backends/vllm
          command:
          - python3
          - -m
          - dynamo.vllm
          args:
            - --model
            - Qwen/Qwen3-8B
            - --disaggregation-mode
            - prefill
            - --gpu-memory-utilization
            - "0.23"
            - --max-model-len
            - "32000"
            - --enforce-eager
            - --kv-transfer-config
            - '{"kv_connector":"PdConnector","kv_role":"kv_both","kv_connector_extra_config":{"connectors":[{"kv_connector":"DynamoConnector","kv_connector_module_path":"kvbm.vllm_integration.connector","kv_role":"kv_both"},{"kv_connector":"NixlConnector","kv_role":"kv_both"}]},"kv_connector_module_path":"kvbm.vllm_integration.connector"}'
            - --tensor-parallel-size
            - "2"

```

#### disagg\_kvbm\_2p2d.yaml · Disaggregated KVBM, 2 prefill + 2 decode workers

```yaml title={"disagg_kvbm_2p2d.yaml"} maxLines={0}
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

apiVersion: nvidia.com/v1alpha1
kind: DynamoGraphDeployment
metadata:
  name: vllm-disagg-kvbm-2p2d
spec:
  services:
    Frontend:
      componentType: frontend
      replicas: 1
      extraPodSpec:
        mainContainer:
          image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:my-tag
    VllmDecodeWorker:
      envFromSecret: hf-token-secret
      componentType: worker
      replicas: 2
      resources:
        limits:
          gpu: "1"
      extraPodSpec:
        mainContainer:
          image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:my-tag
          workingDir: /workspace/examples/backends/vllm
          command:
          - python3
          - -m
          - dynamo.vllm
          args:
            - --model
            - Qwen/Qwen3-8B
            - --disaggregation-mode
            - decode
            - --kv-transfer-config
            - '{"kv_connector":"NixlConnector","kv_role":"kv_both"}'
            - --max-model-len
            - "32000"
            - --enforce-eager
    VllmPrefillWorker:
      envFromSecret: hf-token-secret
      componentType: worker
      replicas: 2
      resources:
        requests:
          gpu: "1"
          memory: "200Gi"
        limits:
          gpu: "1"
          memory: "250Gi"
      envs:
        - name: DYN_KVBM_CPU_CACHE_GB
          value: "100"
      extraPodSpec:
        mainContainer:
          image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:my-tag
          workingDir: /workspace/examples/backends/vllm
          command:
          - python3
          - -m
          - dynamo.vllm
          args:
            - --model
            - Qwen/Qwen3-8B
            - --disaggregation-mode
            - prefill
            - --max-model-len
            - "32000"
            - --enforce-eager
            - --kv-transfer-config
            - '{"kv_connector":"PdConnector","kv_role":"kv_both","kv_connector_extra_config":{"connectors":[{"kv_connector":"DynamoConnector","kv_connector_module_path":"kvbm.vllm_integration.connector","kv_role":"kv_both"},{"kv_connector":"NixlConnector","kv_role":"kv_both"}]},"kv_connector_module_path":"kvbm.vllm_integration.connector"}'

```

#### disagg\_tracing.yaml · Disaggregated with OpenTelemetry tracing

```yaml title={"disagg_tracing.yaml"} maxLines={0}
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

# Disaggregated vLLM deployment with OpenTelemetry tracing enabled.
# Base deployment: disagg.yaml
# See docs/observability/tracing.md for setup instructions.

apiVersion: nvidia.com/v1alpha1
kind: DynamoGraphDeployment
metadata:
  name: vllm-disagg-tracing
spec:
  envs:
    - name: DYN_LOGGING_JSONL
      value: "true"
    - name: OTEL_EXPORT_ENABLED
      value: "true"
    - name: OTEL_EXPORTER_OTLP_TRACES_ENDPOINT
      value: "http://tempo.observability.svc.cluster.local:4317"
  services:
    Frontend:
      componentType: frontend
      replicas: 1
      extraPodSpec:
        mainContainer:
          image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:my-tag
          env:
            - name: OTEL_SERVICE_NAME
              value: "dynamo-frontend"
    VllmDecodeWorker:
      envFromSecret: hf-token-secret
      componentType: worker
      subComponentType: decode
      replicas: 1
      resources:
        limits:
          gpu: "1"
        requests:
          custom:
            # Increase this value for larger models
            ephemeral-storage: "2Gi"
      extraPodSpec:
        mainContainer:
          image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:my-tag
          workingDir: /workspace/examples/backends/vllm
          command:
          - python3
          - -m
          - dynamo.vllm
          args:
            - --model
            - Qwen/Qwen3-0.6B
            - --disaggregation-mode
            - decode
          env:
            - name: OTEL_SERVICE_NAME
              value: "dynamo-worker-decode"
    VllmPrefillWorker:
      envFromSecret: hf-token-secret
      componentType: worker
      subComponentType: prefill
      replicas: 1
      resources:
        limits:
          gpu: "1"
        requests:
          custom:
            # Increase this value for larger models
            ephemeral-storage: "2Gi"
      extraPodSpec:
        mainContainer:
          image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:my-tag
          workingDir: /workspace/examples/backends/vllm
          command:
          - python3
          - -m
          - dynamo.vllm
          args:
            - --model
            - Qwen/Qwen3-0.6B
            - --disaggregation-mode
            - prefill
            - --kv-transfer-config
            - '{"kv_connector":"NixlConnector","kv_role":"kv_both"}'
          env:
            - name: OTEL_SERVICE_NAME
              value: "dynamo-worker-prefill"

```

#### disagg-multinode.yaml · Disaggregated across multiple nodes

```yaml title={"disagg-multinode.yaml"} maxLines={0}
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

apiVersion: nvidia.com/v1alpha1
kind: DynamoGraphDeployment
metadata:
  name: vllm-disagg
spec:
  services:
    Frontend:
      componentType: frontend
      replicas: 1
      extraPodSpec:
        mainContainer:
          image: my-registry/vllm-runtime:my-tag
          workingDir: /workspace/examples/backends/vllm
          command:
          - python3
          - -m
          - dynamo.frontend
          args:
            - --http-port
            - "8000"
    decode:
      envFromSecret: hf-token-secret
      componentType: worker
      replicas: 1
      multinode:
        nodeCount: 2
      resources:
        limits:
          gpu: "1"
      extraPodSpec:
        mainContainer:
          image: my-registry/vllm-runtime:my-tag
          workingDir: /workspace/examples/backends/vllm
          command:
          - python3
          - -m
          - dynamo.vllm
          args:
          - --model
          - Qwen/Qwen3-0.6B
          - --tensor-parallel-size
          - "2"
          - --disaggregation-mode
          - decode
          - --kv-transfer-config
          - '{"kv_connector": "NixlConnector", "kv_role": "kv_both", "engine_id":
            "vllm-disagg-decode-engine-0abc123"}'
    prefill:
      envFromSecret: hf-token-secret
      componentType: worker
      replicas: 1
      multinode:
        nodeCount: 2
      resources:
        limits:
          gpu: "1"
      extraPodSpec:
        mainContainer:
          image: my-registry/vllm-runtime:my-tag
          workingDir: /workspace/examples/backends/vllm
          command:
          - python3
          - -m
          - dynamo.vllm
          args:
          - --model
          - Qwen/Qwen3-0.6B
          - --tensor-parallel-size
          - "2"
          - --disaggregation-mode
          - prefill
          - --kv-transfer-config
          - '{"kv_connector": "NixlConnector", "kv_role": "kv_both", "engine_id":
            "vllm-disagg-prefill-engine-0abc123"}'

```

#### disagg\_xpu\_dra.yaml · Disaggregated with Dynamic Resource Allocation (bundles a ResourceClaimTemplate)

```yaml title={"disagg_xpu_dra.yaml"} maxLines={0}
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

apiVersion: resource.k8s.io/v1
kind: ResourceClaimTemplate
metadata:
  name: gpu-template
spec:
  spec:
    devices:
      requests:
        - name: gpu
          exactly:
            deviceClassName: gpu.intel.com
            count: 1
---
apiVersion: nvidia.com/v1alpha1
kind: DynamoGraphDeployment
metadata:
  name: vllm-disagg-xpu-dra
spec:
  services:
    Frontend:
      envFromSecret: hf-token-secret
      componentType: frontend
      replicas: 1
      extraPodSpec:
        mainContainer:
          image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:my-tag
    VllmDecodeWorker:
      envFromSecret: hf-token-secret
      componentType: worker
      subComponentType: decode
      replicas: 1
      resources:
        requests:
          custom:
            # Increase this value for larger models
            ephemeral-storage: "2Gi"
      extraPodSpec:
        resourceClaims:
          - name: gpu
            resourceClaimTemplateName: gpu-template
        # NOTE: Uncomment if your environment requires specific group access
        # securityContext:
        #   runAsUser: 1000
        #   runAsGroup: 1000
        #   supplementalGroups:
        #     - 44   # render group
        #     - 991  # video group
        mainContainer:
          image: nvcr.io/nvidia/ai-dynamo/vllm-runtime-xpu:my-tag
          resources:
            claims:
              - name: gpu
          env:
            - name: VLLM_TARGET_DEVICE
              value: xpu
          workingDir: /workspace/examples/backends/vllm
          command:
            - python3
            - -m
            - dynamo.vllm
          args:
            - --model
            - Qwen/Qwen3-0.6B
            - --disaggregation-mode
            - decode
            - --kv-transfer-config
            - '{"kv_connector":"NixlConnector","kv_role":"kv_consumer","kv_buffer_device":"xpu"}'
            - --block-size
            - "64"
    VllmPrefillWorker:
      envFromSecret: hf-token-secret
      componentType: worker
      subComponentType: prefill
      replicas: 1
      resources:
        requests:
          custom:
            # Increase this value for larger models
            ephemeral-storage: "2Gi"
      extraPodSpec:
        resourceClaims:
          - name: gpu
            resourceClaimTemplateName: gpu-template
        # NOTE: Uncomment if your environment requires specific group access
        # securityContext:
        #   runAsUser: 1000
        #   runAsGroup: 1000
        #   supplementalGroups:
        #     - 44   # render group
        #     - 991  # video group
        mainContainer:
          image: nvcr.io/nvidia/ai-dynamo/vllm-runtime-xpu:my-tag
          resources:
            claims:
              - name: gpu
          env:
            - name: VLLM_TARGET_DEVICE
              value: xpu
          workingDir: /workspace/examples/backends/vllm
          command:
            - python3
            - -m
            - dynamo.vllm
          args:
            - --model
            - Qwen/Qwen3-0.6B
            - --disaggregation-mode
            - prefill
            - --kv-transfer-config
            - '{"kv_connector":"NixlConnector","kv_role":"kv_both","kv_buffer_device":"xpu"}'
            - --block-size
            - "64"

```

## Other

#### gms-failover.yaml · Multinode GPU Memory Service with failover (bundles a ResourceClaimTemplate)

```yaml title={"gms-failover.yaml"} maxLines={0}
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

# Example: DynamoGraphDeployment with inter-pod GMS (GPU Memory Service)
# failover on vLLM.
#
# Inter-pod GMS failover splits the traditional single-engine pod into:
#   * a dedicated GMS weight-server pod (per rank) that owns the model weights
#     and exposes them over a shared-GPU UDS, and
#   * N engine pods (per rank) that attach to the same GPUs via DRA and race
#     for a flock; the winner becomes primary, the others are hot shadows.
#
# This file contains two commented-out variants you can copy into .spec.services:
#
# Single-node GMS:
#   Creates per PCSG replica:
#     - 1 GMS weight-server pod   (<service>-gms-0)
#     - numShadows + 1 engine pods (<service>, replicas = numShadows + 1)
#   All engine pods + the GMS pod share the same GPUs via DRA ResourceClaims.
#   service.replicas controls how many PCSG replicas are created
#   (horizontal scale).
#
# Multinode GMS (N nodes):
#   Creates per PCSG replica:
#     - 1 GMS weight-server pod per rank   (<service>-gms-<rank>)
#     - numShadows + 1 engine pods per rank
#         rank 0: <service>-ldr   (leader,   replicas = numShadows + 1)
#         rank R: <service>-wkr-R (worker R, replicas = numShadows + 1)
#   Each rank's GMS + engine pods share GPUs via DRA within that node.
#   service.replicas controls horizontal PCSG replicas.
apiVersion: nvidia.com/v1alpha1
kind: DynamoGraphDeployment
metadata:
  name: llm-serving-mn
spec:
  backendFramework: vllm
  services:
    # ─── Single-node GMS failover ───
    agg:
      componentType: worker
      replicas: 1
      resources:
        limits:
          gpu: "1"
      gpuMemoryService:
        enabled: true
        mode: interPod
      failover:
        enabled: true
        mode: interPod
        numShadows: 1        # 1 primary + 1 shadow = 2 engine pods per PCSG replica
      extraPodSpec:
        mainContainer:
          image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:latest
          command: ["python3", "-m", "dynamo.vllm"]
          args: ["--model", "Qwen/Qwen3-0.6B", "--tensor-parallel-size", "1", "--enforce-eager", "--gpu-memory-utilization", "0.85"]
    #   sharedMemory:
    #     size: 16Gi

    # ─── Multinode GMS failover (2 nodes) ───
    # agg:
    #   envFromSecret: hf-token-secret
    #   componentType: worker
    #   replicas: 1
    #   multinode:
    #     nodeCount: 2
    #   resources:
    #     limits:
    #       gpu: "1"
    #   gpuMemoryService:
    #     enabled: true
    #     mode: interPod
    #   failover:
    #     enabled: true
    #     mode: interPod
    #     numShadows: 1       # 1 primary + 1 shadow = 2 engine pods per rank
    #   extraPodSpec:
    #     mainContainer:
    #       image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:latest
    #       command: ["python3", "-m", "dynamo.vllm"]
    #       # args: ["--model", "Qwen/Qwen3-235B-A22B", "--tensor-parallel-size", "8", "--enforce-eager", "--gpu-memory-utilization", "0.85"]
    #       args: ["--model", "Qwen/Qwen3-0.6B", "--tensor-parallel-size", "2", "--enforce-eager", "--gpu-memory-utilization", "0.85"]

      # sharedMemory:
      #   size: 16Gi

    # ─── Regular frontend (no failover) ───
    frontend:
      envFromSecret: hf-token-secret
      componentType: frontend
      replicas: 1
      extraPodSpec:
        mainContainer:
          image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:latest
          # command: ["python3", "-m", "dynamo.frontend"]

```

## Source

All templates live in
[`examples/backends/vllm/deploy/`](https://github.com/ai-dynamo/dynamo/tree/main/examples/backends/vllm/deploy).
For local launch commands, see [vLLM Local Deployment Examples](/dynamo/dev/recipes/cli-templates/v-llm).