> For clean Markdown content of this page, append .md to this URL. For the complete documentation index, see https://docs.nvidia.com/dynamo/llms.txt. For full content including API reference and SDK examples, see https://docs.nvidia.com/dynamo/llms-full.txt.

# SGLang Deployment Templates

Copy-paste `DynamoGraphDeployment` (`nvidia.com/v1beta1`) manifests for the SGLang backend, grouped by
topology. Each manifest is embedded from
[`examples/backends/sglang/deploy/`](https://github.com/ai-dynamo/dynamo/tree/v1.4.0/examples/backends/sglang/deploy)
— open an entry, use the copy button, then set your image tag and `hf-token-secret` before applying.

Apply any template with:

```bash
kubectl apply -f agg.yaml
```

## Aggregated

#### agg.yaml · Baseline aggregated serving

```yaml title={"agg.yaml"} maxLines={0}
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

apiVersion: nvidia.com/v1beta1
kind: DynamoGraphDeployment
metadata:
  name: sglang-agg
spec:
  components:
  - name: Frontend
    podTemplate:
      spec:
        containers:
        - image: my-registry/sglang-runtime:my-tag
          name: main
    replicas: 1
    type: frontend
  - name: decode
    podTemplate:
      spec:
        containers:
        - args:
          - --model-path
          - Qwen/Qwen3-0.6B
          - --served-model-name
          - Qwen/Qwen3-0.6B
          - --page-size
          - "16"
          - --tp
          - "1"
          - --trust-remote-code
          - --skip-tokenizer-init
          command:
          - python3
          - -m
          - dynamo.sglang
          envFrom:
          - secretRef:
              name: hf-token-secret
          image: my-registry/sglang-runtime:my-tag
          name: main
          resources:
            limits:
              nvidia.com/gpu: "1"
            requests:
              # Increase this value for larger models.
              ephemeral-storage: 2Gi
          workingDir: /workspace/examples/backends/sglang
    replicas: 1
    type: worker

```

#### agg\_router.yaml · Aggregated with KV-aware routing

```yaml title={"agg_router.yaml"} maxLines={0}
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

apiVersion: nvidia.com/v1beta1
kind: DynamoGraphDeployment
metadata:
  name: sglang-agg-router
spec:
  components:
  - name: Frontend
    podTemplate:
      spec:
        containers:
        - env:
          - name: DYN_ROUTER_MODE
            value: kv
          image: my-registry/sglang-runtime:my-tag
          name: main
    replicas: 1
    type: frontend
  - name: decode
    podTemplate:
      spec:
        containers:
        - args:
          - --model-path
          - Qwen/Qwen3-0.6B
          - --served-model-name
          - Qwen/Qwen3-0.6B
          - --page-size
          - "16"
          - --tp
          - "1"
          - --trust-remote-code
          - --skip-tokenizer-init
          - --kv-events-config
          - '{"publisher":"zmq","topic":"kv-events","endpoint":"tcp://*:5557"}'
          command:
          - python3
          - -m
          - dynamo.sglang
          envFrom:
          - secretRef:
              name: hf-token-secret
          image: my-registry/sglang-runtime:my-tag
          name: main
          resources:
            limits:
              nvidia.com/gpu: "1"
            requests:
              # Increase this value for larger models.
              ephemeral-storage: 2Gi
          workingDir: /workspace/examples/backends/sglang
    replicas: 1
    type: worker

```

#### agg\_gms.yaml · Aggregated with GPU Memory Service sidecar

```yaml title={"agg_gms.yaml"} maxLines={0}
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
#
# GPU Memory Service (GMS) sidecar example.
#
# The operator injects a GMS sidecar init container that provides shared GPU
# memory access via DRA (Dynamic Resource Allocation). The sidecar runs two GMS
# processes per GPU (weights + kv_cache) and communicates with the main container
# over UDS sockets on a shared emptyDir volume.
#
# Requires Kubernetes 1.34+ with DRA v1 enabled and the NVIDIA GPU DRA driver installed.

apiVersion: nvidia.com/v1beta1
kind: DynamoGraphDeployment
metadata:
  name: sglang-agg-gms
spec:
  components:
  - name: Frontend
    podTemplate:
      spec:
        containers:
        - image: my-registry/sglang-runtime:my-tag
          name: main
    replicas: 1
    type: frontend
  - experimental:
      gpuMemoryService: {}
    name: decode
    podTemplate:
      spec:
        containers:
        - args:
          - --model-path
          - Qwen/Qwen3-0.6B
          - --served-model-name
          - Qwen/Qwen3-0.6B
          - --page-size
          - "16"
          - --tp
          - "1"
          - --trust-remote-code
          - --skip-tokenizer-init
          - --load-format
          - gms
          command:
          - python3
          - -m
          - dynamo.sglang
          envFrom:
          - secretRef:
              name: hf-token-secret
          image: my-registry/sglang-runtime:my-tag
          name: main
          resources:
            limits:
              nvidia.com/gpu: "1"
            requests:
              # Increase this value for larger models.
              ephemeral-storage: 2Gi
          workingDir: /workspace/examples/backends/sglang
    replicas: 1
    type: worker

```

#### agg\_logging.yaml · Aggregated with structured logging

```yaml title={"agg_logging.yaml"} maxLines={0}
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

apiVersion: nvidia.com/v1beta1
kind: DynamoGraphDeployment
metadata:
  name: sglang-agg-logging
spec:
  components:
  - name: Frontend
    podTemplate:
      spec:
        containers:
        - image: my-registry/sglang-runtime:my-tag
          name: main
    replicas: 1
    type: frontend
  - name: decode
    podTemplate:
      spec:
        containers:
        - args:
          - --model-path
          - Qwen/Qwen3-0.6B
          - --served-model-name
          - Qwen/Qwen3-0.6B
          - --page-size
          - "16"
          - --tp
          - "1"
          - --trust-remote-code
          - --skip-tokenizer-init
          command:
          - python3
          - -m
          - dynamo.sglang
          envFrom:
          - secretRef:
              name: hf-token-secret
          image: my-registry/sglang-runtime:my-tag
          name: main
          resources:
            limits:
              nvidia.com/gpu: "1"
          workingDir: /workspace/examples/backends/sglang
    replicas: 1
    type: worker
  env:
  - name: DYN_LOGGING_JSONL
    value: "1"

```

## Disaggregated

#### disagg.yaml · Baseline disaggregated prefill/decode

```yaml title={"disagg.yaml"} maxLines={0}
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

apiVersion: nvidia.com/v1beta1
kind: DynamoGraphDeployment
metadata:
  name: sglang-disagg
spec:
  components:
  - name: Frontend
    podTemplate:
      spec:
        containers:
        - image: my-registry/sglang-runtime:my-tag
          name: main
    replicas: 1
    type: frontend
  - name: decode
    podTemplate:
      spec:
        containers:
        - args:
          - --model-path
          - Qwen/Qwen3-0.6B
          - --served-model-name
          - Qwen/Qwen3-0.6B
          - --page-size
          - "16"
          - --tp
          - "1"
          - --trust-remote-code
          - --skip-tokenizer-init
          - --disaggregation-mode
          - decode
          - --disaggregation-transfer-backend
          - nixl
          - --disaggregation-bootstrap-port
          - "12345"
          - --host
          - 0.0.0.0
          command:
          - python3
          - -m
          - dynamo.sglang
          envFrom:
          - secretRef:
              name: hf-token-secret
          image: my-registry/sglang-runtime:my-tag
          name: main
          resources:
            limits:
              nvidia.com/gpu: "1"
          workingDir: /workspace/examples/backends/sglang
    replicas: 1
    type: decode
  - name: prefill
    podTemplate:
      spec:
        containers:
        - args:
          - --model-path
          - Qwen/Qwen3-0.6B
          - --served-model-name
          - Qwen/Qwen3-0.6B
          - --page-size
          - "16"
          - --tp
          - "1"
          - --trust-remote-code
          - --skip-tokenizer-init
          - --disaggregation-mode
          - prefill
          - --disaggregation-transfer-backend
          - nixl
          - --disaggregation-bootstrap-port
          - "12345"
          - --host
          - 0.0.0.0
          command:
          - python3
          - -m
          - dynamo.sglang
          envFrom:
          - secretRef:
              name: hf-token-secret
          image: my-registry/sglang-runtime:my-tag
          name: main
          resources:
            limits:
              nvidia.com/gpu: "1"
          workingDir: /workspace/examples/backends/sglang
    replicas: 1
    type: prefill

```

#### disagg\_planner.yaml · Disaggregated with Dynamo Planner autoscaling

```yaml title={"disagg_planner.yaml"} maxLines={0}
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

apiVersion: nvidia.com/v1beta1
kind: DynamoGraphDeployment
metadata:
  name: sglang-disagg-planner
spec:
  components:
  - name: Frontend
    podTemplate:
      spec:
        containers:
        - image: my-registry/sglang-runtime:my-tag
          name: main
    replicas: 1
    type: frontend
  - name: Planner
    podTemplate:
      spec:
        containers:
        - args:
          - --config
          - '{"environment": "kubernetes", "backend": "sglang", "optimization_target": "sla",
            "enable_throughput_scaling": true, "enable_load_scaling": true, "pre_deployment_sweeping_mode":
            "none", "throughput_adjustment_interval_seconds": 60, "load_adjustment_interval_seconds": 5}'
          command:
          - python3
          - -m
          - dynamo.planner
          envFrom:
          - secretRef:
              name: hf-token-secret
          # Planner image selection:
          #   Dynamo >= 1.1.0: use the dedicated planner image
          #     <registry>/dynamo-planner:<version>
          #     (backend runtime images no longer ship planner runtime deps
          #     such as kubernetes_asyncio, pmdarima, prophet, aiconfigurator).
          #   Dynamo <  1.1.0: use the backend runtime image
          #     <registry>/sglang-runtime:<version>.
          image: my-registry/dynamo-planner:my-tag
          name: main
    replicas: 1
    type: planner
  - name: decode
    podTemplate:
      spec:
        containers:
        - args:
          - --model-path
          - Qwen/Qwen3-0.6B
          - --served-model-name
          - Qwen/Qwen3-0.6B
          - --page-size
          - "16"
          - --tp
          - "1"
          - --trust-remote-code
          - --skip-tokenizer-init
          - --disaggregation-mode
          - decode
          - --disaggregation-transfer-backend
          - nixl
          - --disaggregation-bootstrap-port
          - "12345"
          - --host
          - 0.0.0.0
          command:
          - python3
          - -m
          - dynamo.sglang
          envFrom:
          - secretRef:
              name: hf-token-secret
          image: my-registry/sglang-runtime:my-tag
          name: main
          resources:
            limits:
              nvidia.com/gpu: "1"
          workingDir: /workspace/examples/backends/sglang
    replicas: 2
    type: decode
  - name: prefill
    podTemplate:
      spec:
        containers:
        - args:
          - --model-path
          - Qwen/Qwen3-0.6B
          - --served-model-name
          - Qwen/Qwen3-0.6B
          - --page-size
          - "16"
          - --tp
          - "1"
          - --trust-remote-code
          - --skip-tokenizer-init
          - --disaggregation-mode
          - prefill
          - --disaggregation-transfer-backend
          - nixl
          - --disaggregation-bootstrap-port
          - "12345"
          - --host
          - 0.0.0.0
          command:
          - python3
          - -m
          - dynamo.sglang
          envFrom:
          - secretRef:
              name: hf-token-secret
          image: my-registry/sglang-runtime:my-tag
          name: main
          resources:
            limits:
              nvidia.com/gpu: "1"
          workingDir: /workspace/examples/backends/sglang
    replicas: 2
    type: prefill

```

#### disagg-multinode.yaml · Disaggregated across multiple nodes

```yaml title={"disagg-multinode.yaml"} maxLines={0}
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

apiVersion: nvidia.com/v1beta1
kind: DynamoGraphDeployment
metadata:
  name: sglang-disagg-multinode
spec:
  backendFramework: sglang
  components:
  - name: Frontend
    podTemplate:
      spec:
        containers:
        - image: my-registry/sglang-runtime:my-tag
          name: main
    replicas: 1
    type: frontend
  - multinode:
      nodeCount: 2
    name: decode
    podTemplate:
      spec:
        containers:
        - args:
          - --model-path
          - Qwen/Qwen3-0.6B
          - --served-model-name
          - Qwen/Qwen3-0.6B
          - --tp-size
          - "8"
          - --trust-remote-code
          - --skip-tokenizer-init
          - --disaggregation-mode
          - decode
          - --disaggregation-transfer-backend
          - nixl
          - --disaggregation-bootstrap-port
          - "30001"
          - --host
          - 0.0.0.0
          - --mem-fraction-static
          - "0.82"
          command:
          - python3
          - -m
          - dynamo.sglang
          envFrom:
          - secretRef:
              name: hf-token-secret
          image: my-registry/sglang-runtime:my-tag
          name: main
          resources:
            limits:
              nvidia.com/gpu: "4"
          workingDir: /workspace/examples/backends/sglang
    replicas: 1
    type: decode
  - multinode:
      nodeCount: 2
    name: prefill
    podTemplate:
      spec:
        containers:
        - args:
          - --model-path
          - Qwen/Qwen3-0.6B
          - --served-model-name
          - Qwen/Qwen3-0.6B
          - --tp-size
          - "8"
          - --trust-remote-code
          - --skip-tokenizer-init
          - --disaggregation-mode
          - prefill
          - --disaggregation-transfer-backend
          - nixl
          - --disaggregation-bootstrap-port
          - "30001"
          - --mem-fraction-static
          - "0.82"
          - --host
          - 0.0.0.0
          command:
          - python3
          - -m
          - dynamo.sglang
          envFrom:
          - secretRef:
              name: hf-token-secret
          image: my-registry/sglang-runtime:my-tag
          name: main
          resources:
            limits:
              nvidia.com/gpu: "4"
          workingDir: /workspace/examples/backends/sglang
    replicas: 1
    type: prefill
  env:
  - name: HF_TOKEN
    valueFrom:
      secretKeyRef:
        key: HF_TOKEN
        name: hf-token-secret
  - name: GLOO_SOCKET_IFNAME
    value: eth0

```

## Source

All templates live in
[`examples/backends/sglang/deploy/`](https://github.com/ai-dynamo/dynamo/tree/v1.4.0/examples/backends/sglang/deploy).
For local launch commands, see [SGLang Local Deployment Examples](/dynamo/recipes/cli-templates/sg-lang).