> For clean Markdown content of this page, append .md to this URL. For the complete documentation index, see https://docs.nvidia.com/dynamo/llms.txt. For full content including API reference and SDK examples, see https://docs.nvidia.com/dynamo/llms-full.txt.

# SGLang Deployment Templates

Copy-paste `DynamoGraphDeployment` (`nvidia.com/v1alpha1`) manifests for the SGLang backend, grouped by
topology. Each manifest is embedded from
[`examples/backends/sglang/deploy/`](https://github.com/ai-dynamo/dynamo/tree/main/examples/backends/sglang/deploy)
— open an entry, use the copy button, then set your image tag and `hf-token-secret` before applying.

Apply any template with:

```bash
kubectl apply -f agg.yaml
```

## Aggregated

#### agg.yaml · Baseline aggregated serving

```yaml title={"agg.yaml"} maxLines={0}
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

apiVersion: nvidia.com/v1alpha1
kind: DynamoGraphDeployment
metadata:
  name: sglang-agg
spec:
  services:
    Frontend:
      componentType: frontend
      replicas: 1
      extraPodSpec:
        mainContainer:
          image: my-registry/sglang-runtime:my-tag
    decode:
      envFromSecret: hf-token-secret
      componentType: worker
      replicas: 1
      resources:
        limits:
          gpu: "1"
        requests:
          custom:
            # Increase this value for larger models
            ephemeral-storage: "2Gi"
      extraPodSpec:
        mainContainer:
          image: my-registry/sglang-runtime:my-tag
          workingDir: /workspace/examples/backends/sglang
          command:
          - python3
          - -m
          - dynamo.sglang
          args:
            - --model-path
            - Qwen/Qwen3-0.6B
            - --served-model-name
            - Qwen/Qwen3-0.6B
            - --page-size
            - "16"
            - --tp
            - "1"
            - --trust-remote-code
            - --skip-tokenizer-init


```

#### agg\_router.yaml · Aggregated with KV-aware routing

```yaml title={"agg_router.yaml"} maxLines={0}
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

apiVersion: nvidia.com/v1alpha1
kind: DynamoGraphDeployment
metadata:
  name: sglang-agg-router
spec:
  services:
    Frontend:
      componentType: frontend
      replicas: 1
      extraPodSpec:
        mainContainer:
          image: my-registry/sglang-runtime:my-tag
      envs:
        - name: DYN_ROUTER_MODE
          value: kv
    decode:
      envFromSecret: hf-token-secret
      componentType: worker
      replicas: 1
      resources:
        limits:
          gpu: "1"
        requests:
          custom:
            # Increase this value for larger models
            ephemeral-storage: "2Gi"
      extraPodSpec:
        mainContainer:
          image: my-registry/sglang-runtime:my-tag
          workingDir: /workspace/examples/backends/sglang
          command:
          - python3
          - -m
          - dynamo.sglang
          args:
            - --model-path
            - Qwen/Qwen3-0.6B
            - --served-model-name
            - Qwen/Qwen3-0.6B
            - --page-size
            - "16"
            - --tp
            - "1"
            - --trust-remote-code
            - --skip-tokenizer-init
            - --kv-events-config
            - '{"publisher":"zmq","topic":"kv-events","endpoint":"tcp://*:5557"}'

```

#### agg\_gms.yaml · Aggregated with GPU Memory Service sidecar

```yaml title={"agg_gms.yaml"} maxLines={0}
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
#
# GPU Memory Service (GMS) sidecar example.
#
# The operator injects a GMS sidecar init container that provides shared GPU
# memory access via DRA (Dynamic Resource Allocation). The sidecar runs two GMS
# processes per GPU (weights + kv_cache) and communicates with the main container
# over UDS sockets on a shared emptyDir volume.
#
# Requires Kubernetes 1.34+ with DRA v1 enabled and the NVIDIA GPU DRA driver installed.

apiVersion: nvidia.com/v1alpha1
kind: DynamoGraphDeployment
metadata:
  name: sglang-agg-gms
spec:
  services:
    Frontend:
      componentType: frontend
      replicas: 1
      extraPodSpec:
        mainContainer:
          image: my-registry/sglang-runtime:my-tag
    decode:
      envFromSecret: hf-token-secret
      componentType: worker
      replicas: 1
      resources:
        limits:
          gpu: "1"
        requests:
          custom:
            # Increase this value for larger models
            ephemeral-storage: "2Gi"
      gpuMemoryService:
        enabled: true
      extraPodSpec:
        mainContainer:
          image: my-registry/sglang-runtime:my-tag
          workingDir: /workspace/examples/backends/sglang
          command:
          - python3
          - -m
          - dynamo.sglang
          args:
            - --model-path
            - Qwen/Qwen3-0.6B
            - --served-model-name
            - Qwen/Qwen3-0.6B
            - --page-size
            - "16"
            - --tp
            - "1"
            - --trust-remote-code
            - --skip-tokenizer-init
            - --load-format
            - gms

```

#### agg\_logging.yaml · Aggregated with structured logging

```yaml title={"agg_logging.yaml"} maxLines={0}
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

apiVersion: nvidia.com/v1alpha1
kind: DynamoGraphDeployment
metadata:
  name: sglang-agg
spec:
  envs:
  - name: DYN_LOGGING_JSONL
    value: "1"
  services:
    Frontend:
      componentType: frontend
      replicas: 1
      extraPodSpec:
        mainContainer:
          image: my-registry/sglang-runtime:my-tag
    decode:
      envFromSecret: hf-token-secret
      componentType: worker
      replicas: 1
      resources:
        limits:
          gpu: "1"
      extraPodSpec:
        mainContainer:
          image: my-registry/sglang-runtime:my-tag
          workingDir: /workspace/examples/backends/sglang
          command:
          - python3
          - -m
          - dynamo.sglang
          args:
            - --model-path
            - Qwen/Qwen3-0.6B
            - --served-model-name
            - Qwen/Qwen3-0.6B
            - --page-size
            - "16"
            - --tp
            - "1"
            - --trust-remote-code
            - --skip-tokenizer-init
```

## Disaggregated

#### disagg.yaml · Baseline disaggregated prefill/decode

```yaml title={"disagg.yaml"} maxLines={0}
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

apiVersion: nvidia.com/v1alpha1
kind: DynamoGraphDeployment
metadata:
  name: sglang-disagg
spec:
  services:
    Frontend:
      componentType: frontend
      replicas: 1
      extraPodSpec:
        mainContainer:
          image: my-registry/sglang-runtime:my-tag
    decode:
      envFromSecret: hf-token-secret
      componentType: worker
      subComponentType: decode
      replicas: 1
      resources:
        limits:
          gpu: "1"
      extraPodSpec:
        mainContainer:
          image: my-registry/sglang-runtime:my-tag
          workingDir: /workspace/examples/backends/sglang
          command:
          - python3
          - -m
          - dynamo.sglang
          args:
            - --model-path
            - Qwen/Qwen3-0.6B
            - --served-model-name
            - Qwen/Qwen3-0.6B
            - --page-size
            - "16"
            - --tp
            - "1"
            - --trust-remote-code
            - --skip-tokenizer-init
            - --disaggregation-mode
            - decode
            - --disaggregation-transfer-backend
            - nixl
            - --disaggregation-bootstrap-port
            - "12345"
            - --host
            - "0.0.0.0"
    prefill:
      envFromSecret: hf-token-secret
      componentType: worker
      subComponentType: prefill
      replicas: 1
      resources:
        limits:
          gpu: "1"
      extraPodSpec:
        mainContainer:
          image: my-registry/sglang-runtime:my-tag
          workingDir: /workspace/examples/backends/sglang
          command:
          - python3
          - -m
          - dynamo.sglang
          args:
            - --model-path
            - Qwen/Qwen3-0.6B
            - --served-model-name
            - Qwen/Qwen3-0.6B
            - --page-size
            - "16"
            - --tp
            - "1"
            - --trust-remote-code
            - --skip-tokenizer-init
            - --disaggregation-mode
            - prefill
            - --disaggregation-transfer-backend
            - nixl
            - --disaggregation-bootstrap-port
            - "12345"
            - --host
            - "0.0.0.0"
```

#### disagg\_planner.yaml · Disaggregated with Dynamo Planner autoscaling

```yaml title={"disagg_planner.yaml"} maxLines={0}
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

apiVersion: nvidia.com/v1alpha1
kind: DynamoGraphDeployment
metadata:
  name: sglang-disagg-planner
spec:
  services:
    Frontend:
      componentType: frontend
      replicas: 1
      extraPodSpec:
        mainContainer:
          image: my-registry/sglang-runtime:my-tag
    Planner:
      envFromSecret: hf-token-secret
      componentType: planner
      replicas: 1
      extraPodSpec:
        mainContainer:
          # Planner image selection:
          #   Dynamo >= 1.1.0: use the dedicated planner image
          #     <registry>/dynamo-planner:<version>
          #     (backend runtime images no longer ship planner runtime deps
          #     such as kubernetes_asyncio, pmdarima, prophet, aiconfigurator).
          #   Dynamo <  1.1.0: use the backend runtime image
          #     <registry>/sglang-runtime:<version>.
          image: my-registry/dynamo-planner:my-tag
          command:
          - python3
          - -m
          - dynamo.planner
          args:
            - --config
            - '{"environment": "kubernetes", "backend": "sglang", "optimization_target": "sla", "enable_throughput_scaling": true, "enable_load_scaling": true, "pre_deployment_sweeping_mode": "none", "throughput_adjustment_interval_seconds": 60, "load_adjustment_interval_seconds": 5}'
    decode:
      envFromSecret: hf-token-secret
      componentType: worker
      subComponentType: decode
      replicas: 2
      resources:
        limits:
          gpu: "1"
      extraPodSpec:
        mainContainer:
          image: my-registry/sglang-runtime:my-tag
          workingDir: /workspace/examples/backends/sglang
          command:
            - python3
          args:
            - -m
            - dynamo.sglang
            - --model-path
            - Qwen/Qwen3-0.6B
            - --served-model-name
            - Qwen/Qwen3-0.6B
            - --page-size
            - "16"
            - --tp
            - "1"
            - --trust-remote-code
            - --skip-tokenizer-init
            - --disaggregation-mode
            - decode
            - --disaggregation-transfer-backend
            - nixl
            - --disaggregation-bootstrap-port
            - "12345"
            - --host
            - "0.0.0.0"
    prefill:
      envFromSecret: hf-token-secret
      componentType: worker
      subComponentType: prefill
      replicas: 2
      resources:
        limits:
          gpu: "1"
      extraPodSpec:
        mainContainer:
          image: my-registry/sglang-runtime:my-tag
          workingDir: /workspace/examples/backends/sglang
          command:
            - python3
          args:
            - -m
            - dynamo.sglang
            - --model-path
            - Qwen/Qwen3-0.6B
            - --served-model-name
            - Qwen/Qwen3-0.6B
            - --page-size
            - "16"
            - --tp
            - "1"
            - --trust-remote-code
            - --skip-tokenizer-init
            - --disaggregation-mode
            - prefill
            - --disaggregation-transfer-backend
            - nixl
            - --disaggregation-bootstrap-port
            - "12345"
            - --host
            - "0.0.0.0"

```

#### disagg-multinode.yaml · Disaggregated across multiple nodes

```yaml title={"disagg-multinode.yaml"} maxLines={0}
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

apiVersion: nvidia.com/v1alpha1
kind: DynamoGraphDeployment
metadata:
  name: sglang-disagg-multinode
spec:
  envs:
  - name: HF_TOKEN
    valueFrom:
      secretKeyRef:
        name: hf-token-secret
        key: HF_TOKEN
  - name: GLOO_SOCKET_IFNAME
    value: "eth0"
  backendFramework: sglang
  services:
    Frontend:
      componentType: frontend
      replicas: 1
      extraPodSpec:
        mainContainer:
          image: my-registry/sglang-runtime:my-tag
    decode:
      multinode:
        nodeCount: 2
      envFromSecret: hf-token-secret
      componentType: worker
      replicas: 1
      resources:
        limits:
          gpu: "4"
      extraPodSpec:
        mainContainer:
          image: my-registry/sglang-runtime:my-tag
          workingDir: /workspace/examples/backends/sglang
          command:
          - python3
          - -m
          - dynamo.sglang
          args:
            - --model-path
            - Qwen/Qwen3-0.6B
            - --served-model-name
            - Qwen/Qwen3-0.6B
            - --tp-size
            - "8"
            - --trust-remote-code
            - --skip-tokenizer-init
            - --disaggregation-mode
            - decode
            - --disaggregation-transfer-backend
            - nixl
            - --disaggregation-bootstrap-port
            - "30001"
            - --host
            - "0.0.0.0"
            - --mem-fraction-static
            - "0.82"
    prefill:
      multinode:
        nodeCount: 2
      envFromSecret: hf-token-secret
      componentType: worker
      replicas: 1
      resources:
        limits:
          gpu: "4"
      extraPodSpec:
        mainContainer:
          image: my-registry/sglang-runtime:my-tag
          workingDir: /workspace/examples/backends/sglang
          command:
          - python3
          - -m
          - dynamo.sglang
          args:
            - --model-path
            - Qwen/Qwen3-0.6B
            - --served-model-name
            - Qwen/Qwen3-0.6B
            - --tp-size
            - "8"
            - --trust-remote-code
            - --skip-tokenizer-init
            - --disaggregation-mode
            - prefill
            - --disaggregation-transfer-backend
            - nixl
            - --disaggregation-bootstrap-port
            - "30001"
            - --mem-fraction-static
            - "0.82"
            - --host
            - "0.0.0.0"
```

## Source

All templates live in
[`examples/backends/sglang/deploy/`](https://github.com/ai-dynamo/dynamo/tree/main/examples/backends/sglang/deploy).
For local launch commands, see [SGLang Local Deployment Examples](/dynamo/dev/recipes/cli-templates/sg-lang).