SGLang Sidecar

dynamo-sglang-sidecar is a CPU-only Dynamo worker that connects to SGLang’s native gRPC service. It preserves the upstream engine process and argument surface while using Dynamo for request handling and distributed serving. See the Sidecar Backends page for the common architecture.

Readiness

This table covers launch topology only. The SGLang feature matrix describes the in-process backend; sidecar feature parity is still under evaluation. See the SGLang sidecar README for current protocol details.

Launch Locally

From a Dynamo source checkout, build or install Dynamo so dynamo-sglang-sidecar is on PATH. Install SGLang v0.5.16 or later, which provides the native --grpc-port server option.

Start Dynamo’s local discovery services, then run the aggregated launcher:

$
docker compose -f dev/docker-compose.yml up -d
$
./lib/sidecar/sglang/launch/agg.sh --model Qwen/Qwen3-0.6B

To run separate prefill and decode engines on two GPUs:

$
./lib/sidecar/sglang/launch/disagg.sh --model Qwen/Qwen3-0.6B

Each launcher starts the Dynamo frontend, the SGLang engine process or processes, and the matching sidecar workers. It binds SGLang’s HTTP and native gRPC endpoints to loopback.

Verify the frontend:

$
curl localhost:8000/v1/chat/completions \
>
  -H "Content-Type: application/json" \
>
  -d '{
>
    "model": "Qwen/Qwen3-0.6B",
>
    "messages": [{"role": "user", "content": "Hello"}],
>
    "max_tokens": 32
>
  }'

Deploy on Kubernetes

No published SGLang sidecar image is available yet. Follow the Kubernetes quick start to build the CPU-only sidecar image and pair it with a stock upstream SGLang image. The source tree includes aggregated and disaggregated manifests.