GLM-5.2

Each target below is a Dynamo + SGLang deployment of Z.AI’s GLM-5.2 tuned for an agentic workload (64K median ISL / 400 median OSL, 90% KV cache hit rate) with KV-aware routing and EAGLE-style MTP speculative decoding. B200 serves the NVFP4 checkpoint at up to 500K context with HiCache CPU offload; H200 serves the FP8 checkpoint at up to 250K context. Both run aggregated or with prefill/decode disaggregation. Pick your GPU architecture and serving topology; every command on this page updates to match.

Prerequisites

Create the namespace and token secret:

$
export NAMESPACE=your-namespace
$
kubectl create namespace ${NAMESPACE}
$
kubectl create secret generic hf-token-secret \
>
  --from-literal=HF_TOKEN="your-token" \
>
  -n ${NAMESPACE}

Deploy

Create the shared model cache, then download the checkpoint for your target SKU. Edit model-cache/model-download.yaml and remove the hf download line for the checkpoint you are not serving:

$
# Edit storageClassName in model-cache/model-cache.yaml first.
$
kubectl apply -f recipes/glm-5.2/model-cache/model-cache.yaml -n ${NAMESPACE}
$
kubectl apply -f recipes/glm-5.2/model-cache/model-download.yaml -n ${NAMESPACE}
$
kubectl wait --for=condition=Complete job/model-download -n ${NAMESPACE} --timeout=7200s

Then deploy the target DGD:

$
kubectl apply -f recipes/glm-5.2/sglang/agg-b200-agentic/deploy.yaml -n ${NAMESPACE}

$
kubectl apply -f recipes/glm-5.2/sglang/disagg-b200-agentic/deploy.yaml -n ${NAMESPACE}

$
kubectl apply -f recipes/glm-5.2/sglang/agg-h200-agentic/deploy.yaml -n ${NAMESPACE}

$
kubectl apply -f recipes/glm-5.2/sglang/disagg-h200-agentic/deploy.yaml -n ${NAMESPACE}

Smoke Test

Send a test request to verify the deployment serves traffic. First forward the frontend port for your target:

$
kubectl port-forward svc/glm52-agg-b200-agentic-frontend 8000:8000 -n ${NAMESPACE}

$
kubectl port-forward svc/glm52-disagg-b200-agentic-frontend 8000:8000 -n ${NAMESPACE}

$
kubectl port-forward svc/glm52-agg-h200-agentic-frontend 8000:8000 -n ${NAMESPACE}

$
kubectl port-forward svc/glm52-disagg-h200-agentic-frontend 8000:8000 -n ${NAMESPACE}

All four targets serve under the same model name, zai-org/GLM-5.2:

$
curl http://localhost:8000/v1/chat/completions \
>
  -H 'Content-Type: application/json' \
>
  -d '{"model":"zai-org/GLM-5.2","messages":[{"role":"user","content":"Write a one-sentence readiness check."}],"max_tokens":64}'

GLM-5.2 reasons before answering. To get the answer in the content field instead of reasoning_content, disable thinking:

$
curl http://localhost:8000/v1/chat/completions \
>
  -H 'Content-Type: application/json' \
>
  -d '{"model":"zai-org/GLM-5.2","messages":[{"role":"user","content":"What is 17 times 24?"}],"chat_template_kwargs":{"enable_thinking":false},"max_tokens":64}'

Benchmark

A single AIPerf trace-replay Job — perf/perf.yaml — covers all four DGDs. It replays a Mooncake-format agentic trace (64K ISL / 400 OSL, 90% KV cache hit rate) at one concurrency value and writes artifacts to the shared model-cache PVC. The benchmark pod is co-located with a DGD frontend through podAffinity.

Edit the env block in perf/perf.yaml to target your deployed DGD — set ENDPOINT to the matching frontend service, SYNTHESIS_MAX_ISL to its context limit, and CONCURRENCY to the value for that target. The CONCURRENCY values below reproduce the Expected Performance numbers:

Then run the Job:

$
kubectl apply -f recipes/glm-5.2/perf/perf.yaml -n ${NAMESPACE}
$
kubectl wait --for=condition=Complete job/glm52-bench -n ${NAMESPACE} --timeout=7200s

For trace staging, concurrency sweeps, and fetching artifacts, see the benchmark README.

Expected Performance

Measured on the 15% agentic trace subset. System throughput is per-GPU output tokens per second; user throughput is the P50 per-request output rate.

Compare All Targets

All four targets serve GLM-5.2 on SGLang with KV-aware routing and EAGLE-style MTP speculative decoding (draft length 3, SpeedBench acceptance length 2.69), benchmarked on the same agentic trace:

Notes

• Speculative decoding uses EAGLE-style MTP with draft length 3, measured at a SpeedBench acceptance length of 2.69.
• All four targets use KV-aware routing at the frontend. The B200 targets add HiCache CPU offload; the agentic traces are shaped to show the value of KV-aware routing and offloading.
• The disaggregated targets transfer KV over NIXL/UCX on InfiniBand.

Limitations

• B200 targets support up to 500K context; the full 1M context length is not supported out of the box. H200 targets support up to 250K context.
• Structured decoding requires reasoning to be disabled ("chat_template_kwargs": {"enable_thinking": false}) so the output lands in the content field instead of reasoning_content.
• n>1 requests are not supported with the disaggregated targets.

Source

• Source README: recipes/glm-5.2/README.md
• Benchmark README: recipes/glm-5.2/perf/README.md
• Aggregated B200: deploy.yaml
• Disaggregated B200: deploy.yaml
• Aggregated H200: deploy.yaml
• Disaggregated H200: deploy.yaml
• Setup assets: model-cache.yaml and model-download.yaml
• Benchmark manifest: perf.yaml