> For clean Markdown content of this page, append .md to this URL. For the complete documentation index, see https://docs.nvidia.com/dynamo/llms.txt. For full content including API reference and SDK examples, see https://docs.nvidia.com/dynamo/llms-full.txt.

# DeepSeek-V4-Pro-0813

Each target below is a Dynamo + vLLM deployment of DeepSeek-V4-Pro-0813 — a 1.6T-parameter MoE model with MXFP4-packed routed experts and hybrid sparse attention — with KV-aware routing, FP8 KV cache and prefix caching, running aggregated or with prefill/decode disaggregation. Every target serves the model's full 1,048,576-token context without CPU KV offload. Pick your GPU architecture and serving topology; every command on this page updates to match.

Choose your deployment target

GPU

GB200

<input type="radio" id="recipe-sku-h200" name="recipe-sku" value="h200" />

H200

Topology

Aggregated

<input type="radio" id="recipe-variant-disagg" name="recipe-variant" value="disagg" />

Disaggregated

**Checkpoint** deepseek-ai/DeepSeek-V4-Pro-0813 **Precision** MXFP4 experts, FP8 KV **GPUs** 8x GB200 (2 nodes) **Context** 1,048,576 tokens **Routing** KV-aware

**Checkpoint** deepseek-ai/DeepSeek-V4-Pro-0813 **Precision** MXFP4 experts, FP8 KV **GPUs** 8x GB200 prefill + 8x GB200 decode (1P1D) **Context** 1,048,576 tokens **Routing** KV-aware, NIXL KV transfer

**Checkpoint** deepseek-ai/DeepSeek-V4-Pro-0813 **Precision** MXFP4 experts, FP8 KV **GPUs** 8x H200, one worker **Context** 1,048,576 tokens **Routing** KV-aware

**Checkpoint** deepseek-ai/DeepSeek-V4-Pro-0813 **Precision** MXFP4 experts, FP8 KV **GPUs** 8x H200 prefill + 8x H200 decode (1P1D) **Context** 1,048,576 tokens **Routing** KV-aware, NIXL KV transfer

## Prerequisites

* A Dynamo Platform install on your Kubernetes cluster — see the [Kubernetes quickstart](/dynamo/kubernetes/getting-started/quickstart).
* A Hugging Face token with access to `deepseek-ai/DeepSeek-V4-Pro-0813`.
* A ReadWriteMany storage class. The checkpoint is \~832 GiB; the PVC in this recipe requests 1200 Gi.

Eight H200 GPUs on a single node.

Sixteen H200 GPUs across two nodes — one prefill worker and one decode worker, eight GPUs each.

Eight GB200 GPUs across two nodes, connected over NVLink (MNNVL) using a Kubernetes ComputeDomain.

Sixteen GB200 GPUs — eight for prefill and eight for decode, each spanning two nodes over NVLink (MNNVL) using Kubernetes ComputeDomains.

## Deploy

Create the namespace and Hugging Face secret:

```bash
export NAMESPACE=your-namespace
kubectl create namespace ${NAMESPACE}
kubectl create secret generic hf-token-secret \
  --from-literal=HF_TOKEN="your-token" \
  -n ${NAMESPACE}
```

Create the model cache and download the checkpoint. Edit `storageClassName` in `model-cache.yaml` to a ReadWriteMany class available on your cluster:

```bash
kubectl apply -f recipes/deepseek-v4/deepseek-v4-pro-0813/model-cache/model-cache.yaml -n ${NAMESPACE}
kubectl apply -f recipes/deepseek-v4/deepseek-v4-pro-0813/model-cache/model-download.yaml -n ${NAMESPACE}
kubectl wait --for=condition=Complete job/model-download -n ${NAMESPACE} --timeout=7200s
```

Deploy the graph:

```bash
kubectl apply -f recipes/deepseek-v4/deepseek-v4-pro-0813/vllm/agg-h200-agentic/deploy.yaml -n ${NAMESPACE}
kubectl wait --for=condition=Ready dgd/dsv4-pro-0813-agg-h200-agentic -n ${NAMESPACE} --timeout=3600s
```

```bash
kubectl apply -f recipes/deepseek-v4/deepseek-v4-pro-0813/vllm/disagg-h200-agentic/deploy.yaml -n ${NAMESPACE}
kubectl wait --for=condition=Ready dgd/dsv4-pro-0813-disagg-h200-agentic -n ${NAMESPACE} --timeout=3600s
```

```bash
kubectl apply -f recipes/deepseek-v4/deepseek-v4-pro-0813/vllm/agg-gb200-agentic/deploy.yaml -n ${NAMESPACE}
kubectl wait --for=condition=Ready dgd/dsv4-pro-0813-agg-gb200-agentic -n ${NAMESPACE} --timeout=21600s
```

```bash
kubectl apply -f recipes/deepseek-v4/deepseek-v4-pro-0813/vllm/disagg-gb200-agentic/deploy.yaml -n ${NAMESPACE}
kubectl wait --for=condition=Ready dgd/dsv4-pro-0813-disagg-gb200-agentic -n ${NAMESPACE} --timeout=21600s
```

Loading the 832 GiB checkpoint takes roughly 15–20 minutes on first start on H200.
GB200 is slower — allow several hours, which is why the GB200 manifests set
`VLLM_ENGINE_READY_TIMEOUT_S: 21600` and the waits above use a six-hour timeout.

## Smoke Test

Port-forward the frontend and send one request:

```bash
kubectl port-forward svc/dsv4-pro-0813-agg-h200-agentic-frontend 8000:8000 -n ${NAMESPACE}
```

```bash
kubectl port-forward svc/dsv4-pro-0813-disagg-h200-agentic-frontend 8000:8000 -n ${NAMESPACE}
```

```bash
kubectl port-forward svc/dsv4-pro-0813-agg-gb200-agentic-frontend 8000:8000 -n ${NAMESPACE}
```

```bash
kubectl port-forward svc/dsv4-pro-0813-disagg-gb200-agentic-frontend 8000:8000 -n ${NAMESPACE}
```

```bash
curl -s localhost:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "deepseek-ai/DeepSeek-V4-Pro-0813",
    "messages": [{"role": "user", "content": "What is 17 * 23?"}],
    "max_tokens": 2048
  }' | jq -r '.choices[0].message | .content // .reasoning_content'
```

This is a reasoning model. The chain of thought is returned in `reasoning_content` and the answer in `content`; a small `max_tokens` can leave `content` empty because the budget is consumed by reasoning.

## Benchmark

See [`perf/README.md`](https://github.com/ai-dynamo/dynamo/blob/main/recipes/deepseek-v4/deepseek-v4-pro-0813/perf/README.md) for the full workflow — staging the Mooncake trace on the PVC, running the AIPerf trace-replay Job, sweeping concurrency, and fetching artifacts.

## Expected Performance

Agentic Mooncake trace, 64K median ISL / 400 median OSL, \~90% KV reuse. SLA is user output ≥ 50 tok/s.

| Concurrency | System tok/s/GPU | User output tok/s | TTFT p50 |
| ----------- | ---------------- | ----------------- | -------- |
| 8           | 69.45            | 51.85             | 403 ms   |

| Concurrency | System tok/s/GPU | User output tok/s | TTFT p50 |
| ----------- | ---------------- | ----------------- | -------- |
| 10          | 50.58            | 53.42             | 1,237 ms |

| Concurrency | System tok/s/GPU | User output tok/s | TTFT p50 |
| ----------- | ---------------- | ----------------- | -------- |
| 4           | 21.4             | 51.8              | 322 ms   |

| Concurrency | System tok/s/GPU | User output tok/s | TTFT p50 |
| ----------- | ---------------- | ----------------- | -------- |
| 4           | 13.1             | 57.2              | 441 ms   |

## Compare All Targets

| Target                 | GPUs      | Topology   | Context   | Status          |
| ---------------------- | --------- | ---------- | --------- | --------------- |
| `agg-gb200-agentic`    | 8x GB200  | Aggregated | 1,048,576 | Trace-validated |
| `disagg-gb200-agentic` | 16x GB200 | 1P1D       | 1,048,576 | Trace-validated |
| `agg-h200-agentic`     | 8x H200   | Aggregated | 1,048,576 | Trace-validated |
| `disagg-h200-agentic`  | 16x H200  | 1P1D       | 1,048,576 | Trace-validated |

## Notes

* **One config per topology serves both workloads.** The 64K agentic workload and 1M-context requests use the same deployment; GPU KV holds roughly 3.1M tokens, about 3x a full 1M request, so no CPU KV offload is needed. At 1M context, time-to-first-token is minutes — treat it as a batch capability, not an interactive one.
* **FP8 KV cache is required, not a tuning choice.** vLLM asserts `DeepseekV4 fp8_ds_mla layout only supports fp8 kv-cache`; the routed experts ship MXFP4 in the checkpoint. Serving precision is fixed by the model.
* **Speculative decoding is not enabled** on the H200 targets. It measured a large regression on this checkpoint.
* **Different checkpoint from `DeepSeek-V4-Pro`.** See the [DeepSeek-V4-Pro recipe](/dynamo/recipes/model-recipes/deepseek-v4-pro) for that model. The two use different Hugging Face repositories and must not share a model cache.

## Source

* Recipe: [`recipes/deepseek-v4/deepseek-v4-pro-0813`](https://github.com/ai-dynamo/dynamo/tree/main/recipes/deepseek-v4/deepseek-v4-pro-0813)
* Benchmark: [`perf/`](https://github.com/ai-dynamo/dynamo/tree/main/recipes/deepseek-v4/deepseek-v4-pro-0813/perf)