> For clean Markdown content of this page, append .md to this URL. For the complete documentation index, see https://docs.nvidia.com/dynamo/latest/llms.txt. For section-specific indexes, append /llms.txt to any section URL.

# DeepSeek-V4-Flash

Each target below is an agentic-workload vLLM deployment of DeepSeek-V4-Flash — a MoE model (284B total / 13B active) with hybrid CSA + HCA attention, FP8 KV cache, and KV-aware routing. B200 serves the `nvidia/DeepSeek-V4-Flash-NVFP4` checkpoint; H200 serves the public `deepseek-ai/DeepSeek-V4-Flash`. Pick your GPU and topology; every command on this page updates to match. Text only; reasoning + tool calling supported.

<p>
  Choose your deployment target
</p>

GPU

B200 Recommended

<input type="radio" id="recipe-sku-h200" name="recipe-sku" value="h200" />

H200

Topology

Disaggregated Higher throughput

<input type="radio" id="recipe-variant-agg" name="recipe-variant" value="agg" />

Aggregated

<b>Checkpoint</b> nvidia/DeepSeek-V4-Flash-NVFP4

<b>GPUs</b> 4x B200, TP4 + EP

<b>MoE / KV</b> FLASHINFER\_TRTLLM, FP8 KV (block 256)

<b>Routing</b> KV-aware, prefix caching

<b>Context</b> 1,048,576

<b>Checkpoint</b> nvidia/DeepSeek-V4-Flash-NVFP4

<b>GPUs</b> 12x B200 — 2P1D (2 prefill + 1 decode, TP4 each)

<b>MoE / KV</b> FLASHINFER\_TRTLLM, FP8 KV (block 256)

<b>Transfer</b> NIXL over RDMA/GDR

<b>Context</b> 1,048,576

<b>Checkpoint</b> deepseek-ai/DeepSeek-V4-Flash

<b>GPUs</b> 4x H200, DP4 + TP1 + EP

<b>MoE / spec</b> MARLIN (FLASHINFER\_MLA attn), MTP-1

<b>Routing</b> KV-aware, prefix caching

<b>Context</b> 1,048,576

<b>Checkpoint</b> deepseek-ai/DeepSeek-V4-Flash

<b>GPUs</b> 28x H200 — 4P3D (DP4+TP1+EP per worker)

<b>MoE / spec</b> MARLIN, decode MTP-1

<b>Transfer</b> NIXL over RDMA/GDR

<b>Context</b> 1,048,576

## Prerequisites

* A Kubernetes cluster with the Dynamo Platform (operator) installed.

- **4x B200** (aggregated) or **12x B200** (disaggregated, 2P1D) available.
- A Hugging Face token with access to `nvidia/DeepSeek-V4-Flash-NVFP4`.

* **4x H200** (aggregated) or **28x H200** (disaggregated, 4P3D) available.
* A Hugging Face token with access to `deepseek-ai/DeepSeek-V4-Flash`.

- The `nvcr.io/nvidia/ai-dynamo/vllm-runtime:1.3.0` image (pinned in each `deploy.yaml`).

Create the namespace and token secret:

```bash
export NAMESPACE=your-namespace
kubectl create namespace ${NAMESPACE}
kubectl create secret generic hf-token-secret --from-literal=HF_TOKEN="your-token" -n ${NAMESPACE}
```

Edit cluster-specific values before applying: `storageClassName` in `model-cache/model-cache.yaml` (a RWX class). For **disaggregated** targets, also set `VLLM_GPU_NIC_PCIE_MAPPING` in the manifest to your node's per-rank affine NICs — see [Per-rank NIC mapping](https://github.com/ai-dynamo/dynamo/blob/main/recipes/deepseek-v4/README.md#per-rank-nic-mapping-b200--h200-disaggregated).

## Deploy

Create storage, download the checkpoint into the PVC, then apply the target's `deploy.yaml`:

```bash
kubectl apply -f recipes/deepseek-v4/deepseek-v4-flash/model-cache/model-cache.yaml -n ${NAMESPACE}
kubectl apply -f recipes/deepseek-v4/deepseek-v4-flash/model-cache/model-download.yaml -n ${NAMESPACE}
kubectl wait --for=condition=Complete job/model-download -n ${NAMESPACE} --timeout=7200s
```

```bash
RECIPE=recipes/deepseek-v4/deepseek-v4-flash/vllm/agg-b200-agentic
```

```bash
RECIPE=recipes/deepseek-v4/deepseek-v4-flash/vllm/disagg-b200-agentic
```

```bash
RECIPE=recipes/deepseek-v4/deepseek-v4-flash/vllm/agg-h200-agentic
```

```bash
RECIPE=recipes/deepseek-v4/deepseek-v4-flash/vllm/disagg-h200-agentic
```

```bash
kubectl apply -f ${RECIPE}/deploy.yaml -n ${NAMESPACE}

# DGD name = metadata.name in that deploy.yaml:
DGD=$(awk '/^metadata:/{m=1} m && /name:/{print $2; exit}' ${RECIPE}/deploy.yaml)
kubectl wait --for=condition=Ready pod \
  -l nvidia.com/dynamo-graph-deployment-name=${DGD} -n ${NAMESPACE} --timeout=3600s
```

First launch loads weights and warms CUDA graphs; the startup probes allow for it.

## Smoke Test

The `model` field must be the **served model name for your SKU** (the `--served-model-name` in the deploy): B200 serves `nvidia/DeepSeek-V4-Flash-NVFP4`, H200 serves `deepseek-ai/DeepSeek-V4-Flash`.

```bash
kubectl port-forward svc/${DGD}-frontend 8000:8000 -n ${NAMESPACE}

curl http://localhost:8000/v1/chat/completions -H 'Content-Type: application/json' \
  -d '{"model":"nvidia/DeepSeek-V4-Flash-NVFP4","messages":[{"role":"user","content":"Hello!"}],"max_tokens":512}'
```

```bash
kubectl port-forward svc/${DGD}-frontend 8000:8000 -n ${NAMESPACE}

curl http://localhost:8000/v1/chat/completions -H 'Content-Type: application/json' \
  -d '{"model":"deepseek-ai/DeepSeek-V4-Flash","messages":[{"role":"user","content":"Hello!"}],"max_tokens":512}'
```

Flash reasons by default: the chain-of-thought fills `message.reasoning_content` and the answer fills `message.content`. With too small a `max_tokens` the budget is spent on reasoning before any `content` (`content: null`, `finish_reason: "length"`) — expected; raise `max_tokens`.

## Benchmark

The shared [`perf.yaml`](https://github.com/ai-dynamo/dynamo/blob/main/recipes/deepseek-v4/perf/perf.yaml) replays the agentic MoonTrace (64K avg ISL / 400 OSL, 90% KV reuse) with AIPerf against the deployed frontend. Point `ENDPOINT` at `${DGD}-frontend:8000` and apply it:

```bash
kubectl apply -f recipes/deepseek-v4/perf/perf.yaml -n ${NAMESPACE}
```

## Compare All Targets

| Target             | GPUs     | Parallelism             | MoE backend                   | Spec. decode | Transfer |
| ------------------ | -------- | ----------------------- | ----------------------------- | ------------ | -------- |
| agg-b200           | 4x B200  | TP4 + EP                | FLASHINFER\_TRTLLM            | none         | —        |
| disagg-b200 (2P1D) | 12x B200 | TP4 / TP4               | FLASHINFER\_TRTLLM            | none         | NIXL GDR |
| agg-h200           | 4x H200  | DP4 + TP1 + EP          | MARLIN (FLASHINFER\_MLA attn) | MTP-1        | —        |
| disagg-h200 (4P3D) | 28x H200 | DP4+TP1+EP / DP4+TP1+EP | MARLIN                        | decode MTP-1 | NIXL GDR |

## Notes

* **Reasoning / tool parsers** are wired via the Dynamo variants (`--dyn-reasoning-parser deepseek_v4`, `--dyn-tool-call-parser deepseek_v4`); the engine-native parsers do not feed the Dynamo OpenAI renderer.
* **Disaggregated** targets require the per-rank NIC map (`VLLM_GPU_NIC_PCIE_MAPPING`) for GPU-Direct RDMA KV transfer; B200 ships `rdma/shared_ib`, H200 ships `rdma/ib`. See the [top-level recipe README](https://github.com/ai-dynamo/dynamo/blob/main/recipes/deepseek-v4/README.md#per-rank-nic-mapping-b200--h200-disaggregated).
* **AGG scales by independent single-replica DGDs** — KV-routed multi-replica AGG does not improve per-GPU throughput.
* Day-0 / GB200 SGLang variants exist in the recipe tree but are experimental (dev images) and not shown here.

## Source

* Recipe README: [recipes/deepseek-v4/deepseek-v4-flash/README.md](https://github.com/ai-dynamo/dynamo/blob/main/recipes/deepseek-v4/deepseek-v4-flash/README.md)
* Top-level DeepSeek-V4 README (shared workloads, NIC mapping, limitations): [recipes/deepseek-v4/README.md](https://github.com/ai-dynamo/dynamo/blob/main/recipes/deepseek-v4/README.md)
* Manifests: [vllm/](https://github.com/ai-dynamo/dynamo/tree/main/recipes/deepseek-v4/deepseek-v4-flash/vllm) and shared [perf/perf.yaml](https://github.com/ai-dynamo/dynamo/blob/main/recipes/deepseek-v4/perf/perf.yaml)