> For clean Markdown content of this page, append .md to this URL. For the complete documentation index, see https://docs.nvidia.com/dynamo/llms.txt. For full content including API reference and SDK examples, see https://docs.nvidia.com/dynamo/llms-full.txt.

# Kimi-K3

Each target below is a Dynamo + vLLM deployment of Moonshot AI's Kimi-K3 — a multimodal MoE model serving up to 1M-token context — with KV-aware routing, MXFP4-packed routed experts, and FP8 KV cache, running aggregated or with prefill/decode disaggregation on GB200 and GB300 NVL72 racks. KV transfer and tensor parallelism run over NVLink (MNNVL) using Kubernetes ComputeDomains. Pick your GPU architecture and serving topology; every command on this page updates to match.

<p>
  Choose your deployment target
</p>

GPU

GB200

<input type="radio" id="recipe-sku-gb300" name="recipe-sku" value="gb300" />

GB300

Topology

Aggregated

<input type="radio" id="recipe-variant-disagg" name="recipe-variant" value="disagg" />

Disaggregated

<b>Checkpoint</b> moonshotai/Kimi-K3

<b>Precision</b> MXFP4 experts + BF16 dense, FP8 KV

<b>GPUs</b> 16x GB200 (4 nodes), one worker

<b>Parallelism</b> TP16 over MNNVL

<b>Routing</b> KV-aware

<b>Checkpoint</b> moonshotai/Kimi-K3

<b>Precision</b> MXFP4 experts + BF16 dense, FP8 KV

<b>GPUs</b> 16x GB200 prefill + 16x GB200 decode (1P1D)

<b>Parallelism</b> TP16 / TP16 over MNNVL

<b>Routing</b> KV-aware, NIXL KV transfer over RDMA

<b>Checkpoint</b> moonshotai/Kimi-K3

<b>Precision</b> MXFP4 experts + BF16 dense, FP8 KV

<b>GPUs</b> 16x GB300 — two 8-GPU replicas

<b>Parallelism</b> TP8 per replica, KV-routed

<b>Routing</b> KV-aware across replicas

<b>Checkpoint</b> moonshotai/Kimi-K3

<b>Precision</b> MXFP4 experts + BF16 dense, FP8 KV

<b>GPUs</b> 8x GB300 prefill + 16x GB300 decode (1P2D)

<b>Parallelism</b> TP8 per replica over MNNVL

<b>Routing</b> KV-aware, NIXL/MNNVL KV transfer

## Prerequisites

* A Kubernetes cluster with the Dynamo platform installed and GB200 GPUs available — 16x (4 nodes) for aggregated, 32x (8 nodes, 4 prefill + 4 decode) for disaggregated. See the [Kubernetes Deployment Guide](/dynamo/dev/kubernetes/getting-started/quickstart).
* The NVIDIA DRA driver with ComputeDomain support installed (required for multi-node NVLink).
* A Hugging Face token with access to `moonshotai/Kimi-K3`.

- A Kubernetes cluster with the Dynamo platform installed and GB300 GPUs available — 16x (4 nodes) for aggregated, 24x (6 nodes) for disaggregated. See the [Kubernetes Deployment Guide](/dynamo/dev/kubernetes/getting-started/quickstart).
- The NVIDIA DRA driver with ComputeDomain support installed (required for multi-node NVLink).
- A Hugging Face token with access to `moonshotai/Kimi-K3`.

Create the namespace and token secret:

```bash
export NAMESPACE=your-namespace
kubectl create namespace ${NAMESPACE}
kubectl create secret generic hf-token-secret \
  --from-literal=HF_TOKEN="your-token" \
  -n ${NAMESPACE}
```

If you use the `model-cache` PVC (required on GB300, optional on GB200), edit `storageClassName` in `model-cache/model-cache.yaml` to a ReadWriteMany storage class on your cluster (`kubectl get storageclass`) before applying it. Review namespace, image tags, node selectors, and resource claims in the manifests as well.

## Deploy

The two SKUs source the checkpoint differently, so follow the block for your target.

The GB300 manifests read the checkpoint from a `model-cache` PVC (`HF_HOME=/model-cache`, `--model moonshotai/Kimi-K3`). Create the cache and download the weights:

```bash
# Edit storageClassName in model-cache/model-cache.yaml first.
kubectl apply -f recipes/kimi-k3/model-cache/model-cache.yaml -n ${NAMESPACE}
kubectl apply -f recipes/kimi-k3/model-cache/model-download.yaml -n ${NAMESPACE}
kubectl wait --for=condition=Complete job/model-download -n ${NAMESPACE} --timeout=14400s
```

The GB200 manifests read the checkpoint from node-local storage, not the `model-cache` PVC: they mount the host path `/mnt/stateful_partition/kube-ephemeral-ssd/models` (a GKE layout) at `/models` and serve `--model /models/model_weight` with `HF_HUB_OFFLINE=1`. Pick one of:

* **Pre-stage the weights on every GB200 node** in the deployment at `<host-path>/model_weight`, then edit the `models` `hostPath` in the deploy manifest to match your cluster's path.
* **Switch the manifest to the PVC flow** used by the GB300 recipes: create the cache and run the download job below, then in the deploy manifest replace the `models` `hostPath` volume with a `persistentVolumeClaim` (`claimName: model-cache`) mounted at `/model-cache`, add `HF_HOME=/model-cache`, and set `MODEL_PATH` to `moonshotai/Kimi-K3`.

```bash
# PVC flow only. Edit storageClassName in model-cache/model-cache.yaml first.
kubectl apply -f recipes/kimi-k3/model-cache/model-cache.yaml -n ${NAMESPACE}
kubectl apply -f recipes/kimi-k3/model-cache/model-download.yaml -n ${NAMESPACE}
kubectl wait --for=condition=Complete job/model-download -n ${NAMESPACE} --timeout=14400s
```

Then deploy the target DGD:

```bash
kubectl apply -f recipes/kimi-k3/vllm/agg-gb200-agentic/deploy.yaml -n ${NAMESPACE}
```

```bash
kubectl apply -f recipes/kimi-k3/vllm/disagg-gb200-agentic/deploy.yaml -n ${NAMESPACE}
```

```bash
kubectl apply -f recipes/kimi-k3/vllm/agg-gb300-agentic/deploy.yaml -n ${NAMESPACE}
```

```bash
kubectl apply -f recipes/kimi-k3/vllm/disagg-gb300-agentic/deploy.yaml -n ${NAMESPACE}
```

First worker launch loads weights and captures CUDA graphs and can take tens of minutes.

## Smoke Test

Send a test request to verify the deployment serves traffic. First forward the frontend port for your target:

```bash
kubectl port-forward svc/kimi-k3-agg-frontend 8000:8000 -n ${NAMESPACE}
```

```bash
kubectl port-forward svc/kimi-k3-disagg-frontend 8000:8000 -n ${NAMESPACE}
```

```bash
kubectl port-forward svc/kimi-k3-agg-frontend 8000:8000 -n ${NAMESPACE}
```

```bash
kubectl port-forward svc/kimi-k3-disagg-frontend 8000:8000 -n ${NAMESPACE}
```

All targets serve under the same model name, `moonshotai/Kimi-K3`:

```bash
curl http://localhost:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"moonshotai/Kimi-K3","messages":[{"role":"user","content":"Write a one-sentence readiness check."}],"max_tokens":64}'
```

Kimi-K3 reasons before answering, and tool calling is supported — the deployment uses the `kimi_k3` reasoning and tool-call parsers at the frontend.

## Compare All Targets

All four targets serve Kimi-K3 on vLLM with FP8 KV cache, KV-aware routing, prefix caching, and the FlashInfer TRT-LLM MoE backend, with tensor parallelism over NVLink (MNNVL):

|                 | GB200 aggregated     | GB200 disaggregated      | GB300 aggregated       | GB300 disaggregated     |
| --------------- | -------------------- | ------------------------ | ---------------------- | ----------------------- |
| **GPUs**        | 16x GB200 (4 nodes)  | 16x prefill + 16x decode | 16x GB300 (2 replicas) | 8x prefill + 16x decode |
| **Workers**     | 1                    | 1P + 1D                  | 2 (KV-routed)          | 1P + 2D                 |
| **Parallelism** | TP16                 | TP16 / TP16              | TP8 per replica        | TP8 per replica         |
| **KV transfer** | —                    | NIXL over RDMA (IB)      | —                      | NIXL over MNNVL         |
| **Attention**   | FlashInfer MLA       | FlashInfer MLA           | FlashInfer MLA         | FlashInfer MLA          |
| **Precision**   | MXFP4 + BF16, FP8 KV | MXFP4 + BF16, FP8 KV     | MXFP4 + BF16, FP8 KV   | MXFP4 + BF16, FP8 KV    |
| **Max context** | 1,048,576            | 1,048,576                | 1,048,576              | 1,048,576               |

## Notes

* All targets use KV-aware routing at the Dynamo frontend with prefix caching and KV events enabled.
* Kimi-K3 is multimodal — the deployments enable the multimodal encoder (`--enable-multimodal`) with data-parallel encoder sharding.
* Multi-node tensor parallelism runs over NVLink (MNNVL) via Kubernetes ComputeDomains (DRA). Disaggregated KV transfer uses NIXL — over NVLink on GB300, over RDMA InfiniBand on GB200 (`UCX_TLS=^cuda_ipc`, GKE `rdma-*` network resources).
* Kimi-K3 support requires the updated Dynamo frontend (tokenizer, `kimi_k3` reasoning and tool-call parsers) that ships with this release.

## Source

* Source README: [recipes/kimi-k3/README.md](https://github.com/ai-dynamo/dynamo/blob/main/recipes/kimi-k3/README.md)
* Aggregated GB200: [deploy.yaml](https://github.com/ai-dynamo/dynamo/blob/main/recipes/kimi-k3/vllm/agg-gb200-agentic/deploy.yaml)
* Disaggregated GB200: [deploy.yaml](https://github.com/ai-dynamo/dynamo/blob/main/recipes/kimi-k3/vllm/disagg-gb200-agentic/deploy.yaml)
* Aggregated GB300: [deploy.yaml](https://github.com/ai-dynamo/dynamo/blob/main/recipes/kimi-k3/vllm/agg-gb300-agentic/deploy.yaml)
* Disaggregated GB300: [deploy.yaml](https://github.com/ai-dynamo/dynamo/blob/main/recipes/kimi-k3/vllm/disagg-gb300-agentic/deploy.yaml)
* Setup assets: [model-cache.yaml](https://github.com/ai-dynamo/dynamo/blob/main/recipes/kimi-k3/model-cache/model-cache.yaml) and [model-download.yaml](https://github.com/ai-dynamo/dynamo/blob/main/recipes/kimi-k3/model-cache/model-download.yaml)