Kimi-K3

Prerequisites

Create the namespace and token secret:

$
export NAMESPACE=your-namespace
$
kubectl create namespace ${NAMESPACE}
$
kubectl create secret generic hf-token-secret \
>
  --from-literal=HF_TOKEN="your-token" \
>
  -n ${NAMESPACE}

Deploy

Create the shared-model-cache PVC and download the Kimi-K3 checkpoint and both DSPARK draft models. The download can take several hours on a cold cache:

$
# Edit storageClassName in model-cache/model-cache.yaml first.
$
kubectl apply -f recipes/kimi-k3/model-cache/model-cache.yaml -n ${NAMESPACE}
$
kubectl apply -f recipes/kimi-k3/model-cache/model-download.yaml -n ${NAMESPACE}
$
kubectl wait --for=condition=Complete job/model-download -n ${NAMESPACE} --timeout=14400s

The workers mount the PVC at /shared-model-cache and resolve the checkpoint and any configured DSPARK draft model from the local Hugging Face cache with HF_HUB_OFFLINE=1.

Deploy the selected DynamoGraphDeployment (DGD):

$
export DGD=kimi-k3-vllm-h200-agg-agentic
$
kubectl apply -f recipes/kimi-k3/vllm/agg-h200-agentic/deploy.yaml -n ${NAMESPACE}

$
export DGD=kimi-k3-vllm-gb200-agg-agentic
$
kubectl apply -f recipes/kimi-k3/vllm/agg-gb200-agentic/deploy.yaml -n ${NAMESPACE}

$
export DGD=kimi-k3-vllm-gb300-agg-agentic
$
kubectl apply -f recipes/kimi-k3/vllm/agg-gb300-agentic/deploy.yaml -n ${NAMESPACE}

$
export DGD=kimi-k3-vllm-gb300-disagg-agentic
$
kubectl apply -f recipes/kimi-k3/vllm/disagg-gb300-agentic/deploy.yaml -n ${NAMESPACE}

$
export DGD=kimi-k3-sglang-gb200-agg-agentic
$
kubectl apply -f recipes/kimi-k3/sglang/agg-gb200-agentic/deploy.yaml -n ${NAMESPACE}

$
export DGD=kimi-k3-sglang-gb200-disagg-agentic
$
kubectl apply -f recipes/kimi-k3/sglang/disagg-gb200-agentic/deploy.yaml -n ${NAMESPACE}

$
export DGD=kimi-k3-sglang-gb300-agg-agentic
$
kubectl apply -f recipes/kimi-k3/sglang/agg-gb300-agentic/deploy.yaml -n ${NAMESPACE}

$
export DGD=kimi-k3-sglang-gb300-disagg-agentic
$
kubectl apply -f recipes/kimi-k3/sglang/disagg-gb300-agentic/deploy.yaml -n ${NAMESPACE}

The first worker launch loads weights and captures CUDA graphs, which can take tens of minutes.

Smoke Test

Send a test request to verify the selected deployment serves traffic. The deploy command sets DGD to its resource name. Forward the frontend port:

$
kubectl port-forward svc/${DGD}-frontend 8000:8000 -n ${NAMESPACE}

All Kimi-K3 recipes serve under the same model name, moonshotai/Kimi-K3:

$
curl http://localhost:8000/v1/chat/completions \
>
  -H 'Content-Type: application/json' \
>
  -d '{"model":"moonshotai/Kimi-K3","messages":[{"role":"user","content":"Write a one-sentence readiness check."}],"max_tokens":64}'

Kimi-K3 reasons before answering, and tool calling is supported — the deployment uses the kimi_k3 reasoning and tool-call parsers at the frontend.

Benchmark

See perf/README.md for the full benchmark workflow: staging the trace on the PVC, running the AIPerf trace-replay Job, running a concurrency sweep, and fetching artifacts.

Optimization Targets

Recipes are optimized for the following agentic workload and target user interactivity:

The benchmark replays the Mooncake-format agentic trace described in perf/README.md.

Performance Results

Benchmarking uses a synthetic acceptance length with the SpeedBench coding AL.

Compare All Targets

Limitations

• With JSON structured decoding, output for non-object top-level items can appear in reasoning_content instead of content.
• Requests containing logprobs or stop_token_ids are not supported and may cause disaggregated recipes to crash or enter a bad state

Source

• Setup assets: model-cache.yaml and model-download.yaml
• Benchmark instructions: recipes/kimi-k3/perf/README.md