MiniMax M3 NVFP4

Each target below is a Dynamo + vLLM deployment of MiniMax M3 — a multimodal model serving up to 1M-token context with MoE and sparse attention — with KV-aware routing, NVFP4-packed routed experts, and FP8 KV cache, running aggregated or with prefill/decode disaggregation on GB200 NVL72 rack. KV transfer and tensor parallelism run over NVLink (MNNVL) using Kubernetes ComputeDomains.

Prerequisites

Both targets require:

• A Kubernetes cluster with the Dynamo platform installed. See the Kubernetes Deployment Guide.
• GB200 GPU nodes: 12 GPUs for aggregated serving or 24 GPUs for disaggregated serving.
• The NVIDIA DRA driver with ComputeDomain support installed (required for multi-node NVLink).
• A Hugging Face token with access to nvidia/MiniMax-M3-NVFP4 and Inferact/MiniMax-M3-EAGLE3-GQA.
• Standalone Kustomize v5.8.1 and a private cluster Kustomization with registry credentials, GB200 placement, cache bindings, and networking appropriate to the cluster.
• The disaggregated target requires network-interface annotations and UCX/NCCL interface settings that match the target cluster. It also requires all six workers to use a compatible rack-scoped ComputeDomain channel.

Set the namespace and create the Hugging Face token secret:

export NAMESPACE=your-namespace
kubectl create namespace "${NAMESPACE}"
kubectl create secret generic hf-token-secret \
  --from-literal=HF_TOKEN="your-token" \
  -n "${NAMESPACE}"

If the namespace and Secret already exist, use them instead. Run the following commands from the Dynamo repository root.

Deploy

Edit storageClassName in the model-cache manifest, then populate the cache:

kubectl apply -f recipes/minimax-m3/model-cache/model-cache.yaml -n "${NAMESPACE}"
kubectl apply -f recipes/minimax-m3/model-cache/model-download.yaml -n "${NAMESPACE}"
kubectl wait --for=condition=complete job/minimax-m3-model-download \
  -n "${NAMESPACE}" --timeout=14400s

Prepare a private cluster Kustomization for the selected deployment using the beta cluster starter. Keep filled site values outside the recipe contribution.

Set CLUSTER_KUSTOMIZATION to the filled private directory, then render and apply the validated composition with standalone Kustomize v5.8.1:

kustomize build --load-restrictor LoadRestrictionsNone "$CLUSTER_KUSTOMIZATION" | \
  kubectl apply --dry-run=server -f - -n "$NAMESPACE"
kustomize build --load-restrictor LoadRestrictionsNone "$CLUSTER_KUSTOMIZATION" | \
  kubectl apply -f - -n "$NAMESPACE"

Use LoadRestrictionsNone only with reviewed paths. It allows the private composition to reference the portable recipe and generated schema.

Monitor the deployment until the frontend and every worker report ready:

kubectl get dynamographdeployments,pods -n "${NAMESPACE}"

Smoke Test

Forward the selected frontend service.

kubectl port-forward \
  svc/minimax-m3-agg-gb200-agentic-frontend 8000:8000 \
  -n "${NAMESPACE}"

kubectl port-forward \
  svc/minimax-m3-disagg-gb200-agentic-frontend 8000:8000 \
  -n "${NAMESPACE}"

In another terminal, send a chat request:

curl http://localhost:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "nvidia/MiniMax-M3-NVFP4",
    "messages": [{"role": "user", "content": "Respond with a short readiness message."}],
    "max_tokens": 64
  }'

A successful smoke test returns an HTTP 200 response containing a chat completion.

Benchmark

See the MiniMax M3 benchmark README for the full workflow—staging the trace on the PVC, running a concurrency sweep, and fetching artifacts—and use the checked-in AIPerf Job to run the trace replay.

Both serving manifests default to real EAGLE3 verification with three draft tokens. The benchmark Job does not change that setting. For synthetic-acceptance experiments, follow the benchmark-only configuration instructions. Use the checked-in aggregated Component or disaggregated Component described there. Both select acceptance length 2.89.

Optimization Targets

The intended agentic workload and interactivity targets are:

The benchmark replays the Mooncake-format agentic trace described in the benchmark README.

Performance Results

The verified results below use synthetic acceptance with the SpeedBench coding acceptance length. They describe three aggregated TP4 workers (12 GPUs) and three prefill plus three decode TP4 workers (24 GPUs), respectively.

Compare All Targets

Both targets use vLLM, TP4 workers, FP8 KV cache, prefix caching, KV events, KV-aware routing, and EAGLE3 speculative decoding with three speculative tokens.

Notes

• Both manifests pin the model checkpoint and the EAGLE3 checkpoint to specific Hugging Face revisions.
• Both manifests select real EAGLE3 verification by default; synthetic acceptance length 2.89 is an explicit benchmark-only option.
• The aggregated target uses vLLM’s OffloadingConnector with CPUOffloadingSpec and sets cpu_bytes_to_use to 400000000000.
• Both targets enable --frontend-decoding. This path supports VP8/VP9 video input but does not support H.264/H.265. Re-encode H.264/H.265 videos to VP9 before sending them to these deployments.

Source

• Recipe README: recipes/minimax-m3/README.md
• Aggregated GB200: deploy.yaml
• Disaggregated GB200: deploy.yaml
• Model cache: model-cache.yaml
• Model download: model-download.yaml