> For clean Markdown content of this page, append .md to this URL. For the complete documentation index, see https://docs.nvidia.com/dynamo/llms.txt. For full content including API reference and SDK examples, see https://docs.nvidia.com/dynamo/llms-full.txt.

# Recipes

Start with the model, runtime, and hardware you need to run. Model cards are sorted newest to oldest by release date. Use [Feature Benchmarks](/dynamo/recipes/benchmarks) when you want evidence that a Dynamo feature or topology helps under controlled traffic.

<form>
  <input type="radio" id="provider-nvidia" name="provider" />

  <input type="radio" id="provider-qwen" name="provider" />

  <input type="radio" id="provider-deepseek" name="provider" />

  <input type="radio" id="provider-moonshot" name="provider" />

  <input type="radio" id="provider-meta" name="provider" />

  <input type="radio" id="provider-openai" name="provider" />

  <input type="radio" id="provider-zai" name="provider" />

  <input type="radio" id="provider-thinkingmachines" name="provider" />

  <input type="radio" id="runtime-vllm" name="runtime" />

  <input type="radio" id="runtime-trtllm" name="runtime" />

  <input type="radio" id="runtime-sglang" name="runtime" />

  <input type="radio" id="hardware-h100" name="hardware" />

  <input type="radio" id="hardware-h200" name="hardware" />

  <input type="radio" id="hardware-gb200" name="hardware" />

  <input type="radio" id="hardware-gb300" name="hardware" />

  <input type="radio" id="hardware-b200" name="hardware" />

  Recipe catalog

  ### 17 model families

  Filter by provider, runtime, and hardware. Each card notes its serving technique and workload shape.

  **95**Deployable configurations Reset filters

  Provider

  All

  NVIDIA

  Qwen

  DeepSeek

  Moonshot

  Meta

  OpenAI

  Z.AI

  Thinking Machines

  Runtime

  All

  vLLM

  TensorRT-LLM

  SGLang

  Hardware

  All

  H100

  H200

  GB200

  GB300

  B200

  ![](https://fdr-prod-docs-files-public.s3.us-east-1.amazonaws.com/dynamo.docs.buildwithfern.com/e60742a0b37f805e654249dedb7b6ca11c785a60a2dd9ae33864668dce217693/assets/img/recipes/providers/zai-org.svg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIA6KXJSKKNFOCF7G4B%2F20261001%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20261001T002356Z&X-Amz-Expires=604800&X-Amz-Signature=455dbf0990c635feab235503d8b8ac6d7f398b5c8bbad594da5be5cc28de2dd4&X-Amz-SignedHeaders=host&x-amz-checksum-mode=ENABLED&x-id=GetObject)

  ### GLM-5.3-Flash

  Z.AI

  vLLM recipes for agentic traffic on GB200 or H200, aggregated or 1P1D disaggregated with KV-aware routing, NIXL KV transfer, and H200 MTP7.

  vLLM4x/8x GB200 · 8x/16x H200Agentic · up to 1M context

  [Open recipe](/dynamo/recipes/glm-5-3-flash)

  ![](https://fdr-prod-docs-files-public.s3.us-east-1.amazonaws.com/dynamo.docs.buildwithfern.com/6c9ba98efd7d603b5476db9313617ebffd57a53acb0f9ef724ef4c7288a8a527/assets/img/recipes/providers/moonshotai.webp?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIA6KXJSKKNFOCF7G4B%2F20261001%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20261001T002356Z&X-Amz-Expires=604800&X-Amz-Signature=68e40214cc9b6ce2af33c60492da55cac59e5e010bd7ab3ab040874393a39e86&X-Amz-SignedHeaders=host&x-amz-checksum-mode=ENABLED&x-id=GetObject)

  ### Kimi-K3

  Moonshot / NVIDIA

  Day-0 vLLM recipes for Moonshot's Kimi-K3 on GB200 and GB300, aggregated or with prefill/decode disaggregation and KV-aware routing.

  vLLMGB200 · GB300Day-0

  [Open recipe](/dynamo/recipes/kimi-k3)

  ![](https://fdr-prod-docs-files-public.s3.us-east-1.amazonaws.com/dynamo.docs.buildwithfern.com/588266b8e1b85bad9da8849973fb14ea59b7422a2b30b9dbf7e7e8db5c246321/assets/img/recipes/providers/qwen.webp?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIA6KXJSKKNFOCF7G4B%2F20261001%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20261001T002356Z&X-Amz-Expires=604800&X-Amz-Signature=596fcee75d0c4b15f6915c8f7c685b1e9234d5ea6387c4b3610fb77bb347af84&X-Amz-SignedHeaders=host&x-amz-checksum-mode=ENABLED&x-id=GetObject)

  ### Qwen3.8-2.4T-A95B

  Qwen

  vLLM and SGLang recipes for chat and agentic traffic on GB300 or GB200, aggregated or disaggregated, with KV-aware routing, MTP speculation, and FP8 weights/KV over MNNVL.

  vLLM + SGLang16x GB300 · 16x GB200Chat + agentic · 262K context

  [Open recipe](/dynamo/recipes/qwen-3-8-2-4t-a95b-fp8)

  ![](https://fdr-prod-docs-files-public.s3.us-east-1.amazonaws.com/dynamo.docs.buildwithfern.com/e60742a0b37f805e654249dedb7b6ca11c785a60a2dd9ae33864668dce217693/assets/img/recipes/providers/zai-org.svg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIA6KXJSKKNFOCF7G4B%2F20261001%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20261001T002356Z&X-Amz-Expires=604800&X-Amz-Signature=455dbf0990c635feab235503d8b8ac6d7f398b5c8bbad594da5be5cc28de2dd4&X-Amz-SignedHeaders=host&x-amz-checksum-mode=ENABLED&x-id=GetObject)

  ### GLM-5.3/5.2

  Z.AI / NVIDIA

  SGLang recipes for long-context agentic traffic on B200 (NVFP4) or H200 (FP8), aggregated or disaggregated, with KV-aware routing, EAGLE MTP speculation, and HiCache CPU offload.

  SGLang4x/12x B200 · 8x/16x H200Agentic · up to 500K context

  [Open recipe](/dynamo/recipes/glm-5-2)

  ![](https://fdr-prod-docs-files-public.s3.us-east-1.amazonaws.com/dynamo.docs.buildwithfern.com/ef907e01669290064ce1db3d7902203fb2786dc110d2c496a8a02ad912d32e7e/assets/img/recipes/providers/thinkingmachines.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIA6KXJSKKNFOCF7G4B%2F20261001%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20261001T002356Z&X-Amz-Expires=604800&X-Amz-Signature=b8eebc41328cddcff781b18d9f2239b8e778624a6bc3dc406d029f3f7ddf8e56&X-Amz-SignedHeaders=host&x-amz-checksum-mode=ENABLED&x-id=GetObject)

  ### Inkling NVFP4

  Thinking Machines

  Day-0 aggregated SGLang recipe for Thinking Machines' first open-weights model — multimodal MoE with controllable reasoning effort, TP8 with EAGLE speculation.

  SGLang8x B200Text + image + audioDay-0

  [Open recipe](/dynamo/recipes/inkling)

  ![](https://fdr-prod-docs-files-public.s3.us-east-1.amazonaws.com/dynamo.docs.buildwithfern.com/6c9ba98efd7d603b5476db9313617ebffd57a53acb0f9ef724ef4c7288a8a527/assets/img/recipes/providers/moonshotai.webp?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIA6KXJSKKNFOCF7G4B%2F20261001%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20261001T002356Z&X-Amz-Expires=604800&X-Amz-Signature=68e40214cc9b6ce2af33c60492da55cac59e5e010bd7ab3ab040874393a39e86&X-Amz-SignedHeaders=host&x-amz-checksum-mode=ENABLED&x-id=GetObject)

  ### Kimi-K2.6

  Moonshot / NVIDIA

  Multi-target vLLM matrix covering B200 and H200 for chat and agentic traffic, with Eagle3 MLA speculation and KV-aware routing.

  vLLM4x B200 / 8x H200Chat + agentic

  [Open recipe](/dynamo/recipes/kimi-k2-6)

  ![](https://fdr-prod-docs-files-public.s3.us-east-1.amazonaws.com/dynamo.docs.buildwithfern.com/75f85aa1b5c95694f9199ae163b0a42ccd45c8ef2870d7dc494c1733b869444d/assets/img/recipes/providers/nvidia.webp?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIA6KXJSKKNFOCF7G4B%2F20261001%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20261001T002356Z&X-Amz-Expires=604800&X-Amz-Signature=e773f7f82b1ec83ab11d98dbbd338d2dd97098afe712dc3c93c2a1aabcd56f4d&X-Amz-SignedHeaders=host&x-amz-checksum-mode=ENABLED&x-id=GetObject)

  ### Nemotron 3.5 Lightning

  NVIDIA

  30B hybrid Mamba/Attention/MoE recipe matrix for H100, H200, B200, and GB200, with NVFP4 and BF16 variants, vLLM MTP/DFlash/DSpark targets, DSpark with KV-routing, and experimental TensorRT-LLM MTP/no-spec.

  vLLM + TRT-LLM1-4x GPUAgentic · 1M context

  [Open recipe](/dynamo/recipes/nemotron-3-5-lightning)

  ![](https://fdr-prod-docs-files-public.s3.us-east-1.amazonaws.com/dynamo.docs.buildwithfern.com/75f85aa1b5c95694f9199ae163b0a42ccd45c8ef2870d7dc494c1733b869444d/assets/img/recipes/providers/nvidia.webp?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIA6KXJSKKNFOCF7G4B%2F20261001%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20261001T002356Z&X-Amz-Expires=604800&X-Amz-Signature=e773f7f82b1ec83ab11d98dbbd338d2dd97098afe712dc3c93c2a1aabcd56f4d&X-Amz-SignedHeaders=host&x-amz-checksum-mode=ENABLED&x-id=GetObject)

  ### Nemotron 3 Ultra

  NVIDIA

  Aggregated vLLM targets for B200 and H200 chat and agentic traffic with MTP speculative decoding and trace-backed benchmarks.

  vLLM4x B200 / 8x H200Chat + agentic

  [Open recipe](/dynamo/recipes/nemotron-3-ultra)

  ![](https://fdr-prod-docs-files-public.s3.us-east-1.amazonaws.com/dynamo.docs.buildwithfern.com/75f85aa1b5c95694f9199ae163b0a42ccd45c8ef2870d7dc494c1733b869444d/assets/img/recipes/providers/nvidia.webp?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIA6KXJSKKNFOCF7G4B%2F20261001%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20261001T002356Z&X-Amz-Expires=604800&X-Amz-Signature=e773f7f82b1ec83ab11d98dbbd338d2dd97098afe712dc3c93c2a1aabcd56f4d&X-Amz-SignedHeaders=host&x-amz-checksum-mode=ENABLED&x-id=GetObject)

  ### Nemotron 3 Super

  NVIDIA

  NVFP4 and FP8 vLLM targets for B200 and H200 chat and agentic traffic with MTP speculative decoding and KV-aware routing.

  vLLM4x B200 / 4x H200Chat + agentic

  [Open recipe](/dynamo/recipes/nemotron-3-super)

  ![](https://fdr-prod-docs-files-public.s3.us-east-1.amazonaws.com/dynamo.docs.buildwithfern.com/e60742a0b37f805e654249dedb7b6ca11c785a60a2dd9ae33864668dce217693/assets/img/recipes/providers/zai-org.svg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIA6KXJSKKNFOCF7G4B%2F20261001%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20261001T002356Z&X-Amz-Expires=604800&X-Amz-Signature=455dbf0990c635feab235503d8b8ac6d7f398b5c8bbad594da5be5cc28de2dd4&X-Amz-SignedHeaders=host&x-amz-checksum-mode=ENABLED&x-id=GetObject)

  ### GLM-5 NVFP4

  NVIDIA / Z.AI

  20x GB200 SGLang P/D recipe for 1K input / 8K output traffic with EAGLE speculative decoding, plus an AWS EFA variant.

  SGLang20x GB200Long output / static ISL-OSL

  [Open recipe](/dynamo/recipes/glm-5-nvfp4)

  ![](https://fdr-prod-docs-files-public.s3.us-east-1.amazonaws.com/dynamo.docs.buildwithfern.com/4cd5dd62104f4bfa1caa6bfa3a13e06b1577d24ea03bfd1e78e4c5aea8ceed9f/assets/img/recipes/providers/deepseek-ai.webp?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIA6KXJSKKNFOCF7G4B%2F20261001%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20261001T002356Z&X-Amz-Expires=604800&X-Amz-Signature=7d03a2f5b05408d7036f7282e45d2be89f53eceb25be89e2f666074297c062fc&X-Amz-SignedHeaders=host&x-amz-checksum-mode=ENABLED&x-id=GetObject)

  ### DeepSeek-V4-Pro

  NVIDIA checkpoint / DeepSeek

  vLLM agentic recipe — MoE 1.6T / 49B active, B200 (NVFP4, 1M ctx) and H200 (FP8), aggregated or disaggregated with MTP-2 and KV-aware routing.

  vLLM8–32x B200/H200Agentic

  [Open recipe](/dynamo/recipes/deepseek-v4-pro)

  ![](https://fdr-prod-docs-files-public.s3.us-east-1.amazonaws.com/dynamo.docs.buildwithfern.com/4cd5dd62104f4bfa1caa6bfa3a13e06b1577d24ea03bfd1e78e4c5aea8ceed9f/assets/img/recipes/providers/deepseek-ai.webp?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIA6KXJSKKNFOCF7G4B%2F20261001%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20261001T002356Z&X-Amz-Expires=604800&X-Amz-Signature=7d03a2f5b05408d7036f7282e45d2be89f53eceb25be89e2f666074297c062fc&X-Amz-SignedHeaders=host&x-amz-checksum-mode=ENABLED&x-id=GetObject)

  ### DeepSeek-V4-Flash

  NVIDIA checkpoint / DeepSeek

  vLLM agentic recipe — MoE 284B / 13B active, B200 (NVFP4) and H200 (FP8), aggregated or disaggregated (2P1D / 4P3D) with KV-aware routing.

  vLLM4–28x B200/H200Agentic

  [Open recipe](/dynamo/recipes/deepseek-v4-flash)

  ![](https://fdr-prod-docs-files-public.s3.us-east-1.amazonaws.com/dynamo.docs.buildwithfern.com/51767fd618f19941f3f2ae5a88c526ccc4ca874a0444226f09f6ab6da7346897/assets/img/recipes/providers/openai.webp?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIA6KXJSKKNFOCF7G4B%2F20261001%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20261001T002356Z&X-Amz-Expires=604800&X-Amz-Signature=b3193a16e3fed0c6436ed11259bb178d70a5e59f85ec95a9034523034a67b281&X-Amz-SignedHeaders=host&x-amz-checksum-mode=ENABLED&x-id=GetObject)

  ### GPT-OSS-120B

  OpenAI

  TensorRT-LLM GB200 targets for static traffic plus vLLM B200/H200 agentic targets (agg + disagg) with KV-aware routing and EAGLE3 speculative decoding.

  TRT-LLM + vLLMGB200 · B200 · H200Static + agentic

  [Open recipe](/dynamo/recipes/gpt-oss-120b)

  ![](https://fdr-prod-docs-files-public.s3.us-east-1.amazonaws.com/dynamo.docs.buildwithfern.com/588266b8e1b85bad9da8849973fb14ea59b7422a2b30b9dbf7e7e8db5c246321/assets/img/recipes/providers/qwen.webp?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIA6KXJSKKNFOCF7G4B%2F20261001%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20261001T002356Z&X-Amz-Expires=604800&X-Amz-Signature=596fcee75d0c4b15f6915c8f7c685b1e9234d5ea6387c4b3610fb77bb347af84&X-Amz-SignedHeaders=host&x-amz-checksum-mode=ENABLED&x-id=GetObject)

  ### Qwen3-32B

  Qwen

  Disaggregated vLLM serving with KV-aware routing on 16x H200, for multi-turn conversational traffic with prefix reuse.

  vLLM16x H200Multi-turn conversationRelated benchmark

  [Open recipe](/dynamo/recipes/qwen3-32b)

  ![](https://fdr-prod-docs-files-public.s3.us-east-1.amazonaws.com/dynamo.docs.buildwithfern.com/588266b8e1b85bad9da8849973fb14ea59b7422a2b30b9dbf7e7e8db5c246321/assets/img/recipes/providers/qwen.webp?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIA6KXJSKKNFOCF7G4B%2F20261001%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20261001T002356Z&X-Amz-Expires=604800&X-Amz-Signature=596fcee75d0c4b15f6915c8f7c685b1e9234d5ea6387c4b3610fb77bb347af84&X-Amz-SignedHeaders=host&x-amz-checksum-mode=ENABLED&x-id=GetObject)

  ### Qwen3.5-122B-A10B

  Qwen

  vLLM recipes for long-context agentic traffic on B200 NVFP4 and H200 FP8, aggregated or disaggregated 1P2D, with KV-aware routing and MTP speculation on H200 aggregated.

  vLLM2-4x B200/H200Agentic trace replay

  [Open recipe](/dynamo/recipes/qwen3-5-122b)

  ![](https://fdr-prod-docs-files-public.s3.us-east-1.amazonaws.com/dynamo.docs.buildwithfern.com/588266b8e1b85bad9da8849973fb14ea59b7422a2b30b9dbf7e7e8db5c246321/assets/img/recipes/providers/qwen.webp?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIA6KXJSKKNFOCF7G4B%2F20261001%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20261001T002356Z&X-Amz-Expires=604800&X-Amz-Signature=596fcee75d0c4b15f6915c8f7c685b1e9234d5ea6387c4b3610fb77bb347af84&X-Amz-SignedHeaders=host&x-amz-checksum-mode=ENABLED&x-id=GetObject)

  ### Qwen3-235B-A22B FP8

  Qwen

  16-GPU TensorRT-LLM recipe matrix for Hopper/Blackwell and aggregate/P-D serving.

  TRT-LLM16x H100/H200Static ISL-OSL

  [Open recipe](/dynamo/recipes/qwen3-235b-a22b-fp8)

  ![](https://fdr-prod-docs-files-public.s3.us-east-1.amazonaws.com/dynamo.docs.buildwithfern.com/588266b8e1b85bad9da8849973fb14ea59b7422a2b30b9dbf7e7e8db5c246321/assets/img/recipes/providers/qwen.webp?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIA6KXJSKKNFOCF7G4B%2F20261001%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20261001T002356Z&X-Amz-Expires=604800&X-Amz-Signature=596fcee75d0c4b15f6915c8f7c685b1e9234d5ea6387c4b3610fb77bb347af84&X-Amz-SignedHeaders=host&x-amz-checksum-mode=ENABLED&x-id=GetObject)

  ### Qwen3-32B FP8

  Qwen

  FP8 recipe set spanning TensorRT-LLM aggregate, TensorRT-LLM P/D, and vLLM P/D targets.

  TRT-LLM2-8x GPUStatic ISL-OSL

  [Open recipe](/dynamo/recipes/qwen3-32b-fp8)

  No recipes match the selected filters.
</form>

## Related Feature Benchmarks

Use [Feature Benchmarks](/dynamo/recipes/benchmarks) when your question starts with a feature or performance claim, such as KV routing, embedding cache, speculative decoding, or topology. Recipe pages link back to Feature Benchmark pages whenever a recipe came from a comparison or feature-stack run.