> For clean Markdown content of this page, append .md to this URL. For the complete documentation index, see https://docs.nvidia.com/dynamo/llms.txt. For full content including API reference and SDK examples, see https://docs.nvidia.com/dynamo/llms-full.txt.

# Recipes

Start with the model, runtime, and hardware you need to run. Model cards are sorted newest to oldest by release date. Use [Feature Benchmarks](/dynamo/v1.3.0/benchmarks) when you want evidence that a Dynamo feature or topology helps under controlled traffic.

<form>
  <input type="radio" id="provider-nvidia" name="provider" />

  <input type="radio" id="provider-qwen" name="provider" />

  <input type="radio" id="provider-deepseek" name="provider" />

  <input type="radio" id="provider-moonshot" name="provider" />

  <input type="radio" id="provider-meta" name="provider" />

  <input type="radio" id="provider-openai" name="provider" />

  <input type="radio" id="provider-zai" name="provider" />

  <input type="radio" id="provider-thinkingmachines" name="provider" />

  <input type="radio" id="runtime-vllm" name="runtime" />

  <input type="radio" id="runtime-trtllm" name="runtime" />

  <input type="radio" id="runtime-sglang" name="runtime" />

  <input type="radio" id="hardware-h100" name="hardware" />

  <input type="radio" id="hardware-h200" name="hardware" />

  <input type="radio" id="hardware-gb200" name="hardware" />

  <input type="radio" id="hardware-b200" name="hardware" />

  Recipe catalog

  ### 14 model families

  Filter by provider, runtime, and hardware. Each card notes its serving technique and workload shape.

  **38**Deployable configurations Reset filters

  Provider

  All

  NVIDIA

  Qwen

  DeepSeek

  Moonshot

  Meta

  OpenAI

  Z.AI

  Thinking Machines

  Runtime

  All

  vLLM

  TensorRT-LLM

  SGLang

  Hardware

  All

  H100

  H200

  GB200

  B200

  ![](https://fdr-prod-docs-files-public.s3.us-east-1.amazonaws.com/dynamo.docs.buildwithfern.com/e60742a0b37f805e654249dedb7b6ca11c785a60a2dd9ae33864668dce217693/pages-v1.3.0/assets/img/recipes/providers/zai-org.svg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIA6KXJSKKNFOCF7G4B%2F20260913%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260913T030128Z&X-Amz-Expires=604800&X-Amz-Signature=faff67df69a984add82c3504ad61d9cdf23e30f66b9964396a45e3753ae5897a&X-Amz-SignedHeaders=host&x-amz-checksum-mode=ENABLED&x-id=GetObject)

  ### GLM-5.2

  Z.AI / NVIDIA

  SGLang recipes for long-context agentic traffic on B200 (NVFP4) or H200 (FP8), aggregated or disaggregated, with KV-aware routing, EAGLE MTP speculation, and HiCache CPU offload.

  SGLang4x/12x B200 · 8x/16x H200Agentic · up to 500K context

  [Open recipe](/dynamo/v1.3.0/recipes/glm-5-2)

  ![](https://fdr-prod-docs-files-public.s3.us-east-1.amazonaws.com/dynamo.docs.buildwithfern.com/ef907e01669290064ce1db3d7902203fb2786dc110d2c496a8a02ad912d32e7e/pages-v1.3.0/assets/img/recipes/providers/thinkingmachines.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIA6KXJSKKNFOCF7G4B%2F20260913%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260913T030128Z&X-Amz-Expires=604800&X-Amz-Signature=65ef4e06a691d4e35ed41d22f5eb5101a59615cfa770529070f3aef000274815&X-Amz-SignedHeaders=host&x-amz-checksum-mode=ENABLED&x-id=GetObject)

  ### Inkling NVFP4

  Thinking Machines

  Day-0 aggregated SGLang recipe for Thinking Machines' first open-weights model — multimodal MoE with controllable reasoning effort, TP8 with EAGLE speculation.

  SGLang8x B200Text + image + audioDay-0

  [Open recipe](/dynamo/v1.3.0/recipes/inkling)

  ![](https://fdr-prod-docs-files-public.s3.us-east-1.amazonaws.com/dynamo.docs.buildwithfern.com/6c9ba98efd7d603b5476db9313617ebffd57a53acb0f9ef724ef4c7288a8a527/pages-v1.3.0/assets/img/recipes/providers/moonshotai.webp?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIA6KXJSKKNFOCF7G4B%2F20260913%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260913T030128Z&X-Amz-Expires=604800&X-Amz-Signature=be101bfb470e92b96af90bd3657f304391e48a811e42d2e88450e798265b7dc8&X-Amz-SignedHeaders=host&x-amz-checksum-mode=ENABLED&x-id=GetObject)

  ### Kimi-K2.6

  Moonshot / NVIDIA

  Multi-target vLLM matrix covering B200 and H200 for chat and agentic traffic, with Eagle3 MLA speculation and KV-aware routing.

  vLLM4x B200 / 8x H200Chat + agentic

  [Open recipe](/dynamo/v1.3.0/recipes/kimi-k2-6)

  ![](https://fdr-prod-docs-files-public.s3.us-east-1.amazonaws.com/dynamo.docs.buildwithfern.com/75f85aa1b5c95694f9199ae163b0a42ccd45c8ef2870d7dc494c1733b869444d/pages-v1.3.0/assets/img/recipes/providers/nvidia.webp?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIA6KXJSKKNFOCF7G4B%2F20260913%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260913T030128Z&X-Amz-Expires=604800&X-Amz-Signature=a1a8f2b05e4de0fed3b6309aca40ad21a24442ae2ec586e0ec1823368cf5e353&X-Amz-SignedHeaders=host&x-amz-checksum-mode=ENABLED&x-id=GetObject)

  ### Nemotron 3 Ultra

  NVIDIA

  Aggregated vLLM targets for B200 and H200 chat and agentic traffic with MTP speculative decoding and trace-backed benchmarks.

  vLLM4x B200 / 8x H200Chat + agentic

  [Open recipe](/dynamo/v1.3.0/recipes/nemotron-3-ultra)

  ![](https://fdr-prod-docs-files-public.s3.us-east-1.amazonaws.com/dynamo.docs.buildwithfern.com/75f85aa1b5c95694f9199ae163b0a42ccd45c8ef2870d7dc494c1733b869444d/pages-v1.3.0/assets/img/recipes/providers/nvidia.webp?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIA6KXJSKKNFOCF7G4B%2F20260913%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260913T030128Z&X-Amz-Expires=604800&X-Amz-Signature=a1a8f2b05e4de0fed3b6309aca40ad21a24442ae2ec586e0ec1823368cf5e353&X-Amz-SignedHeaders=host&x-amz-checksum-mode=ENABLED&x-id=GetObject)

  ### Nemotron 3 Super

  NVIDIA

  NVFP4 and FP8 vLLM targets for B200 and H200 chat and agentic traffic with MTP speculative decoding and KV-aware routing.

  vLLM4x B200 / 4x H200Chat + agentic

  [Open recipe](/dynamo/v1.3.0/recipes/nemotron-3-super)

  ![](https://fdr-prod-docs-files-public.s3.us-east-1.amazonaws.com/dynamo.docs.buildwithfern.com/e60742a0b37f805e654249dedb7b6ca11c785a60a2dd9ae33864668dce217693/pages-v1.3.0/assets/img/recipes/providers/zai-org.svg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIA6KXJSKKNFOCF7G4B%2F20260913%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260913T030128Z&X-Amz-Expires=604800&X-Amz-Signature=faff67df69a984add82c3504ad61d9cdf23e30f66b9964396a45e3753ae5897a&X-Amz-SignedHeaders=host&x-amz-checksum-mode=ENABLED&x-id=GetObject)

  ### GLM-5 NVFP4

  NVIDIA / Z.AI

  20x GB200 SGLang P/D recipe for 1K input / 8K output traffic with EAGLE speculative decoding, plus an AWS EFA variant.

  SGLang20x GB200Long output / static ISL-OSL

  [Open recipe](/dynamo/v1.3.0/recipes/glm-5-nvfp4)

  ![](https://fdr-prod-docs-files-public.s3.us-east-1.amazonaws.com/dynamo.docs.buildwithfern.com/4cd5dd62104f4bfa1caa6bfa3a13e06b1577d24ea03bfd1e78e4c5aea8ceed9f/pages-v1.3.0/assets/img/recipes/providers/deepseek-ai.webp?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIA6KXJSKKNFOCF7G4B%2F20260913%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260913T030128Z&X-Amz-Expires=604800&X-Amz-Signature=0297138121e05642a53f663c7de50248771106874819eec2b1146477dfc9cf31&X-Amz-SignedHeaders=host&x-amz-checksum-mode=ENABLED&x-id=GetObject)

  ### DeepSeek-V4-Pro

  NVIDIA checkpoint / DeepSeek

  vLLM agentic recipe — MoE 1.6T / 49B active, B200 (NVFP4, 1M ctx) and H200 (FP8), aggregated or disaggregated with MTP-2 and KV-aware routing.

  vLLM8–32x B200/H200Agentic

  [Open recipe](/dynamo/v1.3.0/recipes/deepseek-v4-pro)

  ![](https://fdr-prod-docs-files-public.s3.us-east-1.amazonaws.com/dynamo.docs.buildwithfern.com/4cd5dd62104f4bfa1caa6bfa3a13e06b1577d24ea03bfd1e78e4c5aea8ceed9f/pages-v1.3.0/assets/img/recipes/providers/deepseek-ai.webp?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIA6KXJSKKNFOCF7G4B%2F20260913%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260913T030128Z&X-Amz-Expires=604800&X-Amz-Signature=0297138121e05642a53f663c7de50248771106874819eec2b1146477dfc9cf31&X-Amz-SignedHeaders=host&x-amz-checksum-mode=ENABLED&x-id=GetObject)

  ### DeepSeek-V4-Flash

  NVIDIA checkpoint / DeepSeek

  vLLM agentic recipe — MoE 284B / 13B active, B200 (NVFP4) and H200 (FP8), aggregated or disaggregated (2P1D / 4P3D) with KV-aware routing.

  vLLM4–28x B200/H200Agentic

  [Open recipe](/dynamo/v1.3.0/recipes/deepseek-v4-flash)

  ![](https://fdr-prod-docs-files-public.s3.us-east-1.amazonaws.com/dynamo.docs.buildwithfern.com/4cd5dd62104f4bfa1caa6bfa3a13e06b1577d24ea03bfd1e78e4c5aea8ceed9f/pages-v1.3.0/assets/img/recipes/providers/deepseek-ai.webp?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIA6KXJSKKNFOCF7G4B%2F20260913%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260913T030128Z&X-Amz-Expires=604800&X-Amz-Signature=0297138121e05642a53f663c7de50248771106874819eec2b1146477dfc9cf31&X-Amz-SignedHeaders=host&x-amz-checksum-mode=ENABLED&x-id=GetObject)

  ### DeepSeek V3.2 NVFP4

  NVIDIA checkpoint / DeepSeek

  32x GB200 TensorRT-LLM recipe with P/D split, KV-aware routing, and WideEP for long-context coding traces.

  TRT-LLM32x GB200Long-context reuseRelated benchmark

  [Open recipe](/dynamo/v1.3.0/recipes/deepseek-v3-2-nvfp4)

  ![](https://fdr-prod-docs-files-public.s3.us-east-1.amazonaws.com/dynamo.docs.buildwithfern.com/51767fd618f19941f3f2ae5a88c526ccc4ca874a0444226f09f6ab6da7346897/pages-v1.3.0/assets/img/recipes/providers/openai.webp?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIA6KXJSKKNFOCF7G4B%2F20260913%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260913T030128Z&X-Amz-Expires=604800&X-Amz-Signature=aa1021dce8a6353151d9633957e9467ffebeb93fbaf852fc78182f12076e115f&X-Amz-SignedHeaders=host&x-amz-checksum-mode=ENABLED&x-id=GetObject)

  ### GPT-OSS-120B

  OpenAI

  TensorRT-LLM GB200 targets for static traffic plus vLLM B200/H200 agentic targets (agg + disagg) with KV-aware routing and EAGLE3 speculative decoding.

  TRT-LLM + vLLMGB200 · B200 · H200Static + agentic

  [Open recipe](/dynamo/v1.3.0/recipes/gpt-oss-120b)

  ![](https://fdr-prod-docs-files-public.s3.us-east-1.amazonaws.com/dynamo.docs.buildwithfern.com/588266b8e1b85bad9da8849973fb14ea59b7422a2b30b9dbf7e7e8db5c246321/pages-v1.3.0/assets/img/recipes/providers/qwen.webp?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIA6KXJSKKNFOCF7G4B%2F20260913%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260913T030128Z&X-Amz-Expires=604800&X-Amz-Signature=bcbe95bccb3944165642c8719f0649ff9a1a89eba4d27b2ba5792f2f45b946ea&X-Amz-SignedHeaders=host&x-amz-checksum-mode=ENABLED&x-id=GetObject)

  ### Qwen3-32B

  Qwen

  Disaggregated vLLM serving with KV-aware routing on 16x H200, for multi-turn conversational traffic with prefix reuse.

  vLLM16x H200Multi-turn conversationRelated benchmark

  [Open recipe](/dynamo/v1.3.0/recipes/qwen3-32b)

  ![](https://fdr-prod-docs-files-public.s3.us-east-1.amazonaws.com/dynamo.docs.buildwithfern.com/588266b8e1b85bad9da8849973fb14ea59b7422a2b30b9dbf7e7e8db5c246321/pages-v1.3.0/assets/img/recipes/providers/qwen.webp?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIA6KXJSKKNFOCF7G4B%2F20260913%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260913T030128Z&X-Amz-Expires=604800&X-Amz-Signature=bcbe95bccb3944165642c8719f0649ff9a1a89eba4d27b2ba5792f2f45b946ea&X-Amz-SignedHeaders=host&x-amz-checksum-mode=ENABLED&x-id=GetObject)

  ### Qwen3-235B-A22B FP8

  Qwen

  16-GPU TensorRT-LLM recipe matrix for Hopper/Blackwell and aggregate/P-D serving.

  TRT-LLM16x H100/H200Static ISL-OSL

  [Open recipe](/dynamo/v1.3.0/recipes/qwen3-235b-a22b-fp8)

  ![](https://fdr-prod-docs-files-public.s3.us-east-1.amazonaws.com/dynamo.docs.buildwithfern.com/588266b8e1b85bad9da8849973fb14ea59b7422a2b30b9dbf7e7e8db5c246321/pages-v1.3.0/assets/img/recipes/providers/qwen.webp?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIA6KXJSKKNFOCF7G4B%2F20260913%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260913T030128Z&X-Amz-Expires=604800&X-Amz-Signature=bcbe95bccb3944165642c8719f0649ff9a1a89eba4d27b2ba5792f2f45b946ea&X-Amz-SignedHeaders=host&x-amz-checksum-mode=ENABLED&x-id=GetObject)

  ### Qwen3-32B FP8

  Qwen

  FP8 recipe set spanning TensorRT-LLM aggregate, TensorRT-LLM P/D, and vLLM P/D targets.

  TRT-LLM2-8x GPUStatic ISL-OSL

  [Open recipe](/dynamo/v1.3.0/recipes/qwen3-32b-fp8)

  ![](https://fdr-prod-docs-files-public.s3.us-east-1.amazonaws.com/dynamo.docs.buildwithfern.com/b202ff7373a9f14706f9cd4afd55cbb4c513e0f53763675dd93552693a623bf8/pages-v1.3.0/assets/img/recipes/providers/meta-llama.webp?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIA6KXJSKKNFOCF7G4B%2F20260913%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260913T030128Z&X-Amz-Expires=604800&X-Amz-Signature=e0acc00b298080a671138104b142a5e39a3d43ddd9884fe2cb0ff957ddf1b40d&X-Amz-SignedHeaders=host&x-amz-checksum-mode=ENABLED&x-id=GetObject)

  ### Llama-3.3-70B FP8

  Meta / Red Hat

  Llama FP8 topology recipe set for 8K input / 1K output traffic on H100/H200.

  vLLM4x H100/H200Long output / static ISL-OSLRelated benchmark

  [Open recipe](/dynamo/v1.3.0/recipes/llama-3-3-70b)

  No recipes match the selected filters.
</form>

## Related Feature Benchmarks

Use [Feature Benchmarks](/dynamo/v1.3.0/benchmarks) when your question starts with a feature or performance claim, such as KV routing, embedding cache, speculative decoding, or topology. Recipe pages link back to Feature Benchmark pages whenever a recipe came from a comparison or feature-stack run.