> For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.nvidia.com/llm-inference-quick-start-recipes/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/llm-inference-quick-start-recipes/_mcp/server. # LLM Inference Quick Start Recipes > Optimized deployment guides for NVIDIA hardware for the most popular open source LLMs. Optimized deployment guides for NVIDIA hardware for the most popular open source LLMs. ## Dynamo Disaggregated Recipes | Model | Dynamo + SGLang | Dynamo + TRT-LLM | Dynamo + vLLM | | --------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------- | | DeepSeek-R1 | [✅](https://github.com/ai-dynamo/dynamo/tree/main/recipes/deepseek-r1/sglang) ([Slurm](https://github.com/ai-dynamo/dynamo/blob/main/components/backends/sglang/slurm_jobs/README.md)) | [✅](https://github.com/ai-dynamo/dynamo/tree/main/recipes/deepseek-r1/trtllm) | [✅](https://github.com/ai-dynamo/dynamo/tree/main/recipes/deepseek-r1/vllm/disagg) | | DeepSeek-V3.2 (FP4) | — | [✅](https://github.com/ai-dynamo/dynamo/tree/main/recipes/deepseek-v32-fp4/trtllm/disagg-kv-router) | — | | DeepSeek-V4-Flash | [✅](https://github.com/ai-dynamo/dynamo/tree/main/recipes/deepseek-v4/deepseek-v4-flash/sglang) | — | [✅](https://github.com/ai-dynamo/dynamo/tree/main/recipes/deepseek-v4/deepseek-v4-flash/vllm) | | DeepSeek-V4-Pro | [✅](https://github.com/ai-dynamo/dynamo/tree/main/recipes/deepseek-v4/deepseek-v4-pro/sglang) | — | [✅](https://github.com/ai-dynamo/dynamo/tree/main/recipes/deepseek-v4/deepseek-v4-pro/vllm) | | GLM-5 (NVFP4) | [✅](https://github.com/ai-dynamo/dynamo/tree/main/recipes/glm-5-nvfp4/sglang/disagg) | — | — | | GPT-OSS | — | [✅](https://github.com/ai-dynamo/dynamo/blob/main/recipes/gpt-oss-120b/README.md) | — | | Kimi-K2.5 | — | [✅](https://github.com/ai-dynamo/dynamo/tree/main/recipes/kimi-k2.5/trtllm) | — | | Llama-3-70B | — | — | [✅](https://github.com/ai-dynamo/dynamo/tree/main/recipes/llama-3-70b/vllm) | | Nemotron-3-Super | [✅](https://github.com/ai-dynamo/dynamo/tree/main/recipes/nemotron-3-super-fp8/sglang) | [✅](https://github.com/ai-dynamo/dynamo/tree/main/recipes/nemotron-3-super-fp8/trtllm) | [✅](https://github.com/ai-dynamo/dynamo/tree/main/recipes/nemotron-3-super-fp8/vllm) | | Nemotron-3-Ultra | — | — | [✅](https://github.com/ai-dynamo/dynamo/tree/main/recipes/nemotron-3-ultra/vllm) | | Qwen3-235B-A22B (FP8) | — | [✅](https://github.com/ai-dynamo/dynamo/tree/main/recipes/qwen3-235b-a22b-fp8/trtllm/disagg) | — | | Qwen3-32B | — | — | [✅](https://github.com/ai-dynamo/dynamo/tree/main/recipes/qwen3-32b/vllm/disagg-kv-router) | | Qwen3-32B (FP8) | — | [✅](https://github.com/ai-dynamo/dynamo/tree/main/recipes/qwen3-32b-fp8/trtllm/disagg) | [✅](https://github.com/ai-dynamo/dynamo/tree/main/recipes/qwen3-32b-fp8/vllm/disagg) | ## Framework Recipes | Model | SGLang | TRT-LLM | vLLM | | --------------------- | ----------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------- | | DeepSeek-R1 | [✅](https://docs.sglang.io/cookbook/autoregressive/DeepSeek/DeepSeek-R1) | [✅ (0528)](https://github.com/NVIDIA/TensorRT-LLM/blob/main/docs/source/deployment-guide/deployment-guide-for-deepseek-r1-on-trtllm.md) | [✅ (R1/V3)](https://recipes.vllm.ai/deepseek-ai/DeepSeek-V3) | | DeepSeek-V3.2 | [✅](https://docs.sglang.io/cookbook/autoregressive/DeepSeek/DeepSeek-V3_2) | — | — | | DeepSeek-V4 | [✅](https://docs.sglang.io/cookbook/autoregressive/DeepSeek/DeepSeek-V4) | — | — | | DeepSeek-V4-Pro | — | — | [✅](https://recipes.vllm.ai/deepseek-ai/DeepSeek-V4-Pro) | | DiffusionGemma | — | — | [✅](https://recipes.vllm.ai/Google/diffusiongemma-26B-A4B-it) | | GLM-4.7 | [✅](https://docs.sglang.io/cookbook/autoregressive/GLM/GLM-4.7) | — | [✅](https://recipes.vllm.ai/zai-org/GLM-4.7) | | GLM-5 | [✅](https://docs.sglang.io/cookbook/autoregressive/GLM/GLM-5) | [✅](https://github.com/NVIDIA/TensorRT-LLM/blob/main/docs/source/deployment-guide/deployment-guide-for-glm-5-on-trtllm.md) | [✅](https://recipes.vllm.ai/zai-org/GLM-5) | | GLM-5.1 | [✅](https://docs.sglang.io/cookbook/autoregressive/GLM/GLM-5.1) | — | [✅](https://recipes.vllm.ai/zai-org/GLM-5.1) | | GLM-5.2 | [✅](https://docs.sglang.io/cookbook/autoregressive/GLM/GLM-5.2) | — | [✅](https://recipes.vllm.ai/zai-org/GLM-5.2) | | GPT-OSS | [✅](https://docs.sglang.io/cookbook/autoregressive/OpenAI/GPT-OSS) | [✅](https://github.com/NVIDIA/TensorRT-LLM/blob/main/docs/source/deployment-guide/deployment-guide-for-gpt-oss-on-trtllm.md) | [✅](https://recipes.vllm.ai/openai/gpt-oss-120b) | | GPT-OSS + Eagle3 | — | [✅](https://github.com/NVIDIA/TensorRT-LLM/blob/main/docs/source/blogs/tech_blog/blog11_GPT_OSS_Eagle3.md) | — | | Kimi-K2-Thinking | — | [✅](https://github.com/NVIDIA/TensorRT-LLM/blob/main/docs/source/deployment-guide/deployment-guide-for-kimi-k2-thinking-on-trtllm.md) | — | | Kimi-K2.5 | [✅](https://docs.sglang.io/cookbook/autoregressive/Moonshotai/Kimi-K2.5) | — | [✅](https://recipes.vllm.ai/moonshotai/Kimi-K2.5) | | Kimi-K2.6 | [✅](https://docs.sglang.io/cookbook/autoregressive/Moonshotai/Kimi-K2.6) | — | [✅](https://recipes.vllm.ai/moonshotai/Kimi-K2.6) | | Kimi-K2.7-Code | [✅](https://docs.sglang.io/cookbook/autoregressive/Moonshotai/Kimi-K2.7-Code) | — | [✅](https://recipes.vllm.ai/moonshotai/Kimi-K2.7-Code) | | Llama-3.3-70B | [✅](https://docs.sglang.io/cookbook/autoregressive/Llama/Llama3.3-70B) | [✅](https://github.com/NVIDIA/TensorRT-LLM/blob/main/docs/source/deployment-guide/deployment-guide-for-llama3.3-70b-on-trtllm.md) | [✅](https://recipes.vllm.ai/meta-llama/Llama-3.3-70B-Instruct) | | Llama-4-Scout | — | [✅](https://github.com/NVIDIA/TensorRT-LLM/blob/main/docs/source/deployment-guide/deployment-guide-for-llama4-scout-on-trtllm.md) | [✅](https://recipes.vllm.ai/meta-llama/Llama-4-Scout-17B-16E-Instruct) | | MiniMax-M2 | — | — | [✅](https://recipes.vllm.ai/MiniMaxAI/MiniMax-M2) | | MiniMax-M2.5 | [✅](https://docs.sglang.io/cookbook/autoregressive/MiniMax/MiniMax-M2.5) | — | [✅](https://recipes.vllm.ai/MiniMaxAI/MiniMax-M2.5) | | MiniMax-M3 | [✅](https://docs.sglang.io/cookbook/autoregressive/MiniMax/MiniMax-M3) | [✅](https://github.com/NVIDIA/TensorRT-LLM/blob/main/docs/source/deployment-guide/deployment-guide-for-minimax-m3-on-trtllm.md) | [✅](https://recipes.vllm.ai/MiniMaxAI/MiniMax-M3) | | Nemotron-3-Nano | [✅](https://docs.sglang.io/cookbook/autoregressive/NVIDIA/Nemotron3-Nano) | [✅](https://github.com/byshiue/TensorRT-LLM/blob/main/examples/models/core/nemotron/README_nemotron_nano_v3.md) | [✅](https://recipes.vllm.ai/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16) | | Nemotron-3-Super | [✅](https://docs.sglang.io/cookbook/autoregressive/NVIDIA/Nemotron3-Super) | [✅](https://github.com/NVIDIA/TensorRT-LLM/blob/main/docs/source/deployment-guide/deployment-guide-for-nemotron-3-on-trtllm.md) | [✅](https://recipes.vllm.ai/nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16) | | Nemotron-3-Ultra | [✅](https://docs.sglang.io/cookbook/autoregressive/NVIDIA/Nemotron3-Ultra) | — | [✅](https://recipes.vllm.ai/nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16) | | Qwen | — | [✅](https://github.com/byshiue/TensorRT-LLM/tree/main/examples/models/core/qwen) | — | | Qwen3 | — | [✅](https://github.com/NVIDIA/TensorRT-LLM/blob/main/docs/source/deployment-guide/deployment-guide-for-qwen3-on-trtllm.md) | — | | Qwen3-Coder-480B-A35B | — | — | [✅](https://recipes.vllm.ai/Qwen/Qwen3-Coder-480B-A35B-Instruct) | | Qwen3-Next | [✅](https://docs.sglang.io/cookbook/autoregressive/Qwen/Qwen3-Next) | [✅](https://nvidia.github.io/TensorRT-LLM/deployment-guide/quick-start-recipe-for-qwen3-next-on-trtllm.html) | — | | Qwen3.5 | [✅](https://docs.sglang.io/cookbook/autoregressive/Qwen/Qwen3.5) | [✅](https://github.com/NVIDIA/TensorRT-LLM/blob/main/docs/source/deployment-guide/deployment-guide-for-qwen3.5-on-trtllm.md) | [✅](https://recipes.vllm.ai/Qwen/Qwen3.5-397B-A17B) |