User GuidesTool Calling

Tool Call Parsing (Engine Fallback)

Use upstream vLLM or SGLang tool-call parsers when Dynamo does not ship one

View as Markdown

When Dynamo’s registry does not list a tool-call parser for your model, fall back to the upstream engine’s parser via a chat-processor swap, which keeps frontend tokenization and KV routing.

For Dynamo-native parsers, see Tool Call Parsing (Dynamo). For the equivalent reasoning fallback, see Reasoning Parsing (Engine Fallback).

Known Issue: Engine-fallback tool call parsing does not currently work with disaggregated serving (support coming soon). Use the Dynamo-native tool call parser for disaggregated deployments today.

Configurations

Frontend flagsWorker flagsKV routingNotes
vLLM chat processor--dyn-chat-processor vllm --tool-call-parser <name>(none)YesParsing runs in vLLM’s Python preprocessor. See vLLM Chat Processor.
SGLang chat processor--dyn-chat-processor sglang --tool-call-parser <name>(none)YesParsing runs in SGLang’s Python preprocessor. See SGLang Chat Processor.
TRTLLM chat processor(work in progress)(work in progress)—Engine-fallback support for TRTLLM is in progress. Use the Dynamo-native tool call parser for TRTLLM today.

--dyn-tool-call-parser selects the Dynamo-native parser path, while --tool-call-parser selects the engine fallback (vLLM or SGLang) parser path. The accepted values for each flag come from a different registry and may differ slightly based on the definitions from each framework (e.g., SGLang’s deepseekv3 vs Dynamo’s deepseek_v3).

Examples

$# vLLM chat processor
$python -m dynamo.vllm ...
$python -m dynamo.frontend --dyn-chat-processor vllm --tool-call-parser hermes
$
$# SGLang chat processor
$python -m dynamo.sglang ...
$python -m dynamo.frontend --dyn-chat-processor sglang --tool-call-parser kimi_k2

If a tool call comes back wrong, add "logprobs": true to a single repro request and share the response. See Troubleshooting Tool Calls for what to capture and include when reporting an issue.

See Also