Tool Call Parsing (Engine Fallback)
Tool Call Parsing (Engine Fallback)
Use upstream vLLM or SGLang tool-call parsers when Dynamo does not ship one
DocumentationTool Call Parsing (Engine Fallback)
Use upstream vLLM or SGLang tool-call parsers when Dynamo does not ship one
When Dynamo’s registry does not list a tool-call parser for your model, fall back to the upstream engine’s parser via a chat-processor swap, which keeps frontend tokenization and KV routing.
For Dynamo-native parsers, see Tool Call Parsing (Dynamo). For the equivalent reasoning fallback, see Reasoning Parsing (Engine Fallback).
Known Issue: Engine-fallback tool call parsing does not currently work with disaggregated serving (support coming soon). Use the Dynamo-native tool call parser for disaggregated deployments today.
--dyn-tool-call-parser selects the Dynamo-native parser path, while
--tool-call-parser selects the engine fallback (vLLM or SGLang)
parser path. The accepted values for each flag come from a different
registry and may differ slightly based on the definitions from each
framework (e.g., SGLang’s deepseekv3 vs Dynamo’s deepseek_v3).
If a tool call comes back wrong, add "logprobs": true to a single repro
request and share the response. See
Troubleshooting Tool Calls for what to capture and
include when reporting an issue.
logprobs so tool-call issues can be localized