> This page is for version dev. > For other versions, use one of these documentation indexes: > - Latest (v1.6.1) (default): https://docs.nvidia.com/dynamo/latest/llms.txt > - dev: https://docs.nvidia.com/dynamo/dev/llms.txt > - v1.6.1: https://docs.nvidia.com/dynamo/v1.6.1/llms.txt > - v1.6.0: https://docs.nvidia.com/dynamo/v1.6.0/llms.txt > - v1.5.1: https://docs.nvidia.com/dynamo/v1.5.1/llms.txt > - v1.5.0: https://docs.nvidia.com/dynamo/v1.5.0/llms.txt > - v1.4.2: https://docs.nvidia.com/dynamo/v1.4.2/llms.txt > - v1.4.1: https://docs.nvidia.com/dynamo/v1.4.1/llms.txt > - v1.4.0: https://docs.nvidia.com/dynamo/v1.4.0/llms.txt > - v1.3.0: https://docs.nvidia.com/dynamo/v1.3.0/llms.txt > - v1.2.1: https://docs.nvidia.com/dynamo/v1.2.1/llms.txt > - v1.2.0: https://docs.nvidia.com/dynamo/v1.2.0/llms.txt > - v1.1.1: https://docs.nvidia.com/dynamo/v1.1.1/llms.txt > - v1.1.0: https://docs.nvidia.com/dynamo/v1.1.0/llms.txt > - v1.0.2: https://docs.nvidia.com/dynamo/v1.0.2/llms.txt > - v1.0.1: https://docs.nvidia.com/dynamo/v1.0.1/llms.txt > - v1.0.0: https://docs.nvidia.com/dynamo/v1.0.0/llms.txt > - v0.9.1: https://docs.nvidia.com/dynamo/v-0-9-1/llms.txt > - v0.9.0: https://docs.nvidia.com/dynamo/v-0-9-0/llms.txt > - v0.8.1: https://docs.nvidia.com/dynamo/v-0-8-1/llms.txt > - v0.8.0: https://docs.nvidia.com/dynamo/v-0-8-0/llms.txt > - v0.7.1: https://docs.nvidia.com/dynamo/v-0-7-1/llms.txt > - v0.7.0: https://docs.nvidia.com/dynamo/v-0-7-0/llms.txt > For clean Markdown content of this page, append .md to this URL. For the complete documentation index, see https://docs.nvidia.com/dynamo/llms.txt. For full content including API reference and SDK examples, see https://docs.nvidia.com/dynamo/llms-full.txt. # Troubleshooting Tool Calls When a tool call comes back wrong (`tool_calls` is `null`, the arguments look malformed, raw `` markers appear in `message.content`, or `finish_reason` is `"stop"` when you expected `"tool_calls"`), the request and response alone usually do not say *where* the bug is. The model and the parser produce indistinguishable failures from the response side. Adding `"logprobs": true` to a single repro request makes the engine's raw token output visible in the response. That is enough for someone on the Dynamo team to identify whether the issue is in the model, the parser configuration, or the parser itself. This page shows the field to add and what the response will look like, so you can capture and share useful diagnostic info. Recipe applies to non-streaming requests against Dynamo's OpenAI `/v1/chat/completions` endpoint. On the V2 parser route (see [Parser Generation Routing](#parser-generation-routing)), `logprobs.content` lists every token entry the backend returns, in generation order, including reasoning, tool-call, and control tokens that the parsed message does not show. Streamed responses carry each backend chunk's entries once, on one chunk for that choice, so the entries need not line up with the text in `reasoning_content`, `content`, and `tool_calls`. This note does not describe the V1 route, where late entries can arrive in a final empty-`delta` chunk after `finish_reason`. ## The request Add `"logprobs": true` to your failing request. Port-forward the Frontend Service first (`kubectl port-forward svc/-frontend 8000:8000 -n ${NAMESPACE}`): ```bash curl -s http://localhost:8000/v1/chat/completions \ -H 'Content-Type: application/json' \ -d '{ "model": "Qwen/Qwen2.5-7B-Instruct", "messages": [ {"role": "user", "content": "What is the weather in NYC?"} ], "tools": [{ "type": "function", "function": { "name": "get_weather", "parameters": { "type": "object", "properties": { "location": {"type": "string"}, "unit": {"enum": ["celsius", "fahrenheit"]} }, "required": ["location"] } } }], "tool_choice": "auto", "temperature": 0.0, "logprobs": true }' ``` ## The response You will get back the usual fields (`message.tool_calls`, `message.content`, `finish_reason`) plus a new `choices[0].logprobs.content` field carrying the engine's raw token stream: ```json { "choices": [{ "finish_reason": "tool_calls", "message": { "role": "assistant", "content": null, "tool_calls": [{ "type": "function", "function": { "name": "get_weather", "arguments": "{\"location\":\"New York, NY\",\"unit\":\"fahrenheit\"}" } }] }, "logprobs": { "content": [ {"token": "", "bytes": [60, 116, 111, 111, 108, 95, 99, 97, 108, 108, 62]}, {"token": "\n", "bytes": [10]}, {"token": "{\"", "bytes": [123, 34]}, {"token": "name", "bytes": [110, 97, 109, 101]}, "...", {"token": "", "bytes": [60, 47, 116, 111, 111, 108, 95, 99, 97, 108, 108, 62]} ] } }] } ``` Each entry in `logprobs.content` is one generated token and keeps the score the backend returned for it. The `token` text and `bytes` are decoded representations: an end-of-sequence token can appear as its literal text and bytes, such as `<|im_end|>` with `--dyn-chat-processor sglang`, or as empty text and no bytes, and `return_tokens_as_token_ids` can replace the text with a `token_id:` label. Without that option, concatenating the `bytes` in order approximates the model output before any tool-call parser touched it. It can include control-token literals such as a trailing `<|im_end|>`, omits tokens that decode to nothing, and, because each token can be decoded on its own, can lose spacing that the tokenizer adds only in context. That is the key piece for triage: it tells us what the model actually produced, separately from what the parser made of it. For an exact replay, also request the generated token IDs with `"nvext": {"extra_fields": ["completion_token_ids"]}` where the response supports them (see [NVIDIA Request Extensions](/dynamo/dev/additional-resources/nvidia-request-extensions-nvext)). ## What to include when reporting an issue Share these four things in the bug report or issue thread: 1. **The full request body** (model name, messages, tools, sampling params, and `logprobs: true`). 2. **The full response body.** Do not truncate `logprobs.content` -- the per-token entries are the part that matters. 3. **The Dynamo version and the backend** (vLLM, SGLang, TRT-LLM, including versions if known). 4. **The worker launch command**, especially the `--dyn-tool-call-parser` value if set. With those four pieces, the Dynamo team can usually localize the bug without standing up your model. The team will rebuild the raw stream from the per-token entries, using exact token IDs over decoded `bytes` where the response provides them, and compare it against `message.content` and `message.tool_calls` to decide whether the issue is in the model output, the parser configuration, or the parser logic. ## Parser Generation Routing Set `DYN_PARSER_VERSION` on the frontend running the `dynamo` processor and on workers that build parser-dependent guidance. Workers publish `--dyn-tool-call-parser` and `--dyn-reasoning-parser`; the frontend runs parsing. Restart after changing the environment because version selection is cached. For release-specific parser versions, inspect `Cargo.toml` and `Cargo.lock` in your release checkout. The development [Cargo.toml](https://github.com/ai-dynamo/dynamo/blob/main/Cargo.toml) and [Cargo.lock](https://github.com/ai-dynamo/dynamo/blob/main/Cargo.lock) links track `main`. The frontend-crates library examples may target a newer release. | `DYN_PARSER_VERSION` | Compatible Unified configuration | Unavailable or incompatible configuration | | --- | --- | --- | | Unset or `auto` | V2 | V1 if each configured name exists in its V1 registry; otherwise reject | | `1` | V1 if each configured name exists in its V1 registry | Reject names unavailable in V1 | | `2` | V2 | Reject; no V1 configuration fallback | Only `auto`, `1`, and `2` are accepted; `v1` and `v2` stop startup. Invalid combinations are rejected during model setup with an error naming the configured selectors and the missing implementation. Runtime V2 parse errors never retry through V1. On V1 routes, an empty tool selector is treated as unset; an empty reasoning selector is rejected. Dynamo requires exact, case-sensitive V1 selector names. Omit selectors you do not need. For token-input chat using a [Python chat processor](/dynamo/dev/knowledge-base/modular-components/frontend/tool-calling/chat-processors), the processor owns both parser registries, so Dynamo does not validate those names against its Rust registries. Parser-generation environment values are still validated. Completions do not use chat parsers; text-input chat retains Rust selector validation. V2 uses one family grammar for reasoning, text, and tools. Either selector can usually choose that family; when both are set, they must resolve to the same family or a supported shared grammar. Auto requires both selectors to be `beam` for Beam. V1 uses separate reasoning and tool-call parsers, so it can mix formats when both names exist in their respective V1 registries. This table lists parser availability, not model/backend or GPU qualification. | Unified family | Accepted family names and Dynamo aliases | | --- | --- | | DeepSeek V4 | `deepseek_v4`, `deepseek-v4`, `deepseekv4` | | DeepSeek V4.1 | `deepseek_v41` | | Gemma 4 | `gemma4`, `gemma-4` | | GLM 4.7 | `glm47`, `glm45` | | Kimi K2 / K2.5 | `kimi_k2`, `kimi_k25` | | Kimi K3 | `kimi_k3`, `kimi-k3` | | MiniMax M3 | `minimax_m3`, `minimax-m3` | | Qwen3 Coder | `qwen3`, `qwen3_coder` | | Muse Glimmer | `muse_glimmer`, `muse` | | Muse Spark | `muse_spark`, `muse_spark_open` | | Nemotron V3 | `nemotron_v3`, `nemotron3` | | Beam | `beam` | | GPT-OSS / Harmony | `gpt_oss`, `harmony` | Examples below list the tool-call selector first, then the reasoning selector: - `qwen3_coder` + `qwen3`, `glm47` + `glm45`, and `kimi_k2` + `kimi_k25` each resolve to one Unified family, so auto uses V2. `qwen3_coder` + `nemotron_v3` also uses V2 through Nemotron’s shared Qwen XML tool grammar. - `hermes` + `qwen3` and `qwen3_coder` + `deepseek_v4` mix different formats. Auto uses the V1 split pipeline; explicit `2` rejects them. Use `1` when you want to keep a valid V1 combination on V1. - `muse_glimmer` + `qwen3` is rejected: it has no matching Unified family and V1 has no `muse_glimmer` tool parser. With `auto` or `2`, use `muse_glimmer` in both selectors, or configure only one selector, to select its Unified grammar. A single tool selector such as `qwen3_coder`, or reasoning selector such as `deepseek_v41`, selects the whole Unified family grammar. The serving layer still enforces tool policy. Family aliases accepted by V2 are not necessarily registered in the corresponding V1 slot. **Reasoning-only exceptions:** without a tool selector, auto stays on V1 for `qwen3`, `nemotron_v3`, or `nemotron3` reasoning. Qwen thinking/QwQ models can emit Hermes JSON tools rather than Coder XML; a Nemotron reasoning name alone does not identify its tool grammar. Explicit `2` selects the corresponding Unified grammar; use it only for matching output. **MiniMax M3:** auto uses Unified for compatible M3 selectors. Use `1` to select legacy parsing. **Deprecated setting:** replace `DYN_ENABLE_EXPERIMENTAL_PARSERS_V2` with `DYN_PARSER_VERSION`. Legacy `true` means explicit V2; `false` means auto, which now prefers compatible V2. Conflicting pairs (`true` + `1`, `false` + `2`) and malformed booleans reject. For rollback, unset the old variable and use `1` if both selectors exist in V1. Muse Glimmer, Muse Spark, Beam, and DeepSeek V4.1 are Unified-only. Unified emits reasoning, text, and tools in source order. V1 extracts reasoning first and may buffer calls in the tool jail. Streaming exposes incremental output where the family and request mode support it; batch aggregates output into `content`, `reasoning_content`, and `tool_calls`. Both use the same selected generation. With `tool_choice: none`, Unified suppresses calls and decodes recognized markup. V1 bypasses the tool jail for parsers such as `qwen3_coder`, so raw markup may remain in `content`. Unified resolves prompt-prefilled reasoning after template rendering: an opener in the prompt need not recur in generated text. Kimi K3's `<|open|>think<|sep|>` suffix, including supported whitespace between markers, starts reasoning. This is separate from force-reasoning defaults and engine grammar gates. For MiniMax M3, explicit `guided_json`, `guided_regex`, `guided_choice`, or `guided_grammar` disables thinking by default when no explicit thinking control is supplied. Explicit-guided requests that enable reasoning require `--dyn-reasoning-parser minimax_m3`; a tool parser alone does not activate guided-reasoning support. When the rendered prompt ends in ``, both parser routes start in reasoning: reasoning streams before `` arrives and remains `reasoning_content` if truncated. Configure dynamo.vllm with `--reasoning-parser minimax_m3`, or dynamo.sglang with `--reasoning-parser minimax-m3`, to defer the answer grammar until reasoning ends; dynamo.frontend also sends SGLang's `require_reasoning` signal for these prompt-open requests. The frontend does not infer grammar timing from generated text. Adaptive/M3.1 templates that leave reasoning unopened receive no additional gate. Forced choices can install guided JSON or family-native structural tags. A mode requiring the V1 jail cannot silently fall back under explicit V2; unsupported combinations are rejected. See [Structural Tags](/dynamo/dev/knowledge-base/modular-components/frontend/tool-calling/structural-tag) for builder availability and backend activation. ## Known limitations and follow-up The pinned V2 release includes the quoted-control and streaming fixes from [frontend-crates #326](https://github.com/ai-dynamo/frontend-crates/pull/326) and Kimi tool-ID and call-boundary fixes from [#332](https://github.com/ai-dynamo/frontend-crates/pull/332). Two cases remain unqualified: literal native tool-call openers inside quoted content, and V1 prompt prefill when `` splits after its first `<`. Existing reasoning-marker tests do not cover them; see [Dynamo #15577](https://github.com/ai-dynamo/dynamo/pull/15577). For configurations supported by V1, select `DYN_PARSER_VERSION=1` to use the legacy parser. ## See also - [Tool Call Parsing (Dynamo)](/dynamo/dev/knowledge-base/modular-components/frontend/tool-calling/tool-call-parsing) -- Dynamo-native parser names and request examples - [Chat Processors](/dynamo/dev/knowledge-base/modular-components/frontend/tool-calling/chat-processors) -- `--dyn-chat-processor` and the engine-fallback path to vLLM and SGLang parsers - [Frontend Configuration Reference](/dynamo/dev/reference/components/frontend-configuration) -- full CLI flag reference for the frontend and worker