Troubleshooting Tool Calls

When a tool call comes back wrong (tool_calls is null, the arguments look malformed, raw <tool_call> markers appear in message.content, or finish_reason is "stop" when you expected "tool_calls"), the request and response alone usually do not say where the bug is. The model and the parser produce indistinguishable failures from the response side.

Adding "logprobs": true to a single repro request makes the engine’s raw token output visible in the response. That is enough for someone on the Dynamo team to identify whether the issue is in the model, the parser configuration, or the parser itself. This page shows the field to add and what the response will look like, so you can capture and share useful diagnostic info.

The request

Add "logprobs": true to your failing request. Port-forward the Frontend Service first (kubectl port-forward svc/<deployment-name>-frontend 8000:8000 -n ${NAMESPACE}):

curl -s http://localhost:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "Qwen/Qwen2.5-7B-Instruct",
    "messages": [
      {"role": "user", "content": "What is the weather in NYC?"}
    ],
    "tools": [{
      "type": "function",
      "function": {
        "name": "get_weather",
        "parameters": {
          "type": "object",
          "properties": {
            "location": {"type": "string"},
            "unit": {"enum": ["celsius", "fahrenheit"]}
          },
          "required": ["location"]
        }
      }
    }],
    "tool_choice": "auto",
    "temperature": 0.0,
    "logprobs": true
  }'

The response

You will get back the usual fields (message.tool_calls, message.content, finish_reason) plus a new choices[0].logprobs.content field carrying the engine’s raw token stream:

{
  "choices": [{
    "finish_reason": "tool_calls",
    "message": {
      "role": "assistant",
      "content": null,
      "tool_calls": [{
        "type": "function",
        "function": {
          "name": "get_weather",
          "arguments": "{\"location\":\"New York, NY\",\"unit\":\"fahrenheit\"}"
        }
      }]
    },
    "logprobs": {
      "content": [
        {"token": "<tool_call>", "bytes": [60, 116, 111, 111, 108, 95, 99, 97, 108, 108, 62]},
        {"token": "\n", "bytes": [10]},
        {"token": "{\"", "bytes": [123, 34]},
        {"token": "name", "bytes": [110, 97, 109, 101]},
        "...",
        {"token": "</tool_call>", "bytes": [60, 47, 116, 111, 111, 108, 95, 99, 97, 108, 108, 62]}
      ]
    }
  }]
}

Each entry in logprobs.content is one generated token and keeps the score the backend returned for it. The token text and bytes are decoded representations: an end-of-sequence token can appear as its literal text and bytes, such as <|im_end|> with --dyn-chat-processor sglang, or as empty text and no bytes, and return_tokens_as_token_ids can replace the text with a token_id:<id> label. Without that option, concatenating the bytes in order approximates the model output before any tool-call parser touched it. It can include control-token literals such as a trailing <|im_end|>, omits tokens that decode to nothing, and, because each token can be decoded on its own, can lose spacing that the tokenizer adds only in context. That is the key piece for triage: it tells us what the model actually produced, separately from what the parser made of it. For an exact replay, also request the generated token IDs with "nvext": {"extra_fields": ["completion_token_ids"]} where the response supports them (see NVIDIA Request Extensions).

What to include when reporting an issue

Share these four things in the bug report or issue thread:

1. The full request body (model name, messages, tools, sampling params, and logprobs: true).
2. The full response body. Do not truncate logprobs.content — the per-token entries are the part that matters.
3. The Dynamo version and the backend (vLLM, SGLang, TRT-LLM, including versions if known).
4. The worker launch command, especially the --dyn-tool-call-parser value if set.

With those four pieces, the Dynamo team can usually localize the bug without standing up your model. The team will rebuild the raw stream from the per-token entries, using exact token IDs over decoded bytes where the response provides them, and compare it against message.content and message.tool_calls to decide whether the issue is in the model output, the parser configuration, or the parser logic.

Parser Generation Routing

Set DYN_PARSER_VERSION on the frontend running the dynamo processor and on workers that build parser-dependent guidance. Workers publish --dyn-tool-call-parser and --dyn-reasoning-parser; the frontend runs parsing. Restart after changing the environment because version selection is cached. For release-specific parser versions, inspect Cargo.toml and Cargo.lock in your release checkout. The development Cargo.toml and Cargo.lock links track main. The frontend-crates library examples may target a newer release.

Only auto, 1, and 2 are accepted; v1 and v2 stop startup. Invalid combinations are rejected during model setup with an error naming the configured selectors and the missing implementation. Runtime V2 parse errors never retry through V1. On V1 routes, an empty tool selector is treated as unset; an empty reasoning selector is rejected. Dynamo requires exact, case-sensitive V1 selector names. Omit selectors you do not need.

For token-input chat using a Python chat processor, the processor owns both parser registries, so Dynamo does not validate those names against its Rust registries. Parser-generation environment values are still validated. Completions do not use chat parsers; text-input chat retains Rust selector validation.

V2 uses one family grammar for reasoning, text, and tools. Either selector can usually choose that family; when both are set, they must resolve to the same family or a supported shared grammar. Auto requires both selectors to be beam for Beam. V1 uses separate reasoning and tool-call parsers, so it can mix formats when both names exist in their respective V1 registries. This table lists parser availability, not model/backend or GPU qualification.

Examples below list the tool-call selector first, then the reasoning selector:

• qwen3_coder + qwen3, glm47 + glm45, and kimi_k2 + kimi_k25 each resolve to one Unified family, so auto uses V2. qwen3_coder + nemotron_v3 also uses V2 through Nemotron’s shared Qwen XML tool grammar.
• hermes + qwen3 and qwen3_coder + deepseek_v4 mix different formats. Auto uses the V1 split pipeline; explicit 2 rejects them. Use 1 when you want to keep a valid V1 combination on V1.
• muse_glimmer + qwen3 is rejected: it has no matching Unified family and V1 has no muse_glimmer tool parser. With auto or 2, use muse_glimmer in both selectors, or configure only one selector, to select its Unified grammar.

A single tool selector such as qwen3_coder, or reasoning selector such as deepseek_v41, selects the whole Unified family grammar. The serving layer still enforces tool policy. Family aliases accepted by V2 are not necessarily registered in the corresponding V1 slot.

Reasoning-only exceptions: without a tool selector, auto stays on V1 for qwen3, nemotron_v3, or nemotron3 reasoning. Qwen thinking/QwQ models can emit Hermes JSON tools rather than Coder XML; a Nemotron reasoning name alone does not identify its tool grammar. Explicit 2 selects the corresponding Unified grammar; use it only for matching output.

MiniMax M3: auto uses Unified for compatible M3 selectors. Use 1 to select legacy parsing.

Deprecated setting: replace DYN_ENABLE_EXPERIMENTAL_PARSERS_V2 with DYN_PARSER_VERSION. Legacy true means explicit V2; false means auto, which now prefers compatible V2. Conflicting pairs (true + 1, false + 2) and malformed booleans reject. For rollback, unset the old variable and use 1 if both selectors exist in V1. Muse Glimmer, Muse Spark, Beam, and DeepSeek V4.1 are Unified-only.

Unified emits reasoning, text, and tools in source order. V1 extracts reasoning first and may buffer calls in the tool jail. Streaming exposes incremental output where the family and request mode support it; batch aggregates output into content, reasoning_content, and tool_calls. Both use the same selected generation. With tool_choice: none, Unified suppresses calls and decodes recognized markup. V1 bypasses the tool jail for parsers such as qwen3_coder, so raw markup may remain in content.

Unified resolves prompt-prefilled reasoning after template rendering: an opener in the prompt need not recur in generated text. Kimi K3’s <|open|>think<|sep|> suffix, including supported whitespace between markers, starts reasoning. This is separate from force-reasoning defaults and engine grammar gates.

For MiniMax M3, explicit guided_json, guided_regex, guided_choice, or guided_grammar disables thinking by default when no explicit thinking control is supplied. Explicit-guided requests that enable reasoning require --dyn-reasoning-parser minimax_m3; a tool parser alone does not activate guided-reasoning support. When the rendered prompt ends in <mm:think>, both parser routes start in reasoning: reasoning streams before </mm:think> arrives and remains reasoning_content if truncated. Configure dynamo.vllm with --reasoning-parser minimax_m3, or dynamo.sglang with --reasoning-parser minimax-m3, to defer the answer grammar until reasoning ends; dynamo.frontend also sends SGLang’s require_reasoning signal for these prompt-open requests. The frontend does not infer grammar timing from generated text. Adaptive/M3.1 templates that leave reasoning unopened receive no additional gate.

Forced choices can install guided JSON or family-native structural tags. A mode requiring the V1 jail cannot silently fall back under explicit V2; unsupported combinations are rejected. See Structural Tags for builder availability and backend activation.

Known limitations and follow-up

The pinned V2 release includes the quoted-control and streaming fixes from frontend-crates #326 and Kimi tool-ID and call-boundary fixes from #332.

Two cases remain unqualified: literal native tool-call openers inside quoted content, and V1 prompt prefill when </think> splits after its first <. Existing reasoning-marker tests do not cover them; see Dynamo #15577.

For configurations supported by V1, select DYN_PARSER_VERSION=1 to use the legacy parser.

See also

• Tool Call Parsing (Dynamo) — Dynamo-native parser names and request examples
• Chat Processors — --dyn-chat-processor and the engine-fallback path to vLLM and SGLang parsers
• Frontend Configuration Reference — full CLI flag reference for the frontend and worker