Troubleshooting Tool Calls

When a tool call comes back wrong (tool_calls is null, the arguments look malformed, raw <tool_call> markers appear in message.content, or finish_reason is "stop" when you expected "tool_calls"), the request and response alone usually do not say where the bug is. The model and the parser produce indistinguishable failures from the response side.

Adding "logprobs": true to a single repro request makes the engine’s raw token output visible in the response. That is enough for someone on the Dynamo team to identify whether the issue is in the model, the parser configuration, or the parser itself. This page shows the field to add and what the response will look like, so you can capture and share useful diagnostic info.

The request

Add "logprobs": true to your failing request:

$
curl -s http://localhost:8000/v1/chat/completions \
>
  -H 'Content-Type: application/json' \
>
  -d '{
>
    "model": "Qwen/Qwen2.5-7B-Instruct",
>
    "messages": [
>
      {"role": "user", "content": "What is the weather in NYC?"}
>
    ],
>
    "tools": [{
>
      "type": "function",
>
      "function": {
>
        "name": "get_weather",
>
        "parameters": {
>
          "type": "object",
>
          "properties": {
>
            "location": {"type": "string"},
>
            "unit": {"enum": ["celsius", "fahrenheit"]}
>
          },
>
          "required": ["location"]
>
        }
>
      }
>
    }],
>
    "tool_choice": "auto",
>
    "temperature": 0.0,
>
    "logprobs": true
>
  }'

The response

You will get back the usual fields (message.tool_calls, message.content, finish_reason) plus a new choices[0].logprobs.content field carrying the engine’s raw token stream:

1
{
2
  "choices": [{
3
    "finish_reason": "tool_calls",
4
    "message": {
5
      "role": "assistant",
6
      "content": null,
7
      "tool_calls": [{
8
        "type": "function",
9
        "function": {
10
          "name": "get_weather",
11
          "arguments": "{\"location\":\"New York, NY\",\"unit\":\"fahrenheit\"}"
12
        }
13
      }]
14
    },
15
    "logprobs": {
16
      "content": [
17
        {"token": "<tool_call>", "bytes": [60, 116, 111, 111, 108, 95, 99, 97, 108, 108, 62]},
18
        {"token": "\n", "bytes": [10]},
19
        {"token": "{\"", "bytes": [123, 34]},
20
        {"token": "name", "bytes": [110, 97, 109, 101]},
21
        "...",
22
        {"token": "</tool_call>", "bytes": [60, 47, 116, 111, 111, 108, 95, 99, 97, 108, 108, 62]}
23
      ]
24
    }
25
  }]
26
}

Each entry in logprobs.content is one generated token with its exact UTF-8 bytes. Concatenating those bytes in order reconstructs the raw model output, before any tool-call parser touched it. That is the key piece for triage: it tells us what the model actually produced, separately from what the parser made of it.

What to include when reporting an issue

Share these four things in the bug report or issue thread:

1. The full request body (model name, messages, tools, sampling params, and logprobs: true).
2. The full response body. Do not truncate logprobs.content — the per-token entries are the part that matters.
3. The Dynamo version and the backend (vLLM, SGLang, TRT-LLM, including versions if known).
4. The worker launch command, especially the --dyn-tool-call-parser value if set.

With those four pieces, the Dynamo team can usually localize the bug without standing up your model. The team will reconstruct the raw stream from the bytes arrays and compare it against message.content and message.tool_calls to decide whether the issue is in the model output, the parser configuration, or the parser logic.

See also

• Tool Call Parsing (Dynamo) — Dynamo-native parser names and request examples
• Tool Call Parsing (Engine Fallback) — --dyn-chat-processor fallback path to vLLM and SGLang parsers
• Frontend Configuration Reference — full CLI flag reference for the frontend and worker