Skip to navigation
Knowledge BaseModular ComponentsFrontendTool Calling & Reasoning

Tool Call Parsing (Dynamo)

Connect Dynamo to external tools and services using Dynamo’s built-in tool call parsers

View as Markdown

Tool calling lets Dynamo connect models to external tools and services by returning function arguments for your application to execute. Requests use the tool_choice and tools parameters. This page covers Dynamo-native parsing; if your model is not listed, use engine fallback, which also explains valid combinations of --dyn-tool-call-parser, --dyn-chat-processor, and --dyn-reasoning-parser.

See Parser Generation Routing for version selection, compatible names, and the Qwen exception.

Configure Tool Calling

1

Launch Dynamo Backend

Select a --dyn-tool-call-parser that matches your model based on the list below. The backend can be SGLang, TensorRT-LLM, vLLM, or another installed backend.

To inspect the available worker flags, run:

python -m dynamo.<backend> --help

The following example starts an SGLang worker for Qwen3.5:

python -m dynamo.sglang \
  --model Qwen/Qwen3.5-4B \
  --dyn-tool-call-parser qwen3_coder \
  --reasoning-parser qwen3 \
  --dyn-reasoning-parser qwen3

SGLang’s --reasoning-parser qwen3 controls grammar activation; --dyn-reasoning-parser qwen3 selects Dynamo extraction into reasoning_content.

If the model supports tool calling but its default chat template does not, add --custom-jinja-template /path/to/template.jinja to the worker command. The template must be readable inside the container.

If the model emits reasoning content that should be separated from normal output, see Reasoning Parsing (Dynamo) for supported --dyn-reasoning-parser values.

2

Start the Frontend

Start the Dynamo frontend in another terminal:

python -m dynamo.frontend
3

Wait for Readiness

Wait for the frontend health endpoint to return a successful response:

curl --fail http://localhost:8000/health
4

Send a Tool-Calling Request

Send an OpenAI-compatible chat completion request with the available tools:

curl -s http://localhost:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "Qwen/Qwen3.5-4B",
    "messages": [
      {"role": "user", "content": "What is the weather in San Francisco and New York?"}
    ],
    "tools": [{
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Get the current weather for a location.",
        "parameters": {
          "type": "object",
          "properties": {"location": {"type": "string"}},
          "required": ["location"]
        }
      }
    }],
    "tool_choice": "auto"
  }'

Dynamo parses the model output and returns OpenAI-compatible tool_calls.

{
  "id": "chatcmpl-b415caad-9be0-4d9e-ac6d-9d23bfe57703",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": null,
        "reasoning_content": "The user is asking about the weather in two cities: San Francisco and New York. I need to call the get_weather function for each city. I'll make two separate function calls to get the weather information for both locations.\n",
        "tool_calls": [
          {
            "id": "call-56223a95-3d14-4433-a94e-011f106c0e40",
            "type": "function",
            "function": {
              "name": "get_weather",
              "arguments": "{\"location\":\"San Francisco\"}"
            }
          },
          {
            "id": "call-d5b5772b-6b0c-4120-ad01-623278a937fe",
            "type": "function",
            "function": {
              "name": "get_weather",
              "arguments": "{\"location\":\"New York\"}"
            }
          }
        ]
      },
      "finish_reason": "tool_calls",
      "logprobs": null
    }
  ],
  "created": 1778653281,
  "model": "Qwen/Qwen3.5-4B",
  "object": "chat.completion"
}

If a tool call is parsed incorrectly, add "logprobs": true to one reproduction request and share the response. See Troubleshooting Tool Calls for what to capture when reporting an issue.

Custom Freeform Tools

Chat Completions accepts custom tools whose input is freeform text, such as code or a patch. Use the existing tool parser for your model and declare a custom tool:

{
  "tools": [{
    "type": "custom",
    "custom": {
      "name": "run_python",
      "description": "Execute Python source code.",
      "format": {"type": "text"}
    }
  }],
  "tool_choice": {"type": "custom", "custom": {"name": "run_python"}}
}

Returned message.tool_calls entries use type: "custom" with custom.name and custom.input. Your application executes the tool and replays the assistant call followed by a role: "tool" result with its matching tool_call_id. Ordinary function tools can appear in the same request.

Responses supports the same capability using its own custom-tool API format. Custom grammar formats are rejected before generation. Streaming custom input is buffered until the call completes, then emitted as text in custom.input. This streamed custom delta is a Dynamo extension to the published OpenAI Chat streaming schema. The optional tool_call_dispatch events currently cover function tools only; custom calls arrive through ordinary data chunks.

Function-oriented engines generate a JSON wrapper with a string input field. If a completed call does not match that wrapper, Dynamo returns the raw argument text as custom.input, preserving other calls, content, and usage. Applications must validate tool input before executing it. If the token limit interrupts the wrapper, Dynamo preserves finish_reason: "length" without emitting the truncated input.

Supported Tool Call Parsers

Choose a model family, then expand the matching model option to see its parser name and configuration details.

The Upstream name field shows where the vLLM or SGLang parser name differs from Dynamo’s — relevant when using --dyn-chat-processor vllm or sglang (see Parser Engine Fallback). Upstream names are engine-version dependent. A Dynamo-only label describes this catalog’s Dynamo entry; it is not a claim that no engine release can implement the format.

Parser: kimi_k3 (alias kimi-k3)

Notes: XTML tools and reasoning. Pair with --dyn-reasoning-parser kimi_k3. A named tool_choice forces thinking=false and enable_thinking=false before rendering, overriding an explicit request to enable thinking; reasoning_effort is retained. This override is specific to the K3 tool parser. See the Kimi K3 recipe.

Parser: kimi_k2

Upstream name: Same name across Dynamo, vLLM, and SGLang

Notes: For the default Unified route, pair with --dyn-reasoning-parser kimi_k25. The legacy Unicode-delimiter reasoning parser kimi is a separate V1 format; pairing it with kimi_k2 uses the split route under auto and is rejected by explicit V2. For Kimi K2.5 thinking models, use kimi_k25 so <think> blocks and tool calls are parsed from the same response. See Reasoning Parsing (Dynamo).

Parser: minimax_m2

Upstream name: vLLM: minimax_m2

Notes: XML <minimax:tool_call>

Parser: minimax_m3

Upstream name: vLLM: minimax_m3

Notes: MiniMax namespace-token XML. V2 requires minimax_m3 or minimax-m3. The aliases minimax_m3_nom and minimax-m3-nom are V1-only.

Parser: deepseek_v41

Notes: Unified V4.1 reasoning and spaced DSML calls with JSON bodies or typed parameter tags. Use the same reasoning selector when setting both. V1 does not provide this selector. See the DeepSeek V4.1 Flash recipe.

Parser: deepseek_v4

Upstream name: vLLM: deepseek_v4; SGLang: deepseekv4

Notes: DSML tags (<｜DSML｜tool_calls>...). Aliases: deepseek-v4, deepseekv4

Parser: deepseek_v3

Upstream name: SGLang: deepseekv3

Notes: Special Unicode markers

Parser: deepseek_v3_1

Upstream name: Dynamo-only

Notes: JSON separators

Parser: deepseek_v3_2

Upstream name: Dynamo-only

Notes: DSML tags (<｜DSML｜function_calls>...)

Parser: qwen3_coder

Upstream name: Same name across Dynamo, vLLM, and SGLang

Notes: XML <tool_call><function=...>

Parser: hermes

Upstream name: vLLM: qwen2_5; SGLang: qwen25 (for Qwen models)

Notes: <tool_call> JSON. Dynamo also provides the qwen25 parser preset for this format.

Parser: glm47

Upstream name: Dynamo-only

Notes: XML <arg_key>/<arg_value>

Parser: nemotron_deci

Upstream name: Dynamo-only

Notes: <TOOLCALL> JSON

Parser: nemotron_nano

Upstream name: Dynamo-only

Notes: Alias for qwen3_coder

Parser: gemma4

Upstream name: vLLM: gemma4

Notes: Custom non-JSON grammar with <\|"\|> string delimiters and <\|tool_call>...<tool_call\|> markers. Aliases: gemma-4. Pair with --dyn-reasoning-parser gemma4 and --custom-jinja-template examples/chat_templates/gemma4_tool.jinja.

Parser: beam

Upstream name: none (Dynamo-native, served by the dynamo-parsers-v2 unified family)

Notes: Channel grammar: reasoning in <\|channel\|>analysis<\|message\|>...<\|end\|>, the answer in <\|channel\|>final<\|message\|>...<\|return\|>, and one <\|start\|>assistant to=NAME<\|channel\|>analysis <\|constrain\|>xml<\|message\|>...<\|call\|> message per tool call with <rfl:parameter name="KEY">VALUE</rfl:parameter> arguments typed from the tool schema. Pair with --dyn-reasoning-parser beam; both flags are required and the pair routes to the unified parser without DYN_PARSER_VERSION=2. The route accepts the request field preserve_thinking (supplied prior assistant reasoning always stays in the rendered history; true and omitted are the same, false is accepted and has no effect — omit reasoning_content to drop it) and defaults allow_missing_thinking=true so history turns without reasoning render an empty analysis block instead of failing and chat_template_kwargs.builtin_tools (names resolved against tools for rendering only).

Per-turn grammar: on the Beam route the frontend holds every chat turn to this channel grammar with a structural tag (a non-empty analysis channel, then either tool calls whose rfl:parameter values are held to their JSON-schema types or the final answer), mirroring the model builder’s SGLang serving layer; required and named tool choices reuse the same tag, so they are enforced on the Harmony wire. A response_format (json_object or json_schema) is held inside the final channel behind the analysis, so the response keeps its reasoning and the channel structure the parser expects; a forced tool choice ignores response_format, and the nvext guided-decoding fields (guided_json, guided_regex, …) stay bare constraints from the first token. Set DYN_BEAM_TURN_GRAMMAR=0 on the frontend to disable it for automatic tool choice (forced choices keep theirs; response_format then falls back to the bare JSON constraint).

Rejected requests (Beam cannot enforce the requested tool constraints with this schema and prompt configuration): the frontend returns HTTP 400 with this message instead of serving the request best-effort when the request asks for tool enforcement that the turn grammar cannot provide. That happens when the request has:

  • a forced tool choice (required or a named function), but the last assistant message you send as a prefill has already started the final answer, so a tool call can no longer be forced;
  • an applicable strict: true tool (any strict tool under auto or required, the named one under a named choice; tool_choice: "none" makes none applicable) whose schema the grammar cannot hold, for example one using keywords outside its JSON-schema subset such as pattern;
  • parallel_tool_calls: false with usable tools together with an nvext guided-decoding field (guided_json, guided_regex, …), which takes the output away from the grammar.

To resolve it, drop strict or fix the schema, remove the assistant prefill before forcing a call, or drop the guided-decoding field. Independently of the grammar, a strict: true tool whose schema is outside OpenAI’s strict subset is rejected with HTTP 400 on this route as on every other.

Parser: harmony

Upstream name: Dynamo-only

Notes: Harmony channel format

Parser: phi4

Upstream name: vLLM: phi4_mini_json

Notes: functools[...] JSON

Parser: pythonic

Upstream name: Same name across Dynamo, vLLM, and SGLang

Notes: Python-list tool syntax

Parser: llama3_json

Upstream name: Same name across Dynamo, vLLM, and SGLang

Notes: <\|python_tag\|> tool syntax

Parser: mistral

Upstream name: Same name across Dynamo, vLLM, and SGLang

Notes: [TOOL_CALLS]...[/TOOL_CALLS]

Parser: jamba

Upstream name: Dynamo-only

Notes: <tool_calls> JSON

Parser: muse_glimmer (Dynamo alias muse)

Upstream name: Check your engine version’s --help

Notes: Recipient-routed ATEM channels and calls. Unified owns reasoning, visible text, and tool output; no V1 selector exists.

Parser: muse_spark (alias muse_spark_open)

Upstream name: Check your engine version’s --help

Notes: Recipient-routed ATEM calls with visible commentary output. Unified owns the whole response; no V1 tool parser exists.

Parser: inkling

Upstream name: Check your engine version’s --help

Notes: V1 channel-framed JSON tool calls using name and args. Pair with the inkling reasoning parser. Auto uses the legacy route; explicit V2 rejects this name.

The pinned library also registers beam, nemotron_v3 / nemotron3, and gpt_oss / harmony. See the routing table.

Parser: default

Upstream name: Dynamo-only

Notes: Empty JSON config (no start/end tokens). Prefer a model-specific parser for production use.

Structural tags are enabled by default

For parser and backend combinations that support them, Dynamo enables xgrammar structural tags by default so guided decoding matches the parser’s tool-call format at token granularity. See Structural Tag (Guided Decoding for Tool Calls) for the activation policy, compatibility fallbacks, and opt-out settings.