Tool Calling & Reasoning

Chat Processors

Pick who parses model output — the dynamo default, or vLLM/SGLang engine fallback

View as Markdown

Every request to /v1/chat/completions is handled by a chat processor. It renders the chat template, tokenizes, routes to the worker, assembles the OpenAI-compatible response, and — for the formats it supports — parses tool calls and reasoning out of the model’s raw output.

You have three options, selected with the --dyn-chat-processor Frontend flag (env var DYN_CHAT_PROCESSOR):

  • dynamo (default) — Dynamo’s framework-agnostic Rust processor. It works on every backend (vLLM, SGLang, TRT-LLM), supports disaggregated serving, and ships built-in parsers for a broad set of model formats. Use this whenever Dynamo has a parser for your model — which is most of the time.
  • vllm / sglang (engine fallback) — hand parsing back to the engine’s own Python parser. Use these only when Dynamo does not ship a parser for your model’s format.

All three keep Dynamo’s tokenization, routing, and disaggregated serving — the only thing that changes is who runs the parser. Your choice also decides the parser flag names and which component carries them; the tabs below show exactly where they go.

New here? Read the Introduction first for how the chat processor relates to the tool-call and reasoning parsers.

Configuration

Use this whenever Dynamo has a parser for your model. Leave --dyn-chat-processor unset (it defaults to dynamo). Put the parser flags on the worker as --dyn-tool-call-parser / --dyn-reasoning-parser, using names from Dynamo’s registry. The same worker flags work on vLLM, SGLang, or TRT-LLM.

1apiVersion: nvidia.com/v1beta1
2kind: DynamoGraphDeployment
3metadata:
4  name: qwen3-parsers
5spec:
6  components:
7  - name: Frontend            # defaults to `dynamo` — no chat-processor args needed
8    type: frontend
9    # ...
10  - name: VllmWorker          # worker selects the Dynamo parsers
11    type: worker
12    # ...
13    podTemplate:
14      spec:
15        containers:
16        - name: main
17          # ...
18          command:
19          - python3
20          - -m
21          - dynamo.vllm
22          args:
23          - --model
24          - Qwen/Qwen3-0.6B
25          - --dyn-tool-call-parser   # names from Dynamo's registry
26          - hermes
27          - --dyn-reasoning-parser
28          - qwen3

For the parser names, see Tool Call Parsing and Reasoning Parsing.

Engine fallback is supported on vLLM and SGLang only. TRT-LLM engine fallback is a work in progress — on a TRT-LLM worker, keep the default dynamo chat processor.

See Also