Tool Calling & Reasoning

Chat Processors

Pick who parses model output — the dynamo default, or vLLM/SGLang engine fallback

以 Markdown 格式查看

Every request to /v1/chat/completions is handled by a chat processor. It renders the chat template, tokenizes, routes to the worker, assembles the OpenAI-compatible response, and — for the formats it supports — parses tool calls and reasoning out of the model’s raw output.

You have three options, selected with the --dyn-chat-processor Frontend flag (env var DYN_CHAT_PROCESSOR):

  • dynamo (default) — Dynamo’s framework-agnostic Rust processor. It works on every backend (vLLM, SGLang, TRT-LLM), supports disaggregated serving, and ships built-in parsers for a broad set of model formats. Use this whenever Dynamo has a parser for your model — which is most of the time.
  • vllm / sglang (engine fallback) — hand parsing back to the engine’s own Python parser. Use these only when Dynamo does not ship a parser for your model’s format.

All three keep Dynamo’s tokenization, routing, and disaggregated serving — the only thing that changes is who runs the parser. Your choice also decides the parser flag names and which component carries them; the tabs below show exactly where they go.

New here? Read the Overview first for how the chat processor relates to the tool-call and reasoning parsers.

Configuration

Use this whenever Dynamo has a parser for your model. Leave --dyn-chat-processor unset (it defaults to dynamo). Put the parser flags on the worker as --dyn-tool-call-parser / --dyn-reasoning-parser, using names from Dynamo’s registry. The same worker flags work on vLLM, SGLang, or TRT-LLM.

apiVersion: nvidia.com/v1beta1
kind: DynamoGraphDeployment
metadata:
  name: qwen3-parsers
spec:
  components:
  - name: Frontend            # defaults to `dynamo` — no chat-processor args needed
    type: frontend
    # ...
  - name: VllmWorker          # worker selects the Dynamo parsers
    type: worker
    # ...
    podTemplate:
      spec:
        containers:
        - name: main
          # ...
          command:
          - python3
          - -m
          - dynamo.vllm
          args:
          - --model
          - Qwen/Qwen3-0.6B
          - --dyn-tool-call-parser   # names from Dynamo's registry
          - hermes
          - --dyn-reasoning-parser
          - qwen3

For the parser names, see Tool Call Parsing and Reasoning Parsing.

Engine fallback is supported on vLLM and SGLang only. TRT-LLM engine fallback is a work in progress — on a TRT-LLM worker, keep the default dynamo chat processor.

See Also