Is this page helpful?

Custom Parsers and Chat Templates#

NIM LLM and VLM supports built-in reasoning and tool-call parsers on both the vLLM and SGLang backends. Select a parser that matches your model and the inference engine in your container image. Most models can use a built-in parser without a plugin file.

You can supply a custom chat template with --chat-template on either backend. Custom parser plugin files, passed through --reasoning-parser-plugin or --tool-parser-plugin, apply to vLLM. NIM resolves these file paths so you can reference a file by its bare filename when it is included in the model checkpoint.

Select a Built-In Parser#

Pass a reasoning parser name to --reasoning-parser or a tool-call parser name to --tool-call-parser. No plugin argument is needed for a built-in parser:

nim-serve --reasoning-parser <reasoning-parser-name> --tool-call-parser <tool-call-parser-name>

Parser names are backend identifiers. A name accepted by vLLM can differ from the SGLang name for the same model. Refer to the parser documentation for your backend:

Backend

Reasoning Parsers

Tool-Call Parsers

vLLM

Reasoning Outputs

Tool Calling

SGLang

Reasoning Parser

Tool Parser

Check that the parser is available in your container image. For example, nim-serve --help lists the arguments accepted by the backend in that container image. For tool-calling setup, including vLLM’s --enable-auto-tool-choice argument, refer to Tool Calling and MCP Integration.

How Plugin Path Resolution Works#

NIM LLM and VLM inspects the value you pass to each file-path argument and decides whether to treat it as a remote URL, a built-in name, a literal path, or a filename to look up inside the model checkpoint. Understanding this order tells you which of the delivery methods in the next section applies to your deployment, and why a bare filename works without an absolute path.

For each file-path argument, NIM LLM and VLM applies the following rules in order:

  1. URL-shaped values (containing ://, for example https://... or s3://...) pass through to the inference engine unchanged. NIM LLM and VLM does not attempt remote resolution.

  2. Built-in names (no file extension and no path separator, for example tool_chat_template_mistral) pass through to the inference engine unchanged.

  3. Literal path exists (absolute or relative to the working directory) — used as-is, converted to an absolute path.

  4. Bare filename in checkpoint dir — if <checkpoint_dir>/<filename> exists, used and logged.

  5. Not found anywhere — NIM LLM and VLM exits with a non-zero code and a clear error listing all searched locations.

When resolution succeeds against the checkpoint directory, NIM logs the absolute path at INFO level:

INFO: Plugin --reasoning-parser-plugin: resolved 'my_reasoning_parser.py' to /tmp/nim_abc123/my_reasoning_parser.py

Filename Conflicts#

If the same filename exists at both the literal path (typically /opt/nim/) and the checkpoint directory, the literal path wins and a warning is logged. This can happen when a derived Dockerfile copies a file into /opt/nim/, or when a Kubernetes ConfigMap is mounted at the working directory.

WARNING: Plugin --reasoning-parser-plugin: resolved 'my_parser.py' to /opt/nim/my_parser.py
         (a copy also exists at /tmp/nim_abc123/my_parser.py — use an absolute path to select a specific one)

To force a specific file, pass an absolute path:

nim-serve \
  --reasoning-parser-plugin /tmp/nim_abc123/my_parser.py \
  --reasoning-parser my_custom

Supply a Custom Parser or Chat Template#

Deliver a custom parser or chat template file to the container, and then confirm that NIM resolved it. The parser plugin examples below use vLLM. To supply a chat template on either backend, use --chat-template <template-file> with the same file delivery methods. Which delivery method you use depends on whether the file ships inside the model checkpoint, is mounted from the host, or is passed through a Kubernetes workload.

Before you start, complete the following prerequisites:

  1. If you need a custom parser, confirm that no built-in parser covers your model and select a vLLM container image. Refer to Select a Built-In Parser.

  2. Obtain the custom parser or chat template file for your model.

  3. For a custom parser, identify the name that the file registers, such as my_custom.

Place the File in the Model Checkpoint Directory#

Use this method when the model checkpoint already includes the plugin file, which is typical when the model is published with a custom parser. Pass the bare filename, and NIM LLM and VLM finds the file automatically.

nim-serve \
  --reasoning-parser-plugin my_reasoning_parser.py \
  --reasoning-parser my_custom

Volume-Mount at an Absolute Path#

Use this method when you run the container directly and the file lives on the host. Mount your plugin file, or a directory of files, into the container and reference it by absolute path. NIM LLM and VLM passes absolute paths through unchanged.

docker run --gpus all \
  -v /home/user/plugins:/mnt/plugins:ro \
  -p 8000:8000 \
  ${NIM_LLM_MODEL_FREE_IMAGE}:2.1.4 \
  nim-serve \
    --reasoning-parser-plugin /mnt/plugins/my_custom_parser.py \
    --reasoning-parser my_custom

Pass Container Arguments (Kubernetes and Helm)#

Use this method for a Kubernetes deployment. Pass the same arguments through the container args field:

containers:
  - name: nim
    image: <nim_llm_image>
    args:
      - "--reasoning-parser-plugin"
      - "my_reasoning_parser.py"
      - "--reasoning-parser"
      - "my_custom"

For the NIM Helm chart, use customArgs:

customArgs:
  - "--reasoning-parser-plugin"
  - "my_reasoning_parser.py"
  - "--reasoning-parser"
  - "my_custom"

Both forms use the same path-resolution rules as direct Docker arguments and preserve any NIM_PASSTHROUGH_ARGS value provided by the image. For the environment-variable fallback and configuration precedence, refer to Advanced Configuration.

Confirm That NIM Resolved the File#

Whichever delivery method you used, confirm the result before you send traffic. Check the container logs for the INFO line described in How Plugin Path Resolution Works. If the container exits instead, refer to Troubleshooting.

Troubleshooting#

Use the following topics when plugin resolution fails or the wrong file is loaded.

Error: “Plugin file for … not found”#

If you see an error like the following, the plugin file does not exist at any searched location:

ERROR: Plugin file for --reasoning-parser-plugin not found: 'my_parser.py'.
Searched locations:
  - /opt/nim/my_parser.py
  - /tmp/nim_abc123/my_parser.py

To provide a custom plugin, either:
  1. Place the file in the model checkpoint directory
  2. Volume-mount it and use an absolute path
  3. Use a built-in parser name (no file extension needed)

Verify one of the following:

  • The file is included in the model checkpoint at the expected path.

  • The volume mount is correctly configured and uses an absolute path.

  • A built-in parser name, without the .py extension, covers your model. Refer to Select a Built-In Parser.

Wrong Plugin File Loaded#

If the warning about precedence appears in the logs and you intended to use the checkpoint version, pass an absolute path to the checkpoint file as shown in Filename Conflicts.

Next Steps#

After parsers resolve correctly, continue with the following topics: