Is this page helpful?

Custom Parsers and Chat Templates#

NIM LLM and VLM supports the vLLM CLI arguments that take file paths: --reasoning-parser-plugin, --tool-parser-plugin, and --chat-template. NIM resolves these paths for you, so you can reference a plugin file by bare filename without knowing the container’s internal directory layout. For a product overview, refer to Overview.

Most users do not need a plugin file. vLLM ships with a large set of built-in parsers that you select by name, and those names are listed in Built-In Parser Names. Continue to Supply a Custom Parser or Chat Template only when the parser your model needs is not in the built-in list.

Built-In Parser Names#

Select a parser by name from the following lists. When the name is listed, no plugin file and no -plugin argument are required.

Reasoning Parsers#

Pass any of these names to --reasoning-parser:

deepseek_r1, deepseek_v3, ernie45, glm45, granite, holo2, hunyuan_a13b, kimi_k2, minimax_m2, minimax_m2_append_think, mistral, nemotron_v3, olmo3, openai_gptoss, qwen3, seed_oss, step3, step3p5.

# Built-in reasoning parser - no plugin file needed
nim-serve --reasoning-parser nemotron_v3

For the authoritative list, refer to the vLLM reasoning parser registry.

Tool-Call Parsers#

Pass any of these names to --tool-call-parser:

deepseek_v3, deepseek_v31, ernie45, glm45, glm47, granite, granite-20b-fc, granite4, hermes, hunyuan_a13b, internlm, jamba, kimi_k2, llama3_json, llama4_json, llama4_pythonic, longcat, minimax, minimax_m2, mistral, olmo3, openai, phi4_mini_json, pythonic, qwen3_coder, qwen3_xml, seed_oss, step3, step3p5, xlam.

# Built-in tool-call parser - no plugin file needed
nim-serve --tool-call-parser mistral

For the authoritative list, refer to the vLLM tool-call parser registry.

Note

Built-in parser names are vLLM internal identifiers; they can differ from the names shown on a model card. For example, the model card for Nemotron-3 Nano references a nano_v3 parser shipped as an external .py file, but NIM LLM and VLM ships a built-in nemotron_v3 parser that works without any plugin file. When in doubt, try the built-in name first.

How Plugin Path Resolution Works#

NIM LLM and VLM inspects the value you pass to each file-path argument and decides whether to treat it as a remote URL, a built-in name, a literal path, or a filename to look up inside the model checkpoint. Understanding this order tells you which of the delivery methods in the next section applies to your deployment, and why a bare filename works without an absolute path.

For each file-path argument, NIM LLM and VLM applies the following rules in order:

  1. URL-shaped values (containing ://, for example https://... or s3://...) pass through to vLLM unchanged. NIM LLM and VLM does not attempt remote resolution.

  2. Built-in names (no file extension and no path separator, for example tool_chat_template_mistral) pass through to vLLM unchanged.

  3. Literal path exists (absolute or relative to the working directory) — used as-is, converted to an absolute path.

  4. Bare filename in checkpoint dir — if <checkpoint_dir>/<filename> exists, used and logged.

  5. Not found anywhere — NIM LLM and VLM exits with a non-zero code and a clear error listing all searched locations.

When resolution succeeds against the checkpoint directory, NIM logs the absolute path at INFO level:

INFO: Plugin --reasoning-parser-plugin: resolved 'nano_v3_reasoning_parser.py' to /tmp/nim_abc123/nano_v3_reasoning_parser.py

Filename Conflicts#

If the same filename exists at both the literal path (typically /opt/nim/) and the checkpoint directory, the literal path wins and a warning is logged. This can happen when a derived Dockerfile copies a file into /opt/nim/, or when a Kubernetes ConfigMap is mounted at the working directory.

WARNING: Plugin --reasoning-parser-plugin: resolved 'my_parser.py' to /opt/nim/my_parser.py
         (a copy also exists at /tmp/nim_abc123/my_parser.py — use an absolute path to select a specific one)

To force a specific file, pass an absolute path:

nim-serve \
  --reasoning-parser-plugin /tmp/nim_abc123/my_parser.py \
  --reasoning-parser my_custom

Supply a Custom Parser or Chat Template#

Deliver a custom parser or chat template file to the container, and then confirm that NIM resolved it. Which delivery method you use depends on whether the file ships inside the model checkpoint, is mounted from the host, or is passed through a Kubernetes workload.

Before you start, complete the following prerequisites:

  1. Confirm that no built-in name covers your model by checking the lists in Built-In Parser Names.

  2. Obtain the custom parser or chat template file for your model.

  3. Identify the parser name that the file registers, such as nano_v3.

Place the File in the Model Checkpoint Directory#

Use this method when the model checkpoint already includes the plugin file, which is typical when the model is published with a custom parser. Pass the bare filename, and NIM LLM and VLM finds the file automatically.

nim-serve \
  --reasoning-parser-plugin nano_v3_reasoning_parser.py \
  --reasoning-parser nano_v3

Volume-Mount at an Absolute Path#

Use this method when you run the container directly and the file lives on the host. Mount your plugin file, or a directory of files, into the container and reference it by absolute path. NIM LLM and VLM passes absolute paths through unchanged.

docker run --gpus all \
  -v /home/user/plugins:/mnt/plugins:ro \
  -p 8000:8000 \
  ${NIM_LLM_MODEL_FREE_IMAGE}:2.0.13 \
  nim-serve \
    --reasoning-parser-plugin /mnt/plugins/my_custom_parser.py \
    --reasoning-parser my_custom

Pass Container Arguments (Kubernetes and Helm)#

Use this method for a Kubernetes deployment. Pass the same arguments through the container args field:

containers:
  - name: nim
    image: <nim_llm_image>
    args:
      - "--reasoning-parser-plugin"
      - "nano_v3_reasoning_parser.py"
      - "--reasoning-parser"
      - "nano_v3"

For the NIM Helm chart, use customArgs:

customArgs:
  - "--reasoning-parser-plugin"
  - "nano_v3_reasoning_parser.py"
  - "--reasoning-parser"
  - "nano_v3"

Both forms use the same path-resolution rules as direct Docker arguments and preserve any NIM_PASSTHROUGH_ARGS value provided by the image. For the environment-variable fallback and configuration precedence, refer to Advanced Configuration.

Confirm That NIM Resolved the File#

Whichever delivery method you used, confirm the result before you send traffic. Check the container logs for the INFO line described in How Plugin Path Resolution Works. If the container exits instead, refer to Troubleshooting.

Troubleshooting#

Use the following topics when plugin resolution fails or the wrong file is loaded.

Error: “Plugin file for … not found”#

If you see an error like the following, the plugin file does not exist at any searched location:

ERROR: Plugin file for --reasoning-parser-plugin not found: 'my_parser.py'.
Searched locations:
  - /opt/nim/my_parser.py
  - /tmp/nim_abc123/my_parser.py

To provide a custom plugin, either:
  1. Place the file in the model checkpoint directory
  2. Volume-mount it and use an absolute path
  3. Use a built-in parser name (no file extension needed)

Verify one of the following:

  • The file is included in the model checkpoint at the expected path.

  • The volume mount is correctly configured and uses an absolute path.

  • A built-in parser name, without the .py extension, covers your model. Refer to the lists in Built-In Parser Names.

Wrong Plugin File Loaded#

If the warning about precedence appears in the logs and you intended to use the checkpoint version, pass an absolute path to the checkpoint file as shown in Filename Conflicts.

Next Steps#

After parsers resolve correctly, continue with the following topics: