Troubleshoot NVIDIA NIM for LLMs

Use this documentation to troubleshoot issues that arise when you work with NVIDIA NIM for large language models (LLMs).

Guided decoding fails to recognize whitespace pattern

Using the outlines backend for regular expressions might cause the regular expressions to fail to compile. You might see a message such as "guided_whitespace_pattern": "*invalid*pattern*". In this case, use the xgrammar backend instead, which supports a wider range of regular expressions than outlines. For more information, refer to Structured Generation with NVIDIA NIM for LLMs.

vLLM profile fails to deploy

If the vLLM profile fails to deploy, it is likely because of GPU resource problems. You can try the following troubleshooting options:

• Allocate more GPU resources.

• Reduce the value of NIM_MAX_MODEL_LEN. Start by setting it to 70,000. You can lower the value if this does not work. For more information, refer to Configure Your NIM with NVIDIA NIM for LLMs.