Is this page helpful?

Release Notes#

This page lists changes, fixes, and known issues for each NIM LLM release.

Release 2.0.11#

Highlights#

This maintenance release refreshes the model-specific and model-free NIM (vLLM) containers to version 2.0.11 and updates the support matrix to match the profiles and GPUs validated for this release. The inference backend is updated to vLLM 0.27.0.

Model-Free NIM Update#

This release includes version 2.1.2 of the SGLang Model-Free NIM container. The container uses the SGLang inference engine and can be used to run models like GLM-5.3.

For more information, refer to Get Started with GLM-5.3.

Bug Fixes#

No additional customer-facing bug fixes are documented for this maintenance release.

Support Changes#

The following support boundaries apply to NIM 2.0.11. They are listed here, rather than as known issues, because the affected combinations do not start at their default configuration or are not included in the support matrix.

  • For gpt-oss-120b, the MXFP4 TP8 profile on NVIDIA A10G and NVIDIA A100-SXM4-40GB requires NIM_KVCACHE_PERCENT=0.80 so that sampler warm-up has enough device memory. The LoRA TP4 and TP8 profiles on NVIDIA L40S and NVIDIA RTX PRO 4500 Blackwell Server Edition require piecewise CUDA graphs:

    --compilation-config '{"cudagraph_mode":"PIECEWISE"}'

    Pass this option directly to the container. For deployment-specific examples, refer to Advanced Configuration.

    Lower-VRAM LoRA combinations that cannot fit the model are not supported and are omitted from the support matrix.

  • The fused mixture-of-experts kernels used by nemotron-3-nano, gpt-oss-20b, and gpt-oss-120b support LoRA adapters with a maximum rank of 128. Adapters with a greater rank are not supported.

  • The gpt-oss-20b LoRA profile on NVIDIA A10G requires a context length below the default 131,072 token context length. Set NIM_MAX_MODEL_LEN to a value that fits the available KV-cache memory.

  • When the model-free NIM serves apriel-nemotron or codestral, use a GPU with at least 80 GB of usable VRAM. These models require approximately 66 GB for weights and runtime allocations and cannot start on smaller GPUs.

  • For llama-3.1-70b-instruct, the following combinations are not supported in this release: BF16 TP2 with or without LoRA on NVIDIA RTX PRO 6000 Blackwell Server Edition, and BF16 TP8 without LoRA on NVIDIA B200.

  • For nemotron-3-super-120b-a12b, the following LoRA combinations are not supported in this release: BF16 TP2 on NVIDIA GH200-144G-HBM3e and NVIDIA H200-NVL, BF16 TP4 on NVIDIA H100-NVL, and BF16 TP8 on NVIDIA L40S.

  • For nemotron-3-ultra-550b-a55b, the BF16 TP8 LoRA profile is not supported on NVIDIA B200 and NVIDIA B300-SXM6-AC. The profile remains supported on the other GPUs listed in the support matrix.

Known Issues#

This release includes the following known issues and limitations:

  • The llama-3.3-70b-instruct BF16 TP8 profile on NVIDIA A100-SXM4-40GB can intermittently fail to start during model-cache workspace materialization with FileNotFoundError / os error 2. Workaround: Restart or redeploy the NIM.

  • The NVFP4 profiles of nemotron-3-nano can return semantically incorrect output in this release. The server starts and responds normally, but on the NVFP4 precision the model fails stop-string handling and mathematical-conformance checks that the same weights pass on the BF16 and FP8 profiles. This is a serving-stack regression introduced between NIM 2.0.10 and 2.0.11; the identical NVFP4 checkpoint passed these checks on release 2.0.10. Root cause is under investigation upstream. The BF16 and FP8 profiles of nemotron-3-nano are not affected. There is no workaround for this issue in NIM 2.0.11; use a BF16 or FP8 profile of nemotron-3-nano where correct output is required.

  • Some /v1/completions requests to llama-3.3-nemotron-super-49b-v1.5 can take much longer than expected to return on the NVIDIA GB10 (BF16 profile) and NVIDIA L40S GPUs in this release. The server stays healthy and returns a correct HTTP 200 response, but completion requests that set echo=true, frequency_penalty, or user (and their streaming equivalents) can take up to roughly 200 seconds to complete, which can exceed a default 100-second client request timeout and surface as a client-side timeout even though the request eventually succeeds. Chat requests, small completions, and completion requests that do not use these parameters are not affected, and the same GB10 and L40S profiles served these scenarios within the timeout in release 2.0.10; this is a 2.0.11 serving-stack regression on the completions path.

    Workaround: Increase the client-side request timeout.

    For affected /v1/completions requests on GB10 and L40S, increase the client-side request timeout to about 300 seconds. Where a short client timeout is required, avoid the echo, frequency_penalty, and user parameters on those GPUs.

  • The Responses API (/v1/responses) of nemotron-3-nano has known output-correctness and response-format limitations in this release. The server starts and responds normally, but certain /v1/responses requests — including empty-input handling and response-format compatibility, and the equivalent scenarios on LoRA deployments — can return incorrect or non-conformant output. The limitation is precision-independent (it is observed on the BF16, FP8, and NVFP4 profiles) and is not caused by a startup failure. This limitation carries forward from release 2.0.6 and is not new to 2.0.11. There is no workaround for this issue in NIM 2.0.11.

  • When the model-free NIM (nvidia/model-free-nim) serves codestral, the OpenAI-compatible chat endpoint /v1/chat/completions returns HTTP 400. The codestral tokenizer defines no chat template, and beginning with Transformers v4.44 no default chat template is applied, so chat requests are rejected unless a chat template is supplied. The text /v1/completions endpoint is unaffected and serves normally. This is a limitation of the codestral tokenizer and its interaction with Transformers v4.44, not a 2.0.11 regression. Workaround: Use /v1/completions or supply a chat template.

  • When the model-free NIM serves codestral, structured/guided-decoding requests to the text /v1/completions endpoint — whether they set the OpenAI-standard response_format field or the vLLM guided_json extra-body parameter — return HTTP 500. The guided-decoding backend (xgrammar) does not recognize the cached-tokenizer wrapper used for codestral, so structured-output requests fail. Requests that do not use guided decoding serve normally. This limitation is not a 2.0.11 regression; it also occurs in release 2.0.10. Workaround: Do not use response_format or guided_json with codestral; request unstructured completions instead.

  • When the model-free NIM serves gpt-oss-20b, chat completions may not honor the stop parameter and may include the model’s internal harmony reasoning-channel markers (for example <|channel|>analysis) in the returned message content. This affects stop/include_stop_str_in_output behavior across all profiles and GPUs and is an artifact of the gpt-oss-20b harmony response format carried forward from earlier releases; it is not a 2.0.11 regression. There is no server-side workaround in NIM 2.0.11; if strict stop-string enforcement or marker-free content is required, strip the channel markers and apply stop strings client-side.

  • On NVIDIA B200, the nemotron-3-ultra-550b-a55b NVFP4 tensor-parallel-4 throughput profile (vllm-nvfp4-tp4-pp1) shows a measurable output-token-throughput regression in NIM 2.0.11 relative to 2.0.9. The NIM serves correctly and returns valid output, and accuracy is unaffected; only steady-state generation throughput is reduced. Across the B200 benchmark recipes, every measured configuration regressed (aiperf-b200-throughput 6/6, roughly -5.6% to -9.6%; aiperf-b200-generic 6/6, roughly -5.0% to -11.8%), and the multi-token-prediction (MTP) recipe regressed on 2 of 6 configurations (worst case about -26.8% at concurrency 1). The effect is consistent across five distinct B200 nodes, indicating a software regression rather than node or thermal variance. There is no application-side workaround.

  • On NIM Operator versions earlier than 3.1.2, deploying nemotron-3-ultra-550b-a55b fails: the NIMCache controller serializes the model’s full manifest into a single Kubernetes ConfigMap (~1.63 MiB for this model), which exceeds the Kubernetes 1 MiB ConfigMap size limit, so NIMCache never becomes Ready and the dependent NIMService never starts. Workaround: Upgrade to NIM Operator 3.1.2 or later, or use another supported deployment method for this model. This limitation carries forward from releases 2.0.9 and 2.0.10 and is not a 2.0.11 regression.

For information about past updates and older versions, refer to the previous release notes.