Is this page helpful?

Release Notes#

This page lists changes, fixes, and known issues for each NIM LLM and VLM release.

Release 2.0.12#

Highlights#

NVIDIA NIM for Large Language Models is now known as NVIDIA NIM for Large Language Models (LLM) and Vision Language Models (VLM). The updated product name reflects the expanded scope of this release: in addition to large language model (LLM) inference, NIM now includes vision language model (VLM) support. For more information, refer to Image, Audio, and Video Input.

This release adds five models to the support matrix, refreshes the model-specific and model-free NIM containers to version 2.0.12, and updates the inference backend to vLLM 0.27.1.

vLLM 0.27.1#

The vLLM inference backend is updated from 0.27.0 to vLLM 0.27.1, bringing improvements to functionality, performance, and stability. The update applies to both the GPU and CPU container images.

New Models#

This release adds the following models:

Refer to the Support Matrix for NIMs for the precision, tensor-parallel size, and GPU combinations validated for each model.

A variant of the model-specific NIM container was released for deepseek-v4-pro-0813. For more information, refer to Get Started with DeepSeek-V4-Pro-0813.

Bug Fixes#

This release includes the following bug fixes:

  • Hybrid Mamba models no longer discard a prefix-cache hit when speculative decoding resumes generation. Mamba state is now cached at the position where the draft model resumes, rather than only at block boundaries. A matched prefix therefore no longer falls back to the previous block boundary and re-prefills the intervening tokens on every request. This restores time-to-first-token on hybrid Mamba profiles that combine prefix caching with speculative decoding, such as nemotron-3-ultra-550b-a55b.

  • The Helm chart now applies the configured service account to multi-node deployment pods. Multi-node pods previously ran under the namespace default service account, which could fail admission in clusters that require a specific service account. Refer to Multi-Node Deployment for details.

Support Changes#

The support matrix adds kimi-k2.6, mistral-small-4-119b-2603, nemotron-3-nano-omni-30b-a3b-reasoning, qwen3.5-122b-a10b, and qwen3.5-397b-a17b.

Known Issues#

This release includes the following known issues and limitations:

  • The fused mixture-of-experts kernels used by gpt-oss-20b support a maximum LoRA rank of 128. Deploying LoRA adapters with rank 256 fails at startup with a fused_moe_lora_one_shot supports max_lora_rank<=128 assertion error. This carries forward from 2.0.11 and is still present in 2.0.12. Workaround: Use LoRA adapters with rank 128 or lower.

  • The LoRA profile of gpt-oss-20b on NVIDIA A10G fails at startup with a KV-cache out-of-memory error at the default 131,072-token context length. This carries forward from 2.0.11 and is still present in 2.0.12. Workaround: Reduce the context length by setting NIM_MAX_MODEL_LEN to a smaller value.

  • For llama-3.3-nemotron-super-49b-v1.5, several profiles fail to start at the default 131,072-token context length with a KV-cache out-of-memory error, because the KV cache required for that context length exceeds the device memory available after the model weights are loaded. This affects the FP8 (TP1) profiles on NVIDIA H100 80GB HBM3 and NVIDIA GB10, and the NVFP4 (TP2) profile on NVIDIA RTX PRO 4500 Blackwell Server Edition. On NVIDIA H100 80GB HBM3 the same startup failure also causes the container-compliance checks to report an error, because the server never becomes ready. The affected profiles start and serve normally on the other GPUs in the support matrix. Workaround: Reduce the context length by setting NIM_MAX_MODEL_LEN to a value that fits the KV cache in the available device memory.

  • For llama-3.3-nemotron-super-49b-v1.5, the BF16 LoRA profiles on NVIDIA A100-SXM4-40GB (TP4 and TP8) and NVIDIA A10G (TP8) can fail to start with a CUDA out-of-memory error during sampler warm-up, when the default KV-cache reservation leaves insufficient device memory for the warm-up allocation. The affected profiles start and serve normally on higher-VRAM GPUs in the support matrix. Workaround: Lower the KV-cache reservation to leave memory for sampler warm-up, for example NIM_KVCACHE_PERCENT=0.80 (the same mitigation used for gpt-oss-120b).

  • For llama-3.3-nemotron-super-49b-v1.5 on NVIDIA GB10, the NVFP4 (TP1) profiles can intermittently fail to start during vLLM memory profiling on the platform’s unified memory, when the free device memory changes while vLLM is measuring it and the usable memory is under-estimated. The failure is intermittent, and the profiles start and serve normally on the other GPUs in the support matrix. Workaround: Restart or redeploy the NIM.

  • The NVFP4 profiles of nemotron-3-nano can return semantically incorrect output in this release. The server starts and responds normally, but on the NVFP4 precision the model fails stop-string handling and mathematical-conformance checks that the same weights pass on the BF16 and FP8 profiles. This is a serving-stack regression first observed in NIM 2.0.11; the identical NVFP4 checkpoint passed these checks on release 2.0.10. This limitation carries forward from 2.0.11 and is still present in 2.0.12. Root cause is under investigation upstream. The BF16 and FP8 profiles of nemotron-3-nano are not affected. There is no workaround for this issue in NIM 2.0.12; use a BF16 or FP8 profile of nemotron-3-nano where correct output is required.

  • The Responses API (/v1/responses) of nemotron-3-nano has known output-correctness and response-format limitations in this release. The server starts and responds normally, but certain /v1/responses requests, including empty-input handling and response-format compatibility, and the equivalent scenarios on LoRA deployments, can return incorrect or non-conformant output. The limitation is precision-independent (it is observed on the BF16, FP8, and NVFP4 profiles) and is not caused by a startup failure. This limitation carries forward from release 2.0.6 and is still present in 2.0.12. There is no workaround for this issue in NIM 2.0.12.

  • The fused mixture-of-experts kernels used by nemotron-3-nano support a maximum LoRA rank of 128. Deploying LoRA adapters with rank 256 fails at startup with a fused_moe_lora_one_shot supports max_lora_rank<=128 assertion error. This carries forward from 2.0.11 and is still present in 2.0.12. Workaround: Use LoRA adapters with rank 128 or lower.

  • When the model-free NIM serves codestral, structured/guided-decoding requests to the text /v1/completions endpoint — whether they set the OpenAI-standard response_format field or the vLLM guided_json extra-body parameter — return HTTP 500. The guided-decoding backend (xgrammar) does not recognize the cached-tokenizer wrapper used for codestral, so structured-output requests fail. Requests that do not use guided decoding serve normally. This limitation carries forward from 2.0.11 (and 2.0.10) and is still present in 2.0.12. Workaround: Do not use response_format or guided_json with codestral; request unstructured completions instead.

  • When the model-free NIM serves apriel-nemotron on the NVIDIA GB10 GPU, some large /v1/completions requests may not return within a default 100-second client request timeout. During 2.0.12 validation, completion requests with max_tokens=2048 that also set echo=true or frequency_penalty (and their streaming equivalents) did not return within the 100-second window on GB10. The server stayed healthy throughout — /health returned 200 and the server logged no error or crash — so these surface as client-side timeouts rather than server failures. It is not established whether the affected requests complete slowly or do not complete within a practical window; whether this reflects genuine GB10 serving latency for these large-generation requests or a validation-environment timeout budget too tight for this slower GPU is under investigation (NVBug 6667785). Chat requests, smaller completions, and completion requests that do not set these parameters are not affected. Workaround: Pending resolution, on NVIDIA GB10 avoid max_tokens=2048 /v1/completions requests that also set echo=true or frequency_penalty, or serve these requests on a faster GPU. Raising the client-side request timeout above 100 seconds allows a slow-but-completing request to return, but does not help a request that does not complete.

  • deepseek-v4-pro-0813 uses a different base container and has the following limitations. DSpark speculative decoding is enabled by default on the H200, B200, and H20-3e throughput profiles. For more information, refer to Get Started with DeepSeek-V4-Pro-0813.

For information about past updates and older versions, refer to the previous release notes.