Is this page helpful?

Release Notes#

This page lists changes, fixes, and known issues for the NIM LLM Production Branch version 6 (PB6) releases, newest first.

Important

PB releases (indicated by the -pb tag) provide stable, secure AI frameworks and SDKs for mission-critical applications. Refer to NVIDIA AI Enterprise Release Branches for more details.

Release 2.0.4-pb6.9#

NIM LLM 2.0.4-pb6.9 is a point release of the NVIDIA AI Enterprise Production Branch 6 (PB6) for NIM LLM. This release uses the standard NIM LLM 2.0.4 documentation set as its baseline and adds PB-specific release information on this page.

Unless a PB-specific update is listed here, the standard 2.0.4 release notes apply.

Highlights#

This release resolves 34 CVEs across the base image and PB-specific packages, including two critical-severity and 19 high-severity fixes. Refer to Security Fixes for more information.

Security Fixes#

This release includes the following security fixes:

Security Alerts#

This release includes the following security alerts:

  • Supported model scope: LLM only. PB6 supports large language model (LLM) inference. Vision-language (VLM) / multimodal workloads are not supported. VLM code paths are present in the base image but are not a supported PB6 workload.

Bug Fixes#

No PB-specific bug fixes are documented for this release beyond the security fixes above. For fixes from the baseline release, refer to the standard 2.0.4 release notes.

Support Changes#

The support changes introduced in 2.0.4-pb6.5 remain in effect for this release. Refer to Support Changes for the affected models, profiles, and GPUs.

Known Issues#

No new known issues are documented for this release. Refer to Known Issues for previously documented issues that remain open.

For baseline known issues and limitations, refer to the standard 2.0.4 release notes.

Release 2.0.4-pb6.8#

NIM LLM 2.0.4-pb6.8 is a point release of the NVIDIA AI Enterprise Production Branch 6 (PB6) for NIM LLM. This release uses the standard NIM LLM 2.0.4 documentation set as its baseline and adds PB-specific release information on this page.

Unless a PB-specific update is listed here, the standard 2.0.4 release notes apply.

Highlights#

This release resolves 68 CVEs across the base image and PB-specific packages, including two critical-severity and six high-severity fixes in PyJWT and AnyIO. Refer to Security Fixes for more information.

Security Fixes#

This release includes the following security fixes:

Security Alerts#

This release includes the following security alerts:

  • Supported model scope: LLM only. PB6 supports large language model (LLM) inference. Vision-language (VLM) / multimodal workloads are not supported. VLM code paths are present in the base image but are not a supported PB6 workload.

Bug Fixes#

No PB-specific bug fixes are documented for this release beyond the security fixes above. For fixes from the baseline release, refer to the standard 2.0.4 release notes.

Support Changes#

The support changes introduced in 2.0.4-pb6.5 remain in effect for this release. Refer to Support Changes for the affected models, profiles, and GPUs.

Known Issues#

No new known issues are documented for this release. Refer to Known Issues for previously documented issues that remain open.

For baseline known issues and limitations, refer to the standard 2.0.4 release notes.

Release 2.0.4-pb6.7#

NIM LLM 2.0.4-pb6.7 is a point release of the NVIDIA AI Enterprise Production Branch 6 (PB6) for NIM LLM. This release uses the standard NIM LLM 2.0.4 documentation set as its baseline and adds PB-specific release information on this page.

Unless a PB-specific update is listed here, the standard 2.0.4 release notes apply.

Highlights#

The 2.0.4-pb6.7 point release contains the following key changes:

  • Resolved 36 CVEs across the base image and PB-specific packages, including one high-severity fix in the Mooncake transfer engine. For details, refer to Security Fixes.

  • The Completions API has a known issue with stop strings and top_logprobs. Refer to Known Issues before you upgrade.

Security Fixes#

This release includes the following security fixes:

  • Mooncake transfer engine — Updated the bundled gRPC client from 1.82.1 to 1.83.2 to fix GHSA-vp52-pcj8-j9qc (High).

  • System packages — The updated base image moves openssl, libssh, perl, util-linux, curl, zlib, coreutils, and other system packages to patched versions, clearing 35 CVEs.

Security Alerts#

This release includes the following security alerts:

  • Supported model scope: LLM only. PB6 supports large language model (LLM) inference. Vision-language (VLM) / multimodal workloads are not supported. VLM code paths are present in the base image but are not a supported PB6 workload.

  • CVE-2026-15308 (Python, High): Denial of service in the html.parser.HTMLParser incremental HTML parser when processing uncontrolled input. SLA target: 2026-10-11.

Bug Fixes#

No PB-specific bug fixes are documented for this release beyond the security fixes above. For fixes from the baseline release, refer to the standard 2.0.4 release notes.

Support Changes#

The support changes introduced in 2.0.4-pb6.5 remain in effect for this release. Refer to Support Changes for the affected models, profiles, and GPUs.

Known Issues#

This release includes the following known issue:

  • Completion API: stop strings are not honored, and top_logprobs returns one fewer entry than requested. On the completions API, generation may not stop at a configured stop string. Affected requests return finish_reason as length rather than stop, and streamed output does not end at the stop token. Separately, a request specifying logprobs=N can return N-1 top_logprobs entries. This is a behavior change introduced by the updated base image and is not specific to a model, profile, or GPU; inference otherwise operates normally. There is no configuration workaround — set max_tokens to bound the response and validate the returned text client-side until a future base image update restores stop-string handling.

For previously documented issues that remain open, refer to Known Issues.

For baseline known issues and limitations, refer to the standard 2.0.4 release notes.

Release 2.0.4-pb6.6#

NIM LLM 2.0.4-pb6.6 is a point release of the NVIDIA AI Enterprise Production Branch 6 (PB6) for NIM LLM. This release uses the standard NIM LLM 2.0.4 documentation set as its baseline and adds PB-specific release information on this page.

Unless a PB-specific update is listed here, the standard 2.0.4 release notes apply.

Highlights#

The following updates are specific to the 2.0.4-pb6.6 point release:

This release resolves 12 CVEs across the base image and PB-specific packages, including nine high severity fixes. Refer to Security Fixes for more information.

Security Fixes#

This release includes the following security fixes:

Security Alerts#

This release includes the following security alerts:

  • Supported model scope: LLM only. PB6 supports large language model (LLM) inference. Vision-language (VLM) / multimodal workloads are not supported. VLM code paths are present in the base image but are not a supported PB6 workload.

  • CVE-2026-44222 (vLLM, Medium): Do not allow untrusted client input with multimodal serving.

  • CVE-2026-27893 (vLLM, High): Do not load VLMs from untrusted sources. Some VLMs automatically enable remote code execution.

  • CVE-2026-9540 (vLLM, Medium): Do not expose the NIM inference endpoint directly to untrusted networks. A flaw involving improper resource shutdown or release in the OpenAI-compatible serving path can be triggered remotely without authentication, causing a denial of service. The vulnerability affects availability but not confidentiality or integrity. No patched vLLM release is available.

Bug Fixes#

No PB-specific bug fixes are documented for this release beyond the security fixes above. For fixes from the baseline release, refer to the standard 2.0.4 release notes.

Support Changes#

The support changes introduced in 2.0.4-pb6.5 remain in effect for this release. Refer to Support Changes for the affected models, profiles, and GPUs.

Known Issues#

No new known issues are documented for this release. Refer to Known Issues for previously documented issues that remain open.

For baseline known issues and limitations, refer to the standard 2.0.4 release notes.

Release 2.0.4-pb6.5#

NIM LLM 2.0.4-pb6.5 is a point release of the NVIDIA AI Enterprise Production Branch 6 (PB6) for NIM LLM. This release uses the standard NIM LLM 2.0.4 documentation set as its baseline and adds PB-specific release information on this page.

Unless a PB-specific update is listed here, the standard 2.0.4 release notes apply.

Highlights#

The following updates are specific to the 2.0.4-pb6.5 point release:

This release resolves 11 CVEs across the base image and PB-specific packages, including two high severity fixes. Refer to Security Fixes for more information.

Security Fixes#

This release includes the following security fixes:

Security Alerts#

This release includes the following security alerts:

  • Supported model scope: LLM only. PB6 supports large language model (LLM) inference. Vision-language (VLM) / multimodal workloads are not supported. VLM code paths are present in the base image but are not a supported PB6 workload.

  • CVE-2026-44222 (vLLM, Medium): Do not allow untrusted client input with multimodal serving.

  • CVE-2026-27893 (vLLM, High): Do not load VLMs from untrusted sources. Some VLMs automatically enable remote code execution.

Bug Fixes#

No PB-specific bug fixes are documented for this release beyond the security fixes above. For fixes from the baseline release, refer to the standard 2.0.4 release notes.

Support Changes#

This release includes the following support changes:

  • model-free-nim-pb6 no longer lists NVIDIA-A10G as a verified GPU. On NVIDIA-A10G, the model-free NIM server fails to start during benchmarking in this release — a regression from 2.0.4-pb6.4, where the same combination started successfully. NVIDIA-A10G is therefore removed from the model-free NIM’s verified-GPU list for 2.0.4-pb6.5; all other verified GPUs are unaffected. The combination can be re-verified in a later release after the underlying startup regression is fixed.

  • nemotron-3-nano-pb6 no longer lists NVIDIA-GB10 as a verified GPU. On NVIDIA-GB10, the nemotron-3-nano-pb6 server fails to start because the model workspace is incomplete — engine initialization aborts with a FileNotFoundError for a missing model safetensors shard, so the server never reaches a ready state. NVIDIA-GB10 is therefore removed from this NIM’s verified-GPU list for 2.0.4-pb6.5; all other verified GPUs are unaffected. The combination can be re-verified in a later release after the underlying startup defect is fixed.

  • llama-3.3-70b-instruct-pb6 excludes 13 public profile/GPU combinations that cannot start in this release. The support matrix removes:

    • BF16 TP1 without LoRA on NVIDIA-GB200

    • BF16 TP2 with and without LoRA on NVIDIA-H100-NVL and NVIDIA-RTX-PRO-6000-Blackwell-Server-Edition

    • BF16 TP8 without LoRA on NVIDIA-A10G

    • BF16 TP1 with LoRA on NVIDIA-GB200

    • BF16 TP8 with LoRA on NVIDIA-RTX-PRO-4500-Blackwell-Server-Edition

    • FP8 TP4 and TP8 with LoRA on NVIDIA-RTX-PRO-4500-Blackwell-Server-Edition

    • NVFP4 TP1 with and without LoRA on NVIDIA-GB10

    • NVFP4 TP4 with LoRA on NVIDIA-RTX-PRO-4500-Blackwell-Server-Edition

  • Some model-free models require a reduced context length or larger GPUs. On NVIDIA-GB10, codestral-class models require a reduced NIM_MAX_MODEL_LEN so that the KV cache fits. On NVIDIA-L40S, Apriel-Nemotron-15B-class models require NIM_MAX_MODEL_LEN=60144. Large-footprint apriel-nemotron and codestral-class models require at least 80 GB of usable VRAM because their approximately 66 GB weight and runtime footprint does not fit on smaller GPUs.

  • Some FP8 TP1 combinations of llama-3.3-nemotron-super-49b-v1.5-pb6 are not supported. The vllm-fp8-tp1-pp1 profile is not supported on NVIDIA-H100-80GB-HBM3; the vllm-fp8-tp1-pp1-lora profile is not supported on NVIDIA-GB10 or NVIDIA-H100-80GB-HBM3. The combinations are omitted from the support matrix.

  • gpt-oss-120b-pb6 requires memory-specific settings on constrained GPUs. On NVIDIA-A100-SXM4-40GB at TP2, use NIM_KVCACHE_PERCENT=0.92. On NVIDIA-A10G, use a lower KV-cache reservation such as NIM_KVCACHE_PERCENT=0.80 so sampler warm-up can complete.

  • LoRA is not supported with the nemotron-3-nano-pb6 NVFP4 profile on NVIDIA Blackwell GPUs. Use a BF16 or FP8 profile when LoRA is required, or use the NVFP4 profile without LoRA. The unsupported combination is not offered in the support matrix.

Known Issues#

The following known issues are specific to the model-free NIM (model-free-nim-pb6) in this release:

  • apriel-nemotron can lose early-stop behavior on /v1/completions on NVIDIA-GB10. When serving an apriel-nemotron model on NVIDIA-GB10, the /v1/completions endpoint may not honor early stopping on some requests, so a response can continue past the point where generation should have stopped. This is a regression from 2.0.4-pb6.4 and is bounded to NVIDIA-GB10; other verified GPUs are unaffected and the NIM otherwise serves the model normally. The chat-completions (/v1/chat/completions) path is not affected.

The following known issues are specific to the llama-3.3-nemotron-super-49b-v1.5-pb6 NIM in this release:

  • logprobs/top_logprobs requests can fail on NVFP4 profiles. On NVFP4 profiles, a request that sets logprobs or top_logprobs can return HTTP 400 with Out of range float values are not JSON compliant: nan. Other profiles are unaffected.

  • include_stop_str_in_output can report an unexpected finish_reason on NVFP4 profiles. On NVFP4 profiles, a request that sets include_stop_str_in_output can return finish_reason "length" where "stop" is expected. Other profiles are unaffected.

  • /v1/responses can time out on NVIDIA-GB10 with the NVFP4 TP1 profile. On NVIDIA-GB10 with the NVFP4 tensor-parallel-1 profile, some /v1/responses requests (including SSE-format responses) can time out. Other GPUs and profiles are unaffected.

  • /v1/responses can fail on requests with empty input content. A POST /v1/responses request that sends an empty input ("") — for example against a LoRA model — can fail instead of returning a valid response or a graceful error. Observed on NVIDIA-B300-SXM6-AC, NVIDIA-H200, and NVIDIA-RTX-PRO-6000-Blackwell-Server-Edition. No workaround is currently available.

The following known issue is specific to llama-3.1-8b-instruct-pb6 in this release:

  • llama-3.1-8b-instruct-pb6 can intermittently stall response generation on the /v1/responses (SSE) and /v1/completions endpoints. On a small minority of requests, the server accepts the request (HTTP 200) and stays healthy, but the generation loop produces no output for about 100 seconds until the client times out. The stall is intermittent and is not tied to a specific GPU or profile: neighboring requests on the same server succeed, the server recovers on its own, and retrying the affected request generally succeeds. It has been observed on NVIDIA-GB10, NVIDIA-GB200, NVIDIA-H100-80GB-HBM3, and NVIDIA-L40S on the fp8 and bf16 TP1 LoRA profiles. As a workaround, set a client-side request timeout and retry the request.

The following known issues are specific to nemotron-3-nano-pb6 in this release:

  • nemotron-3-nano-pb6 can crash during a sustained PEFT/LoRA (bf16) workload on NVIDIA-H100-NVL. Under a sustained PEFT/LoRA stability workload on NVIDIA-H100-NVL with a bf16 profile, the vLLM engine aborts with a CUBLAS_STATUS_EXECUTION_FAILED error in the LM-head GEMM (logits_processor.py), terminating the server mid-run. The NIM otherwise serves the model normally on this GPU; the failure appears under the sustained PEFT stability workload. Other verified GPUs are unaffected.

  • Repeated dynamic LoRA multi-adapter loading and unloading can leave the server unable to restart on NVIDIA-H100-NVL. On NVIDIA-H100-NVL, after several successful runtime LoRA adapter load/unload cycles, a subsequent NIM-server restart can stall while loading model weights (the engine begins loading safetensors checkpoint shards but never reaches a ready state), and the operation eventually times out. The NIM otherwise serves the model and handles individual LoRA adapters normally on this GPU; the issue is specific to repeated dynamic multi-adapter load/unload cycling. No validated workaround is currently available, and the root cause (NIM/serving stack versus test environment) is still under investigation.

For baseline known issues and limitations, refer to the standard 2.0.4 release notes.

Release 2.0.4-pb6.4#

NIM LLM 2.0.4-pb6.4 is a point release of the NVIDIA AI Enterprise Production Branch 6 (PB6) for NIM LLM. This release uses the standard NIM LLM 2.0.4 documentation set as its baseline and adds PB-specific release information on this page.

Unless a PB-specific update is listed here, the standard 2.0.4 release notes apply.

Highlights#

The following updates are specific to the 2.0.4-pb6.4 point release:

This release resolves 25 CVEs across the base image and PB-specific packages, including a critical severity fix in nginx and 11 high severity fixes in Pillow. Refer to Security Fixes for more information.

Security Fixes#

This release includes the following security fixes:

  • nginx — Updated to 1.30.4 to fix CVE-2026-42533 (Critical), CVE-2026-56434 (High), and CVE-2026-60005 (High).

  • Pillow — Updated to 12.3.0 to clear 13 advisories (10 High, 3 Medium) that landed against the 12.2.0 line since the previous release.

  • pyasn1 — Updated to 0.6.4 to fix GHSA-8ppf-4f7h-5ppj and GHSA-hm4w-wwcw-mr6r.

  • Mooncake transfer engine — Rebuilt with updated Go modules: google.golang.org/grpc 1.82.1 (GHSA-hrxh-6v49-42gf), golang.org/x/net 0.56.0 (GO-2026-5942), and golang.org/x/text 0.39.0 (GO-2026-5970).

  • uv — Removed the unused uv distribution from the image, clearing GHSA-4w2j-m93h-cj5j in its bundled quinn-proto crate. Earlier releases removed only the uv/uvx console scripts, which left the distribution metadata in place.

  • System packages — The updated DLFW base clears CVE-2026-50812 and CVE-2026-50813 in libsqlite3-0, and CVE-2026-54411 in libpam-runtime. It also moves openssh-client to the FIPS-validated 13.18+Fips1 build.

  • Base image — The updated DLFW base (26.03.06) carries additional backported fixes in the vLLM inference backend.

Security Alerts#

This release includes the following security alerts:

  • Supported model scope: LLM only. PB6 supports large language model (LLM) inference. Vision-language (VLM) / multimodal workloads are not supported. VLM code paths are present in the base image but are not a supported PB6 workload.

  • CVE-2026-44222 (vLLM, Medium): Do not allow untrusted client input with multimodal serving.

  • CVE-2026-27893 (vLLM, High): Do not load VLMs from untrusted sources. Some VLMs automatically enable remote code execution.

Bug Fixes#

No PB-specific bug fixes are documented for this release beyond the security fixes above. For fixes from the baseline release, refer to the standard 2.0.4 release notes.

Known Issues#

The following known issues from 2.0.4-pb6.2 remain open in this release:

  • ModelScope downloads can fail with error decoding response body on slow or unstable networks. When pulling models from a modelscope:// source over a slow or long-haul link, the transfer can drop a chunked HTTP response mid-stream and abort startup at the download step with an error decoding response body error. This is a transient, recoverable read failure originating from the source rather than a NIM defect. NIM makes a single download attempt by default (NIM_MANIFEST_DOWNLOAD_MAX_RETRY_COUNT=0), so the first failure ends startup. As a workaround, set a non-zero retry count — for example NIM_MANIFEST_DOWNLOAD_MAX_RETRY_COUNT=3 — so NIM retries the download. For details, refer to Reliability on Slow or Unstable Networks in the model download guide.

  • LoRA is not supported with NVFP4-quantized Mixture-of-Experts (MoE) models. For MoE models such as Nemotron-3-Nano, profiles that combine NVFP4 quantization with LoRA (for example, nvfp4 TP1 LoRA) fail to start. vLLM injects LoRA adapters through its Triton-based fused-MoE kernel, but NVFP4 checkpoints dispatch to a specialized NVFP4 experts kernel instead, so engine startup aborts with an AssertionError in vllm/lora/layers/fused_moe.py. This is an upstream vLLM limitation (vLLM 0.17.1). The startup-failure fix is tracked in vLLM PR #34421, and broader native NVFP4 MoE LoRA support (Cutlass kernels) is tracked in vLLM issue #42629. As a workaround, serve these models with a non-NVFP4 profile (for example, BF16 or FP8) when LoRA is required, or use an NVFP4 profile without LoRA.

  • gpt-oss-120b can fail to start on two NVIDIA-A100-SXM4-40GB GPUs with No available memory for the cache blocks. On NVIDIA-A100-SXM4-40GB (40 GB, Ampere) at tensor-parallel 2, gpt-oss-120b’s MXFP4 weights are served through vLLM’s Marlin dequantization kernels (Ampere has no native FP4), whose extra buffers leave too little GPU memory for the KV cache under this release’s default NIM_KVCACHE_PERCENT=0.90 — so the vLLM engine aborts at startup with No available memory for the cache blocks. Reducing the context length (NIM_MAX_MODEL_LEN) does not help, because the shortfall is in the overall memory budget, not the cache size. As a workaround, raise the KV-cache budget with NIM_KVCACHE_PERCENT=0.92, which frees enough memory for the server to start. Larger deployments (tp=4, tp=8) and other GPUs are unaffected.

For baseline known issues and limitations, refer to the standard 2.0.4 release notes.

Release 2.0.4-pb6.3#

NIM LLM 2.0.4-pb6.3 is a point release of the NVIDIA AI Enterprise Production Branch 6 (PB6) for NIM LLM. This release uses the standard NIM LLM 2.0.4 documentation set as its baseline and adds PB-specific release information on this page.

Unless a PB-specific update is listed here, the standard 2.0.4 release notes apply.

Highlights#

The following updates are specific to the 2.0.4-pb6.3 point release:

This release resolves 36 CVEs across the base image and PB-specific packages, including one critical and one high severity fix in the vLLM inference backend. Refer to Security Fixes for more information.

Security Fixes#

This release includes the following security fixes:

  • vLLM — The updated base image includes backported fixes for CVE-2026-48746 (Critical) and CVE-2026-41523 (High) in the vLLM inference backend.

  • System packages — Updated python3.12, curl, gzip, tar, libnghttp2, libtinfo6, and libgcrypt20 to clear known CVEs.

  • Mooncake transfer engine — Rebuilt with Go 1.25.12 to fix CVE-2026-39822, a symlink-escape flaw in the bundled Go standard library.

  • wget — Removed the unused wget package from the image to reduce the container’s vulnerability surface, clearing four Medium-severity CVEs (CVE-2026-58469, CVE-2026-58470, CVE-2026-58471, and CVE-2026-58472).

Security Alerts#

This release includes the following security alerts:

  • Supported model scope: LLM only. PB6 supports large language model (LLM) inference. Vision-language (VLM) / multimodal workloads are not supported. VLM code paths are present in the base image but are not a supported PB6 workload.

  • CVE-2026-44222 (vLLM, Medium): Do not allow untrusted client input with multimodal serving.

  • CVE-2026-27893 (vLLM, High): Do not load VLMs from untrusted sources. Some VLMs automatically enable remote code execution.

Bug Fixes#

No PB-specific bug fixes are documented for this release beyond the security fixes above. For fixes from the baseline release, refer to the standard 2.0.4 release notes.

Known Issues#

The following known issues from 2.0.4-pb6.2 remain open in this release:

  • ModelScope downloads can fail with error decoding response body on slow or unstable networks. When pulling models from a modelscope:// source over a slow or long-haul link, the transfer can drop a chunked HTTP response mid-stream and abort startup at the download step with an error decoding response body error. This is a transient, recoverable read failure originating from the source rather than a NIM defect. NIM makes a single download attempt by default (NIM_MANIFEST_DOWNLOAD_MAX_RETRY_COUNT=0), so the first failure ends startup. As a workaround, set a non-zero retry count — for example NIM_MANIFEST_DOWNLOAD_MAX_RETRY_COUNT=3 — so NIM retries the download. For details, refer to Reliability on Slow or Unstable Networks in the model download guide.

  • LoRA is not supported with NVFP4-quantized Mixture-of-Experts (MoE) models. For MoE models such as Nemotron-3-Nano, profiles that combine NVFP4 quantization with LoRA (for example, nvfp4 TP1 LoRA) fail to start. vLLM injects LoRA adapters through its Triton-based fused-MoE kernel, but NVFP4 checkpoints dispatch to a specialized NVFP4 experts kernel instead, so engine startup aborts with an AssertionError in vllm/lora/layers/fused_moe.py. This is an upstream vLLM limitation (vLLM 0.17.1). The startup-failure fix is tracked in vLLM PR #34421, and broader native NVFP4 MoE LoRA support (Cutlass kernels) is tracked in vLLM issue #42629. As a workaround, serve these models with a non-NVFP4 profile (for example, BF16 or FP8) when LoRA is required, or use an NVFP4 profile without LoRA.

  • gpt-oss-120b can fail to start on two NVIDIA-A100-SXM4-40GB GPUs with No available memory for the cache blocks. On NVIDIA-A100-SXM4-40GB (40 GB, Ampere) at tensor-parallel 2, gpt-oss-120b’s MXFP4 weights are served through vLLM’s Marlin dequantization kernels (Ampere has no native FP4), whose extra buffers leave too little GPU memory for the KV cache under this release’s default NIM_KVCACHE_PERCENT=0.90 — so the vLLM engine aborts at startup with No available memory for the cache blocks. Reducing the context length (NIM_MAX_MODEL_LEN) does not help, because the shortfall is in the overall memory budget, not the cache size. As a workaround, raise the KV-cache budget with NIM_KVCACHE_PERCENT=0.92, which frees enough memory for the server to start. Larger deployments (tp=4, tp=8) and other GPUs are unaffected.

For baseline known issues and limitations, refer to the standard 2.0.4 release notes.

Release 2.0.4-pb6.2#

NIM LLM 2.0.4-pb6.2 is a point release of the NVIDIA AI Enterprise Production Branch 6 (PB6) for NIM LLM. This release uses the standard NIM LLM 2.0.4 documentation set as its baseline and adds PB-specific release information on this page.

Unless a PB-specific update is listed here, the standard 2.0.4 release notes apply.

Highlights#

The following updates are specific to the 2.0.4-pb6.2 point release:

  • Updated the DLFW vLLM base image to 26.03.04-py3-stage (from 26.03.03-py3-stage). This base includes an upgrade of DALI to 2.1.1 to resolve an FFMPEG licensing issue.

  • Lowered the default NIM_KVCACHE_PERCENT (GPU memory fraction for the KV cache) from 0.92 to 0.9, matching the default of the vLLM 0.17.1 backend that the PB image runs. The previous 0.92 was inherited from the feature branch build (vLLM 0.20.0) and budgeted GPU memory too aggressively for PB, causing unexpected out-of-memory errors during CUDA graph capture on some hardware. You can still override it explicitly with NIM_KVCACHE_PERCENT.

Security Fixes#

This release includes the following security fixes:

  • nginx — Updated from 1.30.1 to 1.30.3, fixing CVE-2026-9256 (Critical) — a heap overflow in ngx_http_rewrite_module.

  • Python packages — Upgraded aiohttp, cryptography, msgpack, python-multipart, PyJWT, pydantic-settings, and uv to clear known CVEs, including High-severity issues in cryptography (GHSA-537c-gmf6-5ccf), msgpack (GHSA-6v7p-g79w-8964), python-multipart (GHSA-5rvq-cxj2-64vf), and PyJWT (GHSA-xgmm-8j9v-c9wx).

  • System packages — Updated libsqlite3-0 to clear Medium-severity CVEs.

  • Base image (DLFW 26.03.04) — Removed the unused vim package to reduce the container’s vulnerability surface.

Security Alerts#

This release includes the following security alerts:

  • Supported model scope: LLM only. PB6 supports large language model (LLM) inference. Vision-language (VLM) / multimodal workloads are not supported. VLM code paths are present in the base image but are not a supported PB6 workload.

  • CVE-2026-44222 (vLLM 0.17.1): Do not allow untrusted client input with multimodal serving.

  • CVE-2026-27893 (vLLM 0.17.1): Do not load VLMs from untrusted sources. Some VLMs (for example, Kimi-K2.5 and NemotronVL) automatically enable remote code execution.

Bug Fixes#

No PB-specific bug fixes are documented for this release beyond the security fixes above. For fixes from the baseline release, refer to the standard 2.0.4 release notes.

Known Issues#

This release includes the following known issues:

  • ModelScope downloads can fail with error decoding response body on slow or unstable networks. When pulling models from a modelscope:// source over a slow or long-haul link, the transfer can drop a chunked HTTP response mid-stream and abort startup at the download step with an error decoding response body error. This is a transient, recoverable read failure originating from the source rather than a NIM defect. NIM makes a single download attempt by default (NIM_MANIFEST_DOWNLOAD_MAX_RETRY_COUNT=0), so the first failure ends startup. As a workaround, set a non-zero retry count — for example NIM_MANIFEST_DOWNLOAD_MAX_RETRY_COUNT=3 — so NIM retries the download. For details, refer to Reliability on Slow or Unstable Networks in the model download guide.

  • LoRA is not supported with NVFP4-quantized Mixture-of-Experts (MoE) models. For MoE models such as Nemotron-3-Nano, profiles that combine NVFP4 quantization with LoRA (for example, nvfp4 TP1 LoRA) fail to start. vLLM injects LoRA adapters through its Triton-based fused-MoE kernel, but NVFP4 checkpoints dispatch to a specialized NVFP4 experts kernel instead, so engine startup aborts with an AssertionError in vllm/lora/layers/fused_moe.py. This is an upstream vLLM limitation (vLLM 0.17.1). The startup-failure fix is tracked in vLLM PR #34421, and broader native NVFP4 MoE LoRA support (Cutlass kernels) is tracked in vLLM issue #42629. As a workaround, serve these models with a non-NVFP4 profile (for example, BF16 or FP8) when LoRA is required, or use an NVFP4 profile without LoRA.

  • gpt-oss-120b can fail to start on two NVIDIA-A100-SXM4-40GB GPUs with No available memory for the cache blocks. On NVIDIA-A100-SXM4-40GB (40 GB, Ampere) at tensor-parallel 2, gpt-oss-120b’s MXFP4 weights are served through vLLM’s Marlin dequantization kernels (Ampere has no native FP4), whose extra buffers leave too little GPU memory for the KV cache under this release’s default NIM_KVCACHE_PERCENT=0.90 — so the vLLM engine aborts at startup with No available memory for the cache blocks. Reducing the context length (NIM_MAX_MODEL_LEN) does not help, because the shortfall is in the overall memory budget, not the cache size. As a workaround, raise the KV-cache budget with NIM_KVCACHE_PERCENT=0.92, which frees enough memory for the server to start. Larger deployments (tp=4, tp=8) and other GPUs are unaffected.

For baseline known issues and limitations, refer to the standard 2.0.4 release notes.

Release 2.0.4-pb6.1#

NIM LLM 2.0.4-pb6.1 is a point release of the NVIDIA AI Enterprise Production Branch 6 (PB6) for NIM LLM. This release uses the standard NIM LLM 2.0.4 documentation set as its baseline and adds PB-specific release information on this page.

Unless a PB-specific update is listed here, the standard 2.0.4 release notes apply.

Highlights#

The following updates are specific to the 2.0.4-pb6.1 point release:

  • Updated the DLFW vLLM base image to 26.03.03-py3-stage (from 26.03.02-py3-stage). The inference backend remains at vLLM 0.17.1.

  • Reduced the container’s vulnerability surface by removing unused multimedia components from the PB image (see Security Fixes).

The following updates apply to the PB6 documentation set:

  • Updated the Support Matrix with the PB6 model-free and model-specific NIMs.

  • Clarified that PB releases provide stable and secure NVIDIA AI Enterprise release branches for mission-critical deployments.

  • Updated OpenShift SecurityContextConstraints (SCC) guidance for restricted-v2 and restricted-v3.

  • Added guidance for PB6 air-gapped deployments, including runtime cache and telemetry control environment variables.

  • Corrected the documented default for NIM_KVCACHE_PERCENT (GPU memory fraction for the KV cache) to 0.92. Earlier documentation listed it as 0.9.

Security Fixes#

This release includes the following security fixes:

  • nginx — Updated from 1.29.7 to 1.30.1, which fixes the following CVEs:

    • CVE-2026-42945 — “NGINX Rift” RCE in ngx_http_rewrite_module (Critical)

    • CVE-2026-42926, CVE-2026-42934, CVE-2026-42946, CVE-2026-40460, and CVE-2026-40701 (Medium)

  • Mooncake transfer engine — Rebuilt with Go 1.25.11 and upgraded the vulnerable Go module dependencies in the bundled libetcd_wrapper.so:

    • google.golang.org/grpc upgraded to 1.79.3 — gRPC-Go :path authorization bypass (CVE-2026-33186 / GHSA-p77j-4mvh-x3m3)

    • golang.org/x/net upgraded to 0.55.0 — IDNA hostname-validation bypass (CVE-2026-39821 / GO-2026-5026)

    • Go 1.25.11 toolchain also clears CVE-2025-68121, CVE-2025-61732, CVE-2026-25679, and CVE-2026-27142

  • System multimedia libraries — Removed the base image’s system ffmpeg libraries (libavcodec60, libavformat60, libswscale7) and opencv-python-headless from the PB image to reduce CVE and licensing surface. Hardware video decoding and VLM video input are not supported on PB6.

Security Alerts#

This release includes the following security alerts:

  • CVE-2026-42311 (Pillow 12.1.1): For vision language model (VLM) deployments, do not allow untrusted image sources with multimodal serving.

  • CVE-2026-44222 (vLLM 0.17.1): Do not allow untrusted client input with multimodal serving.

  • CVE-2026-27893 (vLLM 0.17.1): Do not load VLMs from untrusted sources. Some VLMs (for example, Kimi-K2.5 and NemotronVL) automatically enable remote code execution.

Bug Fixes#

This release includes the following bug fix:

  • Startup banner now displays the release-appropriate license. The container startup banner reads the container and model license and notice text directly from the on-disk files instead of a static built-in string, so the displayed governing terms match the specific release and bundled model.

For fixes from the baseline release, refer to the standard 2.0.4 release notes.

Known Issues#

This release includes the following known issues:

  • ModelScope downloads can fail with error decoding response body on slow or unstable networks. When pulling models from a modelscope:// source over a slow or long-haul link, the transfer can drop a chunked HTTP response mid-stream and abort startup at the download step with an error decoding response body error. This is a transient, recoverable read failure originating from the source rather than a NIM defect, but NIM makes a single download attempt by default (NIM_MANIFEST_DOWNLOAD_MAX_RETRY_COUNT=0), so the first failure ends startup. As a workaround, set a non-zero retry count — for example NIM_MANIFEST_DOWNLOAD_MAX_RETRY_COUNT=3 — so NIM retries the download. For details, see Reliability on Slow or Unstable Networks in the model download guide.

  • LoRA is not supported with NVFP4-quantized Mixture-of-Experts (MoE) models. For MoE models such as Nemotron-3-Nano, profiles that combine NVFP4 quantization with LoRA (for example, nvfp4 TP1 LoRA) fail to start. vLLM injects LoRA adapters through its Triton-based fused-MoE kernel, but NVFP4 checkpoints dispatch to a specialized NVFP4 experts kernel instead, so engine startup aborts with an AssertionError in vllm/lora/layers/fused_moe.py. This is an upstream vLLM limitation (vLLM 0.17.1). The startup-failure fix is tracked in vLLM PR #34421, and broader native NVFP4 MoE LoRA support (Cutlass kernels) is tracked in vLLM issue #42629. As a workaround, serve these models with a non-NVFP4 profile (for example, BF16 or FP8) when LoRA is required, or use an NVFP4 profile without LoRA.

  • The default NIM_KVCACHE_PERCENT of 0.92 can cause out-of-memory (OOM) errors on some profiles during CUDA graph capture or warmup. The default budgets 92% of GPU memory for model operations, leaving 8% unreserved. On some profiles — particularly smaller GPUs — this is not enough headroom for CUDA graph capture, which can consume 1 to 5 GB. The OOM appears after weights and KV cache are allocated, during graph capture or sampler warmup. As a workaround, lower NIM_KVCACHE_PERCENT (for example, to 0.85) to leave more memory unreserved, or disable CUDA graphs with NIM_DISABLE_CUDA_GRAPH=1. For details, see During CUDA Graph Compilation or Warmup in the memory troubleshooting guide.

For baseline known issues and limitations, refer to the standard 2.0.4 release notes.

Release 2.0.4-pb6#

NIM LLM 2.0.4-pb6 is the NVIDIA AI Enterprise Production Branch 6 (PB6) release of NIM LLM. This release uses the standard NIM LLM 2.0.4 documentation set as its baseline and adds PB-specific release information on this page.

Unless a PB-specific update is listed here, the standard 2.0.4 release notes apply.

Highlights#

The following updates are specific to the 2.0.4-pb6 documentation set:

  • Updated the Support Matrix with the PB6 model-free and model-specific NIMs.

  • Clarified that PB releases provide stable and secure NVIDIA AI Enterprise release branches for mission-critical deployments.

  • Updated OpenShift SecurityContextConstraints (SCC) guidance for restricted-v2 and restricted-v3.

  • Added guidance for PB6 air-gapped deployments, including runtime cache and telemetry control environment variables.

Security Alerts#

This release includes the following security alerts:

  • CVE-2026-42311 (Pillow 12.1.1): For vision language model (VLM) deployments, do not allow untrusted image sources with multimodal serving.

  • CVE-2026-44222 (vLLM 0.17.1): Do not allow untrusted client input with multimodal serving.

  • CVE-2026-27893 (vLLM 0.17.1): Do not load VLMs from untrusted sources. Some VLMs (for example, Kimi-K2.5 and NemotronVL) automatically enable remote code execution.

Bug Fixes#

No PB-specific known issues are documented as fixed in this release. For fixes from the baseline release, refer to the standard 2.0.4 release notes.

Known Issues#

No PB-specific known issues are documented for this release. For baseline known issues and limitations, refer to the standard 2.0.4 release notes.