NVIDIA Optimized Frameworks
NVIDIA Docs Hub Homepage  NVIDIA Optimized Frameworks  NVIDIA Optimized Frameworks  vLLM Release 26.07
Download PDF

vLLM Release 26.07

The NVIDIA vLLM Release 26.07 is made up of two container images available on NGC: vLLM.

Contents of the vLLM container

This container image contains the complete source of the version of vLLM in /opt/vllm. It is pre-built and installed in the default system Python environment (/usr/local/lib/python3.12/dist-packages/vllm) in the container image. Visit vLLM Docs to learn more about vLLM.

The NVIDIA vLLM Container is optimized for use with NVIDIA GPUs, and contains the following software for GPU acceleration

  • Please see to the CUDA section for the list of libraries inherited from the CUDA container.
  • vLLM: 0.24.0
  • flashinfer 0.6.14
  • transformers 5.6.1
  • flash-attention 2.7.4.post1
  • xgrammar 0.2.0
  • Torch 2.13.0a0+9186a08b2c

Driver Requirements

Release 26.07 is based on CUDA 13.3.1 For comprehensive and up-to-date driver compatibility information, please refer to the following documentation:

Key Features and Enhancements

This vLLM release includes the following key features and enhancements.

  • The GCP NCCL plugin package is now installed in the container. On Google Cloud Platform hosts (A3 Ultra, A4, A4X and A4X Max machine types ), the container automatically detects and activates the GCP NCCL network, tuner, profiler, and environment plugins for optimized multi-node communication.
    • On GCP hosts with the GCP NCCL plugins installed, NCCL telemetry is enabled by default starting in the 26.07 release.
    • A runtime advisory message is displayed when running on GCP hosts with the GCP NCCL plugins active, directing users to the GCP NCCL telemetry documentation.
  • Support Nemotron Super V3
  • Support Nemotron 3 Nano Omni
  • Support DeepSeek V4
  • Nemotron NVFP4 Mixture-of-Experts inference on SM100/SM103 (Blackwell) now uses the FlashInfer CUTLASS MoE backend, preventing autotune crashes and ensuring correct output on these architectures.
  • FlashInfer autotune has been re-enabled; the workaround that disabled it in the prior cycle is no longer needed.
  • Added manylinux_2_34 build support for the container's Python wheels.

Announcements

Known Issues

  • vLLM serve uses aggressive GPU memory allocation by default (effectively --gpu-memory-utilization≈1.0). On systems with shared/unified GPU memory (e.g. DGX Spark or Jetson platforms), this can lead to out-of-memory errors. If you encounter OOM, start vllm serve with a lower utilization value, for example: vllm serve <model> --gpu-memory-utilization 0.7.

  • When running Nemotron Nano V3 or Nemotron Super V3 NVFP4 models on Spark it is required to limit the number of sequences to 4:
    • vllm serve <model> --max-num-seqs 4&
  • Security: Restriction header values are now redacted from the OpenAI frontend startup log, and the inference restriction is enforced in the SageMaker MME invoke handler.
  • For NVFP4 MoE models on x86 Blackwell (B200/B300) force the CUTLASS MoE backend: vllm serve <model> --moe-backend flashinfer_cutlass.
© Copyright 2026, NVIDIA. Last updated on Jul 29, 2026
content here