NVIDIA Deep Learning NCCL Documentation
Release Notes (PDF) - v2.21.5 - Last updated June 18, 2024

NCCL Release 2.18.5

This is the NCCL 2.18.5 release notes. For previous NCCL release notes, refer to the NCCL Archives.

Compatibility

NCCL 2.18.5 has been tested with the following:

Fixed Issues

The following issues have been resolved in NCCL 2.18.5:

  • Fixed NVLS search issues.

  • Increased Max IB network interfaces to 32.

  • Fixed inconsistent network device ordering when creating communicators with only one GPU per node.

  • Try to have different GPUs use all network interfaces on systems with more than one network interface per GPU.

Known Issues

  • Send/receive communication using CUDA_VISIBLE_DEVICES and PXN only works if the GPU mappings to local ranks is the same across nodes. Disabing PXN for Send/Receive communication can workaround the issue (NCCL_P2P_PXN_LEVEL=0).

Updating the GPG Repository Key

To best ensure the security and reliability of our RPM and Debian package repositories, NVIDIA is updating and rotating the signing keys used by apt, dnf/yum, and zypper package managers beginning on April 27, 2022. Failure to update your repository signing keys will result in package management errors when attempting to access or install NCCL packages. To ensure continued access to the latest NCCL release, please follow the updated NCCL installation guide.