Using NCCL

Using NCCL is similar to using any other library in your code:

1. Install the NCCL library on your system
2. Modify your application to link to that library
3. Include the header file nccl.h in your application
4. Create a communicator (see Creating a Communicator)
5. Use NCCL collective communication primitives to perform data communication. You can familiarize yourself with the NCCL API documentation to maximize your usage performance.

Collective communication primitives are common patterns of data transfer among a group of CUDA devices. A communication algorithm involves many processors that are communicating together. Each CUDA device is identified within the communication group by a zero-based index or rank. Each rank uses a communicator object to refer to the collection of GPUs that are intended to work together. The creation of a communicator is the first step needed before launching any communication operation.

• Creating a Communicator
  • Creating a communicator with options
  • Creating a communicator using multiple ncclUniqueIds
  • Shrinking a communicator
  • Creating more communicators
  • Using multiple NCCL communicators concurrently
  • Finalizing a communicator
  • Destroying a communicator
• Error handling and communicator abort
  • Asynchronous errors and error handling
• Fault Tolerance
• Quality of Service
• Collective Operations
  • AllReduce
  • Broadcast
  • Reduce
  • AllGather
  • ReduceScatter
  • AlltoAll
  • Gather
  • Scatter
• Data Pointers
• CUDA Stream Semantics
  • Mixing Multiple Streams within the same ncclGroupStart/End() group
• Group Calls
  • Management Of Multiple GPUs From One Thread
  • Aggregated Operations (2.2 and later)
  • Group Operation Ordering Semantics
  • Nonblocking Group Operation
• Point-to-point communication
  • Sendrecv
  • One-to-all (scatter)
  • All-to-one (gather)
  • All-to-all
  • Neighbor exchange
• Thread Safety
• In-place Operations
• Using NCCL with CUDA Graphs
• User Buffer Registration
  • NVLink Sharp Buffer Registration
  • IB Sharp Buffer Registration
  • General Buffer Registration
  • Buffer Registration and PXN
  • Memory Allocator
  • Window Registration
  • Zero-CTA Optimization
• Device-Initiated Communication
  • Device API
  • Host-Side Setup
  • Simple Device Kernel
  • Multimem Device Kernel
  • Thread Groups
  • Teams