Getting Started#
Introduction#
cudaMallocAsync()
, cudaFreeAsync()
, cudaMemcpyAsync()
, etc.Hardware and Software Requirements#
GPU Architectures |
Volta ( |
CUDA |
12.2.2 and above |
CPU architectures |
x86_64, arm64-sbsa |
Operating System |
Linux |
Recommended NVIDIA InfiniBand solutions for accelerated inter-node communication
Required Packages
CUDA Toolkit 12.2.2 and above (https://developer.nvidia.com/cuda-downloads)
HPC-X v2.17.1 and above (https://developer.nvidia.com/networking/hpc-x) - contains OpenUCC and OpenUCX that satisfy cuBLASMp requirements.
NCCL v2.18.5 and above (https://developer.nvidia.com/nccl) - required to achieve better performance
NVSHMEM v3.1 and above (https://developer.nvidia.com/nvshmem)
Recommended Packages
GDRCopy v2.0+ (NVIDIA/gdrcopy) and nv_peer_mem (Mellanox/nv_peer_memory) - Allows underlying communication packages to use GPUDirect RDMA. If you install OpenUCX yourself, it should be configured with GDRCopy support.
Mellanox OFED (https://www.mellanox.com/products/infiniband-drivers/linux/mlnx_ofed) - drivers for NVIDIA Infiniband Adapters (https://www.nvidia.com/en-us/networking/products/infiniband). If you install OpenUCX yourself, it should be configured with IB communications support.
Synchronous Execution#
Data Layout of Local Matrices#
Workflow#
1. Bootstrap CAL communicator: cal_comm_create().2. Initialize the library handle: cublasMpCreate().3. Initialize grid descriptors: cublasMpGridCreate().4. Initialize matrix descriptors: cublasMpMatrixDescriptorCreate().5. Query the host and device buffer sizes for a given routine.6. Allocate host and device workspace buffers for a given routine.7. Execute the routine to perform the desired computation.8. Synchronize local stream to make sure the result is available, if required: cal_stream_sync().9. Deallocate host and device workspace.10. Destroy matrix descriptors: cublasMpMatrixDescriptorDestroy().11. Destroy grid descriptors: cublasMpGridDestroy().12. Destroy cuBLASMp library handle: cublasMpDestroy().13. Destroy CAL library handle: cal_comm_destroy().