How to Use cuBLASMp# This section explains how to use cuBLASMp in your application. Communication Abstraction Library Usage Communication Abstraction Library Creating Communicator Handle with MPI Communication Abstraction Library Data Types Communication Abstraction Library API Using cuBLASMp for Tensor Parallelism in Distributed Machine Learning AllGather+GEMM and GEMM+ReduceScatter in terms of traditional PBLAS On Python and cuBLASMp data ordering AllGather+GEMM GEMM+ReduceScatter GEMM+AllReduce General assumptions and limitations cuBLASMp Logging CUBLASMP_LOG_LEVEL CUBLASMP_LOG_MASK CUBLASMP_LOG_FILE cuBLASMp Data Types Data types Enumerators cuBLASMp C API Library Management Grid Management Matrix Management Matmul Properties Utility Logging Dense Linear Algebra APIs
