How to Use cuBLASMp

This section explains how to use cuBLASMp in your application.

• Communication Abstraction Library Usage
  • Communication Abstraction Library
  • Creating Communicator Handle with MPI
  • Communication Abstraction Library Data Types
  • Communication Abstraction Library API
• Using cuBLASMp for Tensor Parallelism in Distributed Machine Learning
  • AllGather+GEMM and GEMM+ReduceScatter in terms of traditional PBLAS
  • On Python and cuBLASMp data ordering
  • AllGather+GEMM
  • GEMM+ReduceScatter
  • General assumptions and limitations
• cuBLASMp Logging
  • CUBLASMP_LOG_LEVEL
  • CUBLASMP_LOG_MASK
  • CUBLASMP_LOG_FILE
• cuBLASMp Data Types
  • Data types
  • Enumerators
• cuBLASMp C API
  • Library Management
  • Grid Management
  • Matrix Management
  • Matmul Properties
  • Utility
  • Logging
  • Dense Linear Algebra APIs