Is this page helpful?

Release Notes#

cuBLASMp v0.11.0#

Released: October 28, 2026

New features#

  • Added device-side alpha/beta support for cublasMpMatmul via the CUBLASMP_MATMUL_DESCRIPTOR_ATTRIBUTE_POINTER_MODE attribute (cublasMpPointerMode_t). In CUBLASMP_POINTER_MODE_DEVICE the alpha and beta scalars are read from device memory on the stream, avoiding a device-to-host copy and synchronization when they are produced by a preceding GPU kernel. Device pointer mode supports null C, including mixed C/D types, and ignores beta in that case. The handle must outlive captured graphs that use device pointer mode. Explicit FUSED Matmul + AllReduce requests return CUBLASMP_STATUS_NOT_SUPPORTED in device pointer mode.

  • Added GB300 (Compute Capability 10.3) support for fused Matmul + AllReduce.

  • AllGather + cublasMpMatmul now leaves the gathered B at the start of the device workspace, followed by the gathered B scale tensors when B uses block or vector scaling. Applications can reuse the gathered operand in later operations instead of gathering it again; see Reusing the Gathered B.

  • FP8 cublasMpMatmul for regular 2D block-cyclic layouts in which no single process holds all of sub(A), sub(B), sub(C), and sub(D) now supports FP8 D, the AMAX of D, every combination of TRANSA and TRANSB, and matrix dimensions, block sizes, and submatrix offsets that are not multiples of 16, with the default algorithm, the default epilogue, and scalar or no scaling factors.

Performance improvements#

  • Improved performance of cublasMpMatmul and cublasMpGemm for regular 2D block-cyclic layouts in which one input matrix is much larger than the output, such as n much smaller than m and k. Results of these calls can differ from earlier releases in the last bits because of a different summation order, and the workspace size returned by the corresponding _bufferSize function can be larger.

  • Improved performance of cublasMpMatmul and cublasMpGemm for regular 2D block-cyclic layouts in which A or B uses a different process-grid mapping than C, and for complex inputs with one transposed and one conjugate-transposed operand.

  • Removed redundant device-to-device output staging from cublasMpTradd diagonal updates.

  • Improved performance of cublasMpMatmul and cublasMpGemm for regular 2D block-cyclic layouts with more than one block of the inner dimension. The workspace size returned by the corresponding _bufferSize function can be larger.

  • Improved performance of FP8 cublasMpMatmul for regular 2D block-cyclic layouts in which no single process holds all of sub(A), sub(B), sub(C), and sub(D). Results of these calls can differ from earlier releases, because partial sums are no longer rounded to the type of D before the final result, and the workspace size returned by cublasMpMatmul_bufferSize can be larger.

  • Improved performance of Matmul + ReduceScatter and Matmul + AllReduce, most with the default epilogue and scalar or no C and D scaling factors, and of FP8 cublasMpMatmul for regular 2D block-cyclic layouts in which no single process holds all of sub(A), sub(B), sub(C), and sub(D).

Bug fixes#

  • Fixed GELU AUX-AMAX in Matmul + ReduceScatter and Matmul + AllReduce to report the maximum before AUX scaling, matching cuBLASLt.

  • Fixed incorrect results of FP8 cublasMpMatmul with A, B, or C scaling factors for regular 2D block-cyclic layouts in which no single process holds all of sub(A), sub(B), sub(C), and sub(D). The bug existed in 0.10.0.

  • Fixed FP8 cublasMpMatmul for regular 2D block-cyclic layouts in which no single process holds all of sub(A), sub(B), sub(C), and sub(D) hanging, or failing on only some ranks, for example with the NT format or with dimensions that are not multiples of 16. Such configurations are now supported (see New features), and the remaining unsupported ones, such as type combinations that cuBLASLt does not support, return CUBLASMP_STATUS_NOT_SUPPORTED on every rank. The bug existed in 0.10.0.

  • Fixed cublasMpMatmul and cublasMpGemm with a transposed or conjugate-transposed input, and cublasMpGeadd with a transposition, returning CUBLASMP_STATUS_NOT_SUPPORTED on a rank that passed null pointers for matrices it holds no elements of, while the other ranks succeeded. Matmul and GEMM could hang when that rank held part of the output. The bug existed in 0.10.0.

  • Fixed cublasMpSetStream reading freed memory when the previously bound stream had been destroyed after its work completed: the same-context validation interrogated the previous stream instead of the handle’s CUDA context, which could crash or silently read stale memory depending on heap reuse. The previously bound stream is no longer accessed when rebinding, and cublasMpCreate now rejects a stream from a different CUDA context with CUBLASMP_STATUS_INVALID_VALUE. The bug existed in all previous versions with this API (0.3.0 and later).

  • Fixed incorrect or non-finite results in Hopper FP8 fused Matmul + AllReduce with VEC128/VEC128, VEC128/BLK128x128, or BLK128x128/VEC128 input scaling.

  • Fixed cublasMpTradd to bound lower-triangular diagonal-tile accesses by the actual column count on rectangular tail tiles.

  • Fixed cublasMpTradd to skip empty off-diagonal updates after partial final diagonal tiles instead of passing nonpositive extents to cublasMpGeadd.

  • Fixed multicast Matmul reductions with non-16-byte-aligned internal chunks to reject explicit multicast requests and use a safe fallback when the default algorithm is selected.

  • Fixed explicit Matmul algorithm requests to return CUBLASMP_STATUS_NOT_SUPPORTED instead of silently using another implementation when the requested algorithm is unavailable.

  • Fixed incorrect VEC16, VEC32, VEC128, and BLK128x128 matrix B scale-tensor slicing in split Matmul + ReduceScatter and Matmul + AllReduce algorithms. Explicit split requests now accept independently padded cuBLASLt B-scale chunks for VEC16/VEC32/VEC128, supporting partial 128-column pages and multiple local VEC128 K blocks without extra library workspace. Such callers must supply the new chunk-packed layout with one chunk per rank for both operations, independent of AllReduce split-count environment tuning; DEFAULT and NO_OVERLAP keep the full rank-local scale layout. BLK128x128 split restrictions remain unchanged.

  • Removed unnecessary B-scale column-chunk restrictions from deprecated atomic Matmul + ReduceScatter, which uses one full-matrix GEMM rather than independent per-chunk GEMMs.

  • Fixed 2D block-cyclic Matmul workspace sizing for transposed operands with rectangular matrix block sizes.

  • Fixed B-scale chunk offsets when the default split Matmul algorithm operates on a submatrix of a wider B descriptor.

  • Fixed no-overlap Matmul + ReduceScatter to use the requested GEMM dimensions with oversized A/B descriptors.

  • Fixed cublasMpMatmul computing incorrect results and writing outside sub(D) when a submatrix ends within a block of a larger matrix, for regular 2D block-cyclic layouts in which A or B uses a different process-grid mapping than C, for complex inputs with one transposed and one conjugate-transposed operand, and for FP8 inputs. The bug existed in 0.10.0.

  • Fixed FP8/FP4 Matmul auxiliary-buffer type handling so all algorithm families follow the loaded cuBLASLt version’s default when EPILOGUE_AUX_DATA_TYPE is unset, without constraining epilogues that do not use an auxiliary buffer. As in cuBLASLt, a non-bitmask auxiliary epilogue is unsupported when the resolved auxiliary type is CUDA_R_16BF and CType is CUDA_R_16F; users can select an explicitly supported auxiliary type instead. ReLU bit-mask epilogues now reject auxiliary data types other than CUDA_R_8U.

  • Fixed Matmul + AllReduce and Matmul + ReduceScatter post-processing for outputs with more than 1,048,560 local columns.

cuBLASMp v0.10.0#

Released: July 13, 2026

New features#

  • Added CUBLASMP_MATMUL_ALGO_TYPE_FUSED for Matmul + AllReduce on supported NVSwitch systems.

  • Added single-process multi-GPU (SPMG) support (CUDA 13+), allowing one host process to drive multiple GPUs.

Performance improvements#

  • Improved performance of cublasMpMatmul, including Matmul + AllReduce, Matmul + ReduceScatter, and general 2D block-cyclic layouts.

  • Improved performance of ReLU epilogues with auxiliary output.

Bug fixes#

  • Fixed correctness issues in cublasMpMatmul, including conjugate-transposed operands, beta == 0 handling, block scaling, and bias-gradient epilogues in Matmul + AllReduce.

  • Fixed cublasMpMatmul support for regular 2D block-cyclic inputs whose descriptors use distinct grid objects with the same process-grid mapping.

  • Fixed the fallback P2P Matmul path to accept A/B descriptors on same-parent grids with different process-grid mappings when C/D output tiles are co-located.

  • Fixed correctness issues in symmetric/Hermitian rank-k routines, triangular routines, matrix redistribution, and matrix-add routines across complex scalars, empty-k updates, small blocks, large matrices, and sub-matrix edge cases.

  • Improved workspace size reporting and workspace validation across several APIs.

  • Improved validation and error reporting across the library so unsupported inputs and internal failures return error statuses more consistently.

cuBLASMp v0.9.1#

Released: June 4, 2026

New features#

  • cublasMpTrsm supports more flexible sub-matrix inputs, including non-tile-aligned offsets and independent A/B block sizes.

  • cublasMpMatmul has improved performance for additional matrix layouts that previously used a slower fallback implementation.

  • cublasMpGeadd has improved performance and lower workspace requirements for redistribution cases.

Bug fixes#

  • Fixed correctness issues for operations on embedded sub-matrices in cublasMpTrsm, symmetric/Hermitian rank-k routines, cublasMpTrmm, and cublasMpSymm.

  • Fixed edge-case correctness and validation issues in cublasMpMatmul, including null C inputs with beta == 0.

  • Fixed cublasMpGeadd and cublasMpTradd edge cases involving in-place operation and non-square block sizes.

  • Improved input validation across several APIs so invalid arguments return CUBLASMP_STATUS_INVALID_VALUE more consistently.

cuBLASMp v0.9.0#

Released: May 12, 2026

New features#

Bug fixes#

  • Fixed incorrect beta * C results in Matmul + AllReduce when beta != 0.

  • Fixed epilogue correctness in Matmul + AllReduce.

cuBLASMp v0.8.1#

Released: March 24, 2026

New features#

  • Added COMMUNICATION_TYPE support to all Matmul + ReduceScatter variants.

  • Added d_out_scale support for all Matmul + ReduceScatter variants.

  • Added FP4 output support to Matmul + ReduceScatter with the NO_OVERLAP algorithm.

  • Added DRELU and DRELU_BGRAD epilogues support to AllGather + Matmul, Matmul + ReduceScatter, and Matmul + AllReduce variants.

  • Reduced workspace size required for Matmul + AllReduce.

Bug fixes#

  • Fixed Vec16/Vec32 scale tensor layout in Matmul + ReduceScatter and Matmul + AllReduce.

  • Fixed epilogue accuracy issue in Matmul + ReduceScatter when using FP4/FP8 output and communication type.

cuBLASMp v0.8.0#

Released: February 9, 2026

New features#

Breaking changes#

  • Removed NVSHMEM support. Users should migrate to NCCL symmetric memory using the new buffer registration APIs (cublasMpBufferRegister, cublasMpBufferDeregister) or the convenience functions (cublasMpMalloc, cublasMpFree).

  • Renamed cublasMpStreamSet API to cublasMpSetStream.

  • Renamed cublasMpStreamGet API to cublasMpGetStream.

  • Renamed cublasMpMatmulDescriptorAttributeSet API to cublasMpMatmulDescriptorSetAttribute.

  • Renamed cublasMpMatmulDescriptorAttributeGet API to cublasMpMatmulDescriptorGetAttribute.

Bug fixes#

  • Fixed AMAX_D output to be pre-quantization.

  • Fixed an issue that the auxiliary buffer had to be initialized with zeros by the users.

cuBLASMp v0.7.0#

Released: November 24, 2025

New features#

  • Added FP32 emulation support.

  • Added FP4 output support.

  • Added CUBLASMP_MATMUL_ALGO_TYPE_NO_OVERLAP algorithm type for cublasMpMatmul API. These algorithms may perform better on non-NVLINK multi-node systems.

  • Added CUBLASMP_DISABLE_NVSHMEM environment variable to disable NVSHMEM usage at runtime. When set to 1, cuBLASMp will not initialize or use NVSHMEM, which can be useful for debugging or on platforms where NVSHMEM is not supported.

Bug fixes#

  • Fixed epilogue related issues in AllGather + Matmul and Matmul + ReduceScatter.

  • Fixed a bug that prevented NVSHMEM from being finalized.

cuBLASMp v0.6.0#

Released: September 9, 2025

Bug fixes#

  • Fixed redistribution functions (cublasMpGemr2D, cublasMpTrmr2D) to work with unified memory.

  • Fixed Matmul when C matrix is nullptr and D matrix type is BF16 or FP32.

  • Fixed a bug in multicast-based algorithms when used with communicator different from the one used for initialization.

cuBLASMp v0.5.1#

Released: August 11, 2025

  • Added support for CUDA 13 on devices with Compute Capability 8.0 (Ampere) and above.

cuBLASMp v0.5.0#

Released: June 16, 2025

Breaking changes#

  • cuBLASMp has transitioned from using the Communication Abstraction Library (libcal) to using NCCL directly. This is a breaking change and requires changes to cuBLASMp initialization in the user application.

New features#

cuBLASMp v0.4.0#

Released: March 10, 2025

cuBLASMp v0.3.1#

Released: December 10, 2024

  • Add option to set the amount of SMs to be used for communication (currently relevant only for Atomic Matmul + ReduceScatter).

  • Decrease workspace size requirement in TP overlap Matmuls.

  • Remove extra synchronization in TP overlap Matmuls.

  • Allow C matrix to be null when beta is 0.

  • Fix GEMM implementation for complex types with transA / transB being CUBLAS_OP_T.

cuBLASMp v0.3.0#

Released: November 4, 2024

Breaking changes#

cuBLASMp v0.2.1#

Released: May 29, 2024

  • Added mixed and lower precision support.

  • Bug fixes.

cuBLASMp v0.2.0#

Released: April 4, 2024

cuBLASMp v0.1.2#

Released: February 22, 2024

cuBLASMp v0.1.1#

Released: January 11, 2024

cuBLASMp v0.1.0#

Released: December 11, 2023

  • Early access release.

  • This release focuses on functionality.