Release Notes

cuSOLVERMp v0.9.0

Released: July 8, 2026

New features

• Added cusolverMpPolar() for distributed polar decomposition of tall or square (M >= N) matrices using the QDWH algorithm. In the default unperturbed case, the decomposition satisfies A = Up * H. The input is overwritten with the polar factor Up; computing the Hermitian positive semidefinite factor H is optional.

  • Added cusolverMpPolarDescriptorCreate(), cusolverMpPolarDescriptorDestroy(), cusolverMpPolarDescriptorSetAttribute(), and cusolverMpPolarDescriptorGetAttribute() for requesting an optional diagonal perturbation and reading per-call diagnostics (e.g., input Frobenius norm, reciprocal condition estimate).

• Added cusolverMpGesvd() for distributed singular value decomposition A = U * Sigma * V^H, built on the polar decomposition (cusolverMpPolar()). All four (jobu, jobvt) combinations are supported, including values-only, for both M >= N and M < N.

  • Added cusolverMpGesvdDescriptorCreate(), cusolverMpGesvdDescriptorDestroy(), cusolverMpGesvdDescriptorSetAttribute(), and cusolverMpGesvdDescriptorGetAttribute() for selecting the THIN (default) or FULL singular-vector output shape and reading per-call diagnostics (e.g., input Frobenius norm, reciprocal condition estimate, optional reconstruction residual).

Breaking changes

• cuSOLVERMp now depends on cuBLASMp v0.9.1 or newer at runtime.

• Workspace requirements may change between cuSOLVERMp versions; applications should not assume that sizes returned by one version apply to another and should always allocate workspaces based on the sizes reported by the *_bufferSize APIs of the version in use. In cuSOLVERMp 0.9.0, the *_bufferSize APIs of cusolverMpGetrs(), cusolverMpPotrs(), cusolverMpGels(), and cusolverMpSygvd() may report a nonzero host workspace size where previous versions reported zero.

Performance

• Improved performance of cusolverMpSytrd(), cusolverMpStedc(), cusolverMpOrmqr(), cusolverMpOrgqr(), cusolverMpOrmtr(), cusolverMpGels(), cusolverMpSyevd(), and cusolverMpSygvd().

Resolved Issues

• Device info output handling is now consistent across the dense linear algebra computational routines: every routine that exposes a device info output accepts NULL for it and resets a supplied info to 0 on successful execution. Previously, the behavior was inconsistent. Passing NULL skips device-side info reporting, including routine-specific positive diagnostics such as singularity and convergence reports.

• Fixed cusolverMpStedc() failures (incorrect results or misaligned-address errors) with non-power-of-two block sizes and certain single-precision 2D configurations, and removed the documented power-of-two block-size limitation. cusolverMpSyevd() and cusolverMpSygvd() use STEDC internally and benefit from this fix. This bug existed in versions 0.4.2 through 0.8.0.

• Fixed workspace queries that could under-report the required device workspace with cuSOLVER from CUDA Toolkit 13.2 Update 1 and newer, which could cause failures, illegal memory accesses, or hangs in cusolverMpGeqrf(), cusolverMpGels(), cusolverMpOrgqr(), cusolverMpOrmqr(), cusolverMpGetrf(), cusolverMpPotrf(), and cusolverMpSygvd(). cuSOLVERMp versions older than 0.9.0 are affected when used with these CUDA Toolkit versions.

• Tightened input validation across the dense linear algebra APIs: valid but unsupported configurations are now more consistently rejected with CUSOLVER_STATUS_NOT_SUPPORTED and malformed arguments with CUSOLVER_STATUS_INVALID_VALUE.

• Clarified that cusolverMpSyevd() accepts both upper and lower triangular input (selected by uplo). The behavior is unchanged from prior releases; earlier documentation grouped SYEVD with the lower-only routines. The standalone cusolverMpSytrd() and cusolverMpOrmtr() routines remain lower-only.

• The packed version value returned by cusolverMpGetVersion() is now computed as CUSOLVERMP_VER_MAJOR * 10000 + CUSOLVERMP_VER_MINOR * 100 + CUSOLVERMP_VER_PATCH; the previous MAJOR * 1000 encoding could not represent minor versions of 10 and above. Packed values for all previously released versions are unchanged.

cuSOLVERMp v0.8.0

Released: April 13, 2026

New features

• Added cusolverMpNewtonSchulz() for distributed Newton-Schulz orthogonalization of tall or square (M >= N) matrices on supported Px1 process grids. This iterative method approximates the orthogonal polar factor of a matrix (i.e., X^T * X → I), useful for optimizer orthogonalization in deep learning. Supports CUDA_R_16BF (bfloat16) and CUDA_R_32F (float32) value types with CUDA_R_32F compute type. The iteration coefficients are user-supplied (3 floats per iteration); example coefficients optimized for quintic convergence in 5 iterations are provided in the Newton-Schulz sample. 2D block-cyclic grids and wide rectangular matrices are not yet supported.

  • Added cusolverMpBufferRegister() and cusolverMpBufferDeregister() for registering NCCL symmetric memory buffers on the grid communicator. Pre-registering workspace buffers allocated with ncclMemAlloc can improve performance of NCCL collective operations. Currently expected to yield performance benefits when used with cusolverMpNewtonSchulz().

  • Added cusolverMpNewtonSchulzDescriptorCreate(), cusolverMpNewtonSchulzDescriptorDestroy(), cusolverMpNewtonSchulzDescriptorSetAttribute(), and cusolverMpNewtonSchulzDescriptorGetAttribute() for configuring Newton-Schulz behavior (e.g., input normalization, reduction precision).

• Added cusolverMpOrgqr() to explicitly generate the matrix Q with orthonormal columns from a QR factorization computed by cusolverMpGeqrf().

• Added cusolverMpLaset() to initialize distributed matrices with scalar values.

• Added cusolverMpSetStream() API to allow setting a custom CUDA stream on the cuSOLVERMp handle after creation.

Performance

• Improved performance of cusolverMpSyevd() and cusolverMpSygvd().

Resolved Issues

• Removed a restriction where the default (NULL or 0) CUDA stream could not be passed to cusolverMpCreate() (bug 4337214). This limitation existed in all prior versions of cuSOLVERMp.

• Fixed a bug in cusolverMpPotrf() / cusolverMpPotrs() that could produce incorrect results or hangs on non-square process grids (e.g., 3x4, 4x3) when the matrix spanned more tiles than one cycle of the 2D block-cyclic distribution. This bug existed in versions 0.3.0 through 0.7.2.

• Fixed cusolverMpOrmqr() communicator-root selection for distributed submatrix broadcasts. Affected versions: 0.3.0 through 0.7.2. In particular, versions 0.4.3 through 0.7.2 could produce incorrect results on row-major grids when K < N.

cuSOLVERMp v0.7.2

Released: November 10, 2025

Resolved Issues

• Fixed a bug that could cause a race condition (deadlock or incorrect results) in cusolverMpGetrf() / cusolverMpGetrs() in cuSOLVERMp versions 0.7.0 and 0.7.1.

• Fixed a bug that could result in a deadlock when grids of different dimensions were used with the same cuSOLVERMp handle.

cuSOLVERMp v0.7.1

Released: September 18, 2025

Resolved Issues

• Fixed a bug that could cause a race condition in cusolverMpGeqrf(). The bug was observed in tall-skinny QR problems on V100 GPUs.

• Removed a superfluous warning printed when message size exceeded INT32_MAX in cusolverMpStedc().

• Fixed a bug in FP32 emulation mode where the emulation strategy set by cusolverMpSetEmulationStrategy() was not propagated to cuSOLVER.

• Added support for Compute Capability 7.5 (Turing) in CUDA 13 builds.

cuSOLVERMp v0.7.0

Released: August 12, 2025

Breaking changes

• cuSOLVERMp has transitioned from using the Communication Abstraction Library (libcal) to using NCCL directly. This is a breaking change and requires changes to cuSOLVERMp initialization in the user application.

  • See Migrating from CAL to NCCL for steps to transition the application from libcal to NCCL.

  • The steps to initialize cuSOLVERMp with NCCL are described in NCCL Initialization.

  • libcal documentation page is still available at CAL Initialization (Legacy) but it is only applicable to cuSOLVERMp versions older than 0.7.0.

New features

• Added support for CUDA 13 on devices with Compute Capability 8.0 (Ampere) and above.

• cuSOLVERMp leverages techniques for floating point emulation as described in cuBLAS Floating Point Emulation for improved performance (CUDA 13+ and Compute Capability 10+).

  • Introduced new APIs: cusolverMpSetMathMode(), cusolverMpGetMathMode(), cusolverMpSetEmulationStrategy(), and cusolverMpGetEmulationStrategy().

  • FP32 emulation can be enabled by setting the math mode to CUSOLVER_FP32_EMULATED_BF16X9_MATH (see cusolverMpSetMathMode()).

  • The emulation strategy can be further tuned using the cusolverMpSetEmulationStrategy() API.

  • The math mode and emulation strategy are propagated to the internal cuBLAS and cuSOLVER handles. The workspace sizes returned by *_bufferSize APIs may depend on the math mode.

  • The defaults are not changed from the previous version, i.e., emulation is disabled and math mode is set to CUSOLVER_DEFAULT_MATH.

Resolved Issues

• Fixed a bug that could cause cusolverMpGeqrf() to fail on non-square process grids.

• Fixed a bug that could cause cusolverMpGetrf()/cusolverMpGetrs() to fail on non-square process grids.

• Fixed a bug that could cause cuSOLVERMp to crash when logging is enabled.

cuSOLVERMp v0.6.0

Released: February 13, 2025

• Added support for NVIDIA Blackwell GPU architecture.

• Dropped support for CUDA 11.x.

cuSOLVERMp v0.5.1

Released: August 22, 2024

• Fixed a bug in cusolverMpSyevd() where the eigenvalues were not broadcasted to all the processes if the problem fit on a single process.

cuSOLVERMp v0.5.0

Released: May 2, 2024

• Improved the performance of cusolverMpStedc().

• Introduced a new option to force NCCL usage by setting the CUSOLVERMP_FORCE_NCCL=1 environment flag. This is only applicable in parts of the eigensolver for now.

cuSOLVERMp v0.4.3

Released: February 5, 2024

• Supported CUDA 12.1.1.

• Fixed a bug that processors are hanging when a problem is tiny and fits in a single processor.

Known Issues

• CUDA 12.1.1 is compatible with NCCL up to v2.16.x; higher NCCL version may hang intermittently for certain processor grids.

cuSOLVERMp v0.4.2

Released: HPC SDK 23.11

• Fixed a bug in cusolverMpSyevd() that the code returns an internal error for a matrix filled with zero entries; the correct behavior is to return zero eigenvalues and unit eigenvectors.

• Supported CUDA 12.1.1

• Note that the code is compatible with NCCL up to v2.16.x

cuSOLVERMp v0.4.1

Released: HPC SDK 23.7

• Added support for row major grid in SYEVD.

cuSOLVERMp v0.4.0

Released: HPC SDK 23.5

• Added routines for symmetric (Hermitian) generalized eigen solver

  • cusolverMpSygst() reduces the symmetric (Hermitian) generalized eigen problem to standard form.

  • cusolverMpSygvd() computes all eigenvalues and eigenvectors of symmetric (Hermitian) generalized eigen problem.

cuSOLVERMp v0.3.1

Released: HPC SDK 23.3

• Minor bugfixes are included

  • cusolverMpPotrf() fixes to result cleans zeros of the imaginary part of diagonals.

  • cusolverMpStedc() fixes internal memory leak.

cuSOLVERMp v0.3.0

Released: HPC SDK 23.1

• Removed dependency on MPI, now UCC library is the main communication backend

• Provide the following computational APIs:

  • cusolverMpGeqrf_bufferSize(), cusolverMpGeqrf(), cusolverMpOrmqr_bufferSize(), cusolverMpOrmqr(), cusolverMpGels_bufferSize(), cusolverMpGels(), cusolverMpSytrd_bufferSize(), cusolverMpSytrd(), cusolverMpStedc_bufferSize(), cusolverMpStedc(), cusolverMpOrmtr_bufferSize(), cusolverMpOrmtr(), cusolverMpSyevd_bufferSize(), cusolverMpSyevd().

• Note that cusolverMpGels() currently supports least square solutions with no-transpose option only.

• Note that cusolverMpSytrd(), cusolverMpOrmtr() and cusolverMpSyevd() currently support a lower triangular input matrix only.

cuSOLVERMp v0.2.0

Released: HPC SDK 22.05

• Added support for pp64 + SpectrumMPI, targeting ORNL’s Summit Supercomputer.

• Added Cholesky factorization and solve APIs:

  • cusolverMpPotrf_bufferSize(), cusolverMpPotrf(), cusolverMpPotrs_bufferSize(), cusolverMpPotrs().

• Note that cusolverMpGetrs() does not offer support for multiple right-hand sides at this point.

cuSOLVERMp v0.1.0

Released: HPC SDK 21.11

• Initial release.

• Support Linux x86_64 and Compute Capability 8.0.

• Provide the following computational APIs:

  • cusolverMpGetrf_bufferSize(), cusolverMpGetrf(), cusolverMpGetrs_bufferSize(), cusolverMpGetrs().

• Note that cusolverMpGetrs() does not offer support for multiple right-hand sides at this point.