nvmath-python Release Notes#

nvmath-python v1.0.0#

Release Summary#

The first nvmath-python GA release, with key new features including the single-GPU and distributed dense generic direct solver, performance improvements to the sparse generic matrix multiplication API, support for numba-cuda-mlir for device APIs, support for NVFP4 in distributed matrix multiplication, support for explicit batching in generic matrix multiplication, and more. As always, we look forward to your feedback and suggestions as we continue to improve the library to meet your needs.

New Features#

  • Added nvmath.linalg.generic.Matmul.reset_operands_unchecked(), a performance-optimized variant of reset_operands() that skips operand validation and logging.

  • Added experimental free-threaded support on bindings

  • Improved thread-safety for nvmath.linalg. A Matmul instance should now only be used by its creating thread.

  • Calling release_operand(s) more than once is now safe across all stateful APIs; extra calls are a no-op and log an info-level message.

  • Added nvmath.fft.FFT.create_key_from_metadata(), the metadata-based counterpart to nvmath.fft.FFT.create_key(). It builds an FFT key from operand metadata (shape, dtype, and optional strides) together with a memory_space, instead of requiring a fully allocated operand. This is useful, for example, to estimate the workspace size before an operand exists. The execution space is optional and defaults to memory_space.

  • Added support for explicit batching to generic matmul.

  • The sparse advanced direct solver has been updated to use cuDSS v0.8.0, and hence benefits from the performance improvements and bug fixes in this version.

  • The device APIs now support the MLIR-based numba-cuda-mlir compiler in addition to numba-cuda. The FFT, Matmul, and Solver objects are compiler-agnostic; the compiler is selected by the cuda.jit decorator used for the kernel. The nvmath.device.random APIs and the advanced Matmul APIs (opaque tensors, accumulators, and pipelines) remain available only with numba-cuda.

  • Added the float32x4, float64x4, and uint32x4 vector types (and the corresponding *_type numba types) to nvmath.device.

  • The cuBLASDx DevicePipeline now accepts any input arrays supporting DLPack or the CUDA Array Interface.

  • Support DRELU and DRELU_BGRAD epilogues in distributed matrix multiplication.

  • Distributed matrix multiplication now registers operand B with NCCL symmetric memory whenever possible, which improves Allgather+GEMM performance.

  • Support for NVFP4 in the distributed matrix multiplication API.

  • Improved performance of the generic sparse matrix multiplication (SpMM) APIs (nvmath.sparse.Matmul and nvmath.sparse.matmul()) for the code generation path with universal sparse tensor operands due to new kernel emitter capabilities.

Bugs Fixed#

  • Fixed a cuSPARSE handle leak which occurred when nvmath.sparse.generic.Matmul.plan() was called multiple times on the same instance.

  • Fixed nvmath.bindings.nvpl compatibility with MKL 2026.0.0.

  • Fixed an incorrect AttributeError when reusing a nvmath.linalg.advanced.Matmul with block scaling and tensor quantization scales; calling release_operands() → reset_operands() now completes without error.

  • Fixed nvmath.linalg.advanced.helpers.matmul.to_block_scale() to validate the full scale-tensor shape against the operand’s logical block-scaling shape. Previously, only the trailing two dimensions were compared, so scale tensors with permuted batch dimensions could be silently accepted.

  • Fixed logging state leaking from one example test to another

  • Fixed a crash in get_current_device_cc() caused by unpacking the 3-field ComputeCapability.

  • FFT now raises a descriptive error instead of a cryptic KeyError when an incomplete real_fft_options dict is provided.

  • Exported nvmath.device.compile_blas_execute(), which was missing from the public API.

  • Fixed nvmath.device.compile_blas_execute() to convert its code_type argument with parse_code_type, matching the behavior of the other device APIs.

Breaking Changes#

  • Changed nvmath.bindings.cufftMp.XtSubFormat.FORMAT_FORMAT_UNDEFINED to nvmath.bindings.cufftMp.XtSubFormat.FORMAT_UNDEFINED.

  • reset_operand(s) now requires at least one operand to be provided across all stateful APIs; otherwise it raises a ValueError.

  • The common algorithm type (nvmath.sparse.advanced.DirectSolverAlgType) for the sparse advanced direct solver has been removed in favor of new algorithm types specific to each phase such as nvmath.sparse.advanced.DirectSolverFactorizationAlg. This is a result of the changes in cuDSS v0.8.0.

  • Distributed FFT API: renamed reshape option to redistribute.

  • Distributed matrix multiplication API is now inplace by default when C is provided (result is stored in C).

  • Removed the deprecated cuBLASDx API surface from nvmath.device.cublasdx:

    • the matmul() factory function (construct the Matmul class directly instead)

    • the Matmul methods definition() and create()

    • the Matmul properties value_type, input_type, output_type, shared_memory_size, files and codes

    • direct Matmul(...) invocation, replaced by Matmul.execute(...)

  • Removed the deprecated cuFFTDx API surface from nvmath.device.cufftdx:

    • the fft() factory function (construct the FFT class directly instead)

    • the FFT methods definition() and create()

    • the FFT properties requires_workspace and workspace_size, together with the associated workspace support

    • the files property

    • direct FFT(...) invocation, replaced by FFT.execute(...)

  • Removed the deprecated make property from the numba vector-type wrappers in nvmath.device.types. Use numba types directly.

  • Distributed reshape has moved to nvmath.distributed.distribution.Redistribute and the API has been generalized to accept any Distribution type as input and output distribution. Note that the specific distributions that are supported depends on the capabilities of the underlying library (currently cuFFTMp).

  • Support for complex32 operands in sparse matrix multiplication (nvmath.sparse.generic.matmul()) has been removed. Passing complex32 operands now raises a TypeError.

  • The semiring and epilog parameters to the generic sparse matrix multiplication (SpMM) plan (nvmath.sparse.Matmul.plan()) have been removed. In addition, prolog cannot be specified for the c operand. We plan to introduce comprehensive support for these in the near future.

Documentation Changes#

  • Distributed APIs are now described as distributed host APIs.

  • The device API documentation was refreshed.

  • The device API examples were restructured to mirror the CUDALibrarySamples layout, and numba-cuda-mlir variants of the examples were added.

  • New cuSOLVERDx examples were added, matching the cuSOLVERDx 0.3 CUDA samples.

  • Adopted reno as the release notes manager.

Dependency Changes#

  • The numba extra now installs numba-cuda-mlir in addition to numba-cuda.

  • The nvmath-python package now requires cuda-core >=0.5, <2 (updated from cuda-core >=0.4.2, <1).

Known Issues#

  • On Windows, if both torch and nvmath-python[cpu] are installed, there may be two libiomp5md.dll files present in the environment:

    • one shipped with torch, and

    • one coming from transitive dependency: nvmath-python[cpu] -> mkl -> intel_openmp.

    By default, OMP will terminate the program with an error when attempting to load both copies. As a workaround, users can:

    • set KMP_DUPLICATE_LIB_OK=TRUE environment variable to silence the error, or

    • remove one of the libiomp5md.dll copies, for instance, by uninstalling intel_openmp package (e.g. pip uninstall intel-openmp).

  • Sparse matrix multiplication (nvmath.sparse.generic.matmul()) using the CSC format with float16 or bfloat16 operands is not supported on newer CUDA Toolkit versions and raises cuSPARSEError NOT_SUPPORTED.

  • Using the nvmath.distributed.distributions.Box distribution with nvmath.distributed.fft.FFT and nvmath.distributed.distributions.Redistribute APIs may result in a spurious validation error. The workaround is to use an equivalent nvmath.distributed.distributions.Slab distribution.

Security Issues#

nvmath-python v0.9.0#

Beta9 release.

Bugs Fixed#

  • Fixed in-place C2R FFT repeated execution silently producing wrong results due to the input buffer being overwritten.

  • Fixed some incorrectly named enums in cuFFT bindings.

  • pivot_eps_algorithm, pivot_eps, hybrid_device_memory_limit, and hybrid_execute_mode properties were returning the wrong values.

  • cuFFT status codes from CUDA 13 were missing.

  • FFT.create_key() and FFT.get_key() had mismatched outputs.

  • apply_mxfp8_scale() could overflow.

  • BinaryContraction outputs were not fully-owned by user and were overwritten by subsequent calls to BinaryContraction.contract().

  • cuDSS DirectSolver rejected valid F-order matrices with shape (..., m, 1) because stride validation didn’t account for the dummy last dimension (#53).

  • Fixed LTOIR ABI correctness for device APIs where argument names and return types did not match the libmathdx functions.

  • Fixed a missing synchronization for host-to-device torch tensor copy.

  • Fixed missing fields in the cuBLASDx Numba matmul cache key, which could lead to stale cached results.

  • Improved error handling for unsupported cuDSS FactorizationInfo and PlanInfo attributes, which now raise RuntimeError instead of silently returning wrong values.

  • Fixed reset_operands_unchecked semantics for FFT to match the checked version, behaving correctly when called after releasing operands.

Breaking Changes#

Known Issues#

nvmath-python v0.8.0#

Beta8 release.

  • New pipeline and supporting features for device matrix multiplication APIs that enable applications such as floating-point emulation.

  • Support for inplace operation in the advanced Matmul host APIs.

  • Support for implicit batching in the generic Matmul host APIs.

  • Windows support for the tensor contraction APIs.

  • A new experimental nvmath.fft.FFT.reset_operand_unchecked() API to reduce redundant checking and minimize overhead.

  • Added bindings for new APIs introduced in CTK version 13.1.

  • cuBLASMp bindings updated to 0.7

Bugs Fixed#

  • The tensor contraction API always blocked in Beta7, even if asynchronous execution (the default) was requested. This has been fixed.

  • Fixed the outdated references in the documentation that state the CuPy will be installed as part of nvmath-python extras. This was no longer true from Beta7 onwards.

  • The internal references to the tensor contraction and direct solver operands held in those objects relied on garbage collection to be released. This has been fixed, so that the references are now released when the context manager exits or when the object is explicitly freed.

  • A performance issue has been fixed for certain tensor contractions that involve small contraction extents along with large batch extents.

Breaking Changes#

Known Issues#

  • The use of Python logging set to the debug level (logging.DEBUG) may result in a TypeError when compiling Numba kernels.

nvmath-python v0.7.0#

Beta7 release.

  • This release supports CUDA 12 and CUDA 13. Support for CUDA 11 has been dropped.

  • New binary and ternary tensor contraction host APIs on GPU.

  • New generic host Matmul APIs that support dense and structured matrices (such as triangular and diagonal) on GPU and CPU.

  • New distributed Matmul APIs to run on multi-node/multi-GPU systems.

  • Support for 64-bit integer indexing for the sparse direct solver.

  • The FFT and Matmul device APIs are now implicitly linked in kernels and the link= argument to numba.cuda.jit() is no longer needed.

  • The device APIs now use custom types that lower to NumPy (host) or Numba (device) types. As a result of this, nvmath.device.FFT.value_type and nvmath.device.Matmul.value_type return NumPy types.

Bugs Fixed#

  • nvmath-python/#47 Fixed a “key error” bug that prevented use of complex-to-real double precision distributed FFT.

  • cuda-python/#852 An internal symbol table used when loading symbols from libraries was made thread-safe.

Breaking Changes#

Deprecations#

nvmath-python v0.6.0#

Beta6 release.

  • This will be the last release to support CUDA 11.

  • Added support for distributed R2C/C2R FFTs, along with support for non-uniform partition sizes across PEs.

  • The distribution option for distributed FFTs is now a required keyword-only argument.

  • To enable making CuPy an optional dependency, an internal NDBuffer datastructure was introduced that facilitates copying tensors across memory spaces and layouts. Users may notice a one-time latency for each unique layout since the copy kernel is JIT compiled and cached.

  • Replaced internal logic with cuda-pathfinder for locating libraries and components.

Bugs Fixed#

  • The nvmath.linalg.advanced.Matmul.autotune() method in the advanced Matmul APIs may not have selected the best kernel, since the L2-cache wasn’t cleared.

  • The return status of an internal call to a CUDA API wasn’t checked, resulting in a misleading error regarding memory limit.

  • Fixed a use-after-free issue with the batched direct sparse solver.

  • Fixed a deadlock that may occur in certain circumstances during distributed FFT.

  • Added appropriate constraints for cuda-bindings based on the CTK version.

  • Fixed missing logging messages when a Python logger was not created with force=True.

Known Issues#

  • The minimum supported versions for CuPy and PyTorch are out-of-date and will be increased in the next release.

  • An internal symbol table used when loading symbols from libraries needs to be made thread-safe. This will be done in the next release.

nvmath-python v0.5.0#

Beta5 release.

  • New single-GPU and hybrid CPU-GPU sparse direct solver APIs supporting SciPy, CuPy, and PyTorch.

Known Issues#

  • Python overhead for matmul host-APIs has increased since v0.3.0 by 21 microseconds on average. We are investigating.

  • CUDA 12.8.0, 12.8.1 and 12.9.0 have been known to miscompile cuBLASDx in some rare slow-path cases (see cuBLASDx for more details).

nvmath-python v0.4.0#

Beta4 release.

  • New distributed FFT APIs to run on multi-node/multi-GPU systems.

  • New device matrix multiplication tensor API to enable advanced techniques such as cooperative copy and floating-point emulation using integer tensor cores.

  • Transition from CuPy to cuda-python (cuda.core) for core CUDA constructs.

Bugs Fixed#

  • FFT prolog or epilog fails to compile on SM >= 100.

Known Issues#

  • Python overhead for matmul host-APIs has increased since v0.3.0 by 21 microseconds on average. We are investigating.

nvmath-python v0.3.0#

Beta3 release.

  • FP8 and MXFP8 support for the advanced matrix multiplication API.

  • Notebook to illustrate use of FP8 and MXFP8 in the advanced matrix multiplication API.

  • Added bindings for new APIs introduced in CTK version 12.8.

Bugs Fixed#

  • The advanced matrix multiplication API may return an incorrect result when a bias vector is used along with 1-D A and C operands.

API Changes#

  • The last_axis_size option in nvmath.fft.FFTOptions is removed in favor of last_axis_parity to better reflect its semantics.

nvmath-python v0.2.1#

Beta2 update 1 with improved diagnostics, testing enhancements, and bug fixes.

  • New tests for batched epilogs and autotuning with epilogs for the advanced matrix multiplication APIs.

  • Added more hypothesis-based tests for host APIs.

  • Improved algorithm for detecting overlapping memory operands for certain sliced tensors, thereby supporting such layouts for FFTs.

  • Added bindings for new APIs introduced in CTK versions 12.5 and 12.6.

  • Further coding style fixes toward meeting PEP8 recommendations.

  • Clarified batched semantics for matrix multiplication epilogs in the documentation.

  • Code snippets in API docstrings are now tested.

Bugs Fixed#

  • C2R FFT may fail with “illegal memory access” on sliced tensors.

  • Improved diagnostics to detect incompatible combinations of scale and compute types for matrix multiplication, that previously may have resulted in incorrect results.

  • Matrix multiplication provided incorrect results when operand A is a vector (number of dimensions=1).

API Changes#

  • The last_axis_size option in nvmath.fft.FFTOptions is now deprecated in favor of last_axis_parity to better reflect its semantics.

Note

Deprecated APIs will be removed in the next release.

nvmath-python v0.2.0#

Beta2 release.

  • CPU execution space support for FFT libraries that conform to FFTW3 API (for example MKL, NVPL).

  • Support for prolog and epilog callback for FFT, written in Python.

  • New device APIs for random number generation.

  • Notebooks to illustrate use of advanced matrix multiplication APIs.

  • Introduced hypothesis-based tests for host APIs.

  • Reduced Python overhead in execute methods.

Bugs Fixed#

  • Matrix multiplication may fail with “illegal memory access” for K=1 with DRELU and DGELU epilogs.

Packaging#

  • Added support for NumPy 2.

  • Removed Python 3.9 support.

  • Patching changes and pynvjitlink version.

Known issues#

  • When compute_type argument of nvmath.linalg.advanced.Matmul is set to COMPUTE_16F, an incompatible default for scale_type is chosen, resulting in incorrect results for CTKs older than 12.6 and an error for CTK 12.6 and newer. As a workaround we recommend setting both compute_type and scale_type in a compatible manner according to supported data types table.

nvmath-python v0.1.0#

Initial beta release, with single-GPU support only.

  • FFT APIs based on cuFFT.

  • Specialized matrix multiplication APIs based on cuBLASLt.

  • Device APIs for FFT and matrix multiplication based on the MathDx libraries.

The required and optional dependencies are summarized in the cheatsheet.

Limitations:

  • Many matrix multiplication epilogs require CTK 11.5+, and a few require CTK 11.8+. Refer to cuBLAS Release Notes for more details.

Disclaimer#

nvmath-python is in a Beta state. Beta products may not be fully functional, may contain errors or design flaws, and may be changed at any time without notice. We appreciate your feedback to improve and iterate on our Beta products.