Linear Algebra cutile-python API#

nvmath.tile.linalg combines cuBLASDx-style triangular operations with cuSolverDx factorizations, solvers, eigensolvers, and singular-value decompositions.

The equivalent C++ interfaces are split across cuBLASDx in Tile and cuSolverDx in Tile.

Routines that can fail at runtime return an integer info value with the result. The caller must check info. Refer to Status codes. All dense routines accept an optional leading batch extent.

Quick start#

import cuda.tile as ct
import cupy as cp
import nvmath.tile.linalg as mlinalg


@ct.kernel(num_worker_warps=4)
def linalg_kernel(
    triangular: ct.Array,
    rhs: ct.Array,
    positive_definite: ct.Array,
    solution: ct.Array,
    factor: ct.Array,
    status: ct.Array,
):
    a = ct.load(triangular, (0, 0), shape=(16, 16))
    b = ct.load(rhs, (0, 0), shape=(16, 16))
    x = mlinalg.triangular_solve(
        a, b, mlinalg.FillMode.LOWER, mlinalg.DiagMode.NON_UNIT, mlinalg.SideMode.LEFT
    )
    ct.store(solution, (0, 0), x)

    matrix = ct.load(positive_definite, (0, 0), shape=(16, 16))
    cholesky, info = mlinalg.cholesky_factor(matrix, mlinalg.FillMode.LOWER)
    ct.store(factor, (0, 0), cholesky)
    ct.store(status, (0,), info)


def main():
    triangular = cp.tril(cp.ones((16, 16), dtype=cp.float32)) + cp.eye(16, dtype=cp.float32)
    rhs = cp.eye(16, dtype=cp.float32)
    positive_definite = triangular @ triangular.T
    solution = cp.zeros_like(rhs)
    factor = cp.zeros_like(positive_definite)
    status = cp.full((1,), -1, dtype=cp.int32)
    ct.launch(
        cp.cuda.get_current_stream(),
        (1, 1, 1),
        linalg_kernel,
        (triangular, rhs, positive_definite, solution, factor, status),
    )
    cp.cuda.runtime.deviceSynchronize()
    assert int(status[0]) == 0
    assert cp.allclose(triangular @ solution, rhs, atol=1e-4)
    lower = cp.tril(factor)
    assert cp.allclose(lower @ lower.T, positive_definite, atol=1e-4)


if __name__ == "__main__":
    main()

API reference#

Linear algebra inside a CUDA Tile kernel, backed by MathDx tilelibs.

The API currently supports only real float32/float64 tiles with compile-time, power-of-two shapes. Every dense operation is rank-polymorphic: tiles may have one leading batch dimension. With this dimension, one tile block solves a batch of independent problems at once. All operands must have the same batch extent, or all must be unbatched.

The tilelib selects scratch memory, layouts, leading dimensions, and thread counts. Operations that can fail at runtime return an int32 info tile as their final result. It contains one status per batch, or a scalar for an unbatched problem. Zero indicates success. Other operations cannot fail numerically and do not return info.

Options are plain strings. lu_factor defaults to partial pivoting. Every trans argument defaults to no transpose. Other options are mandatory, except lu_solve’s pivot, which defaults to partial pivoting.

class nvmath.tile.linalg.FillMode#

Triangle containing the meaningful symmetric or triangular data.

LOWER#

Select the lower triangle.

UPPER#

Select the upper triangle.

class nvmath.tile.linalg.PivotMode#

Partial-pivot or no-pivot policy for LU operations.

PARTIAL = 'partial_pivot'#

Use partial pivoting.

NONE = 'no_pivot'#

Do not use pivoting.

class nvmath.tile.linalg.SideMode#

Side on which an implicit Q or triangular matrix acts.

LEFT#

Apply the matrix from the left.

RIGHT#

Apply the matrix from the right.

class nvmath.tile.linalg.DiagMode#

Whether a triangular matrix has an explicit or implicit unit diagonal.

UNIT#

Treat the diagonal as implicitly unit.

NON_UNIT#

Read the matrix’s diagonal entries.

class nvmath.tile.linalg.TransposeMode#

Whether \(\operatorname{op}(A)\) denotes \(A\) or \(A^T\).

NO_TRANS = 'no_trans'#

Use the matrix without transposing it.

TRANS = 'trans'#

Use the transpose of the matrix.

class nvmath.tile.linalg.JobMode#

Eigenvector or singular-vector job, subject to each routine’s supported subset.

NO_VECTORS#

Compute no eigenvectors or singular vectors.

ALL_VECTORS#

Compute all eigenvectors or singular vectors.

SOME_VECTORS#

Compute a reduced set of singular vectors.

MULTIPLY_VECTORS#

Multiply supplied vectors by the computed singular vectors.

OVERWRITE_VECTORS#

Overwrite the input with the computed eigenvectors.

nvmath.tile.linalg.cholesky_factor(A, fill_mode)#

Factor symmetric positive-definite A using the selected triangle.

Parameters:
  • A (cuda.tile.Tile) – Matrix with shape (M, M) or batched matrices with shape (B, M, M).

  • fill_mode (FillMode) – Triangle of A to read and of the factor to return.

Returns:

(factor, info), where factor has the same shape as A and its selected triangle contains the Cholesky factor. info has shape () when unbatched or (B,) when batched.

Return type:

tuple[cuda.tile.Tile, cuda.tile.Tile]

nvmath.tile.linalg.lu_factor(A, pivot='partial_pivot')#

Compute packed LU factors, using partial pivoting by default.

Parameters:
  • A (cuda.tile.Tile) – Matrix with shape (M, N) or batched matrices with shape (B, M, N).

  • pivot (PivotMode, optional) – Pivoting policy, defaulting to PivotMode.PARTIAL.

Returns:

With partial pivoting, (LU, pivots, info). Without pivoting, (LU, info). LU has the shape of A, pivots has shape (R,) or (B, R) for R = min(M, N), and info has shape () or (B,).

Return type:

tuple[cuda.tile.Tile, …]

nvmath.tile.linalg.modified_lu_factor(A)#

Compute the modified no-pivot factorization \(A - \operatorname{diag}(S) = L U\).

Parameters:

A (cuda.tile.Tile) – Matrix with shape (M, N) or batched matrices with shape (B, M, N), where \(M \ge N\).

Returns:

(LU, S), where packed factors LU have the shape of A and diagonal corrections S have shape (N,) or (B, N).

Return type:

tuple[cuda.tile.Tile, cuda.tile.Tile]

nvmath.tile.linalg.qr_factor(A)#

Return packed Householder factors (QR, tau) for \(M \ge N\).

Parameters:

A (cuda.tile.Tile) – Matrix with shape (M, N) or batched matrices with shape (B, M, N), where \(M \ge N\).

Returns:

(QR, tau), where QR has the shape of A and tau has shape (N,) or (B, N).

Return type:

tuple[cuda.tile.Tile, cuda.tile.Tile]

nvmath.tile.linalg.lq_factor(A)#

Return packed Householder factors (LQ, tau) for \(M \le N\).

Parameters:

A (cuda.tile.Tile) – Matrix with shape (M, N) or batched matrices with shape (B, M, N), where \(M \le N\).

Returns:

(LQ, tau), where LQ has the shape of A and tau has shape (M,) or (B, M).

Return type:

tuple[cuda.tile.Tile, cuda.tile.Tile]

nvmath.tile.linalg.qr_generate_q(QR, tau)#

Form the explicit thin Q represented by (QR, tau).

Parameters:
  • QR (cuda.tile.Tile) – Packed QR factors with shape (M, N) or (B, M, N).

  • tau (cuda.tile.Tile) – Householder scales with shape (N,) or (B, N).

Returns:

Thin Q with shape (M, N) or (B, M, N).

Return type:

cuda.tile.Tile

nvmath.tile.linalg.lq_generate_q(LQ, tau)#

Form the explicit thin Q represented by (LQ, tau).

Parameters:
  • LQ (cuda.tile.Tile) – Packed LQ factors with shape (M, N) or (B, M, N).

  • tau (cuda.tile.Tile) – Householder scales with shape (M,) or (B, M).

Returns:

Thin Q with shape (M, N) or (B, M, N).

Return type:

cuda.tile.Tile

nvmath.tile.linalg.qr_multiply_by_q(QR, tau, C, side, trans='no_trans')#

Apply the implicit \(\operatorname{op}(Q)\) represented by (QR, tau) to C.

Parameters:
  • QR (cuda.tile.Tile) – Packed QR factors with shape (M, N) or (B, M, N).

  • tau (cuda.tile.Tile) – Householder scales with shape (N,) or (B, N).

  • C (cuda.tile.Tile) – Matrix to transform, with shape (M, K) for a left application, (K, M) for a right application, and an optional common leading batch dimension.

  • side (SideMode) – Whether to compute \(\operatorname{op}(Q) C\) or \(C \operatorname{op}(Q)\).

  • trans (TransposeMode, optional) – Whether \(\operatorname{op}(Q)\) is \(Q\) or \(Q^T\), defaulting to TransposeMode.NO_TRANS.

Returns:

Transformed matrix with the same shape as C.

Return type:

cuda.tile.Tile

nvmath.tile.linalg.lq_multiply_by_q(LQ, tau, C, side, trans='no_trans')#

Apply the implicit \(\operatorname{op}(Q)\) represented by (LQ, tau) to C.

Parameters:
  • LQ (cuda.tile.Tile) – Packed LQ factors with shape (M, N) or (B, M, N).

  • tau (cuda.tile.Tile) – Householder scales with shape (M,) or (B, M).

  • C (cuda.tile.Tile) – Matrix to transform, with shape (N, K) for a left application, (K, N) for a right application, and an optional common leading batch dimension.

  • side (SideMode) – Whether to compute \(\operatorname{op}(Q) C\) or \(C \operatorname{op}(Q)\).

  • trans (TransposeMode, optional) – Whether \(\operatorname{op}(Q)\) is \(Q\) or \(Q^T\), defaulting to TransposeMode.NO_TRANS.

Returns:

Transformed matrix with the same shape as C.

Return type:

cuda.tile.Tile

nvmath.tile.linalg.lu_solve(A, B, pivot='partial_pivot')#

Factor A and return (X, info) solving \(A X = B\).

Parameters:
  • A (cuda.tile.Tile) – Square coefficient matrix with shape (M, M) or batched matrices with shape (B, M, M).

  • B (cuda.tile.Tile) – Right-hand sides with shape (M, K) or (B, M, K).

  • pivot (PivotMode, optional) – Pivoting policy, defaulting to PivotMode.PARTIAL.

Returns:

(X, info), where X has the shape of B and info has shape () or (B,).

Return type:

tuple[cuda.tile.Tile, cuda.tile.Tile]

nvmath.tile.linalg.cholesky_solve(A, B, fill_mode)#

Factor positive-definite A and return (X, info) solving \(A X = B\).

Parameters:
  • A (cuda.tile.Tile) – Positive-definite matrix with shape (M, M) or batched matrices with shape (B, M, M).

  • B (cuda.tile.Tile) – Right-hand sides with shape (M, K) or (B, M, K).

  • fill_mode (FillMode) – Triangle of A containing the meaningful values.

Returns:

(X, info), where X has the shape of B and info has shape () or (B,).

Return type:

tuple[cuda.tile.Tile, cuda.tile.Tile]

nvmath.tile.linalg.cholesky_solve_using_factor(L, B, fill_mode)#

Return X solving \(A X = B\) with an existing Cholesky factor.

Parameters:
  • L (cuda.tile.Tile) – Cholesky factor with shape (M, M) or (B, M, M).

  • B (cuda.tile.Tile) – Right-hand sides with shape (M, K) or (B, M, K).

  • fill_mode (FillMode) – Triangle containing the factor. It must match the factorization.

Returns:

Solution with the same shape as B.

Return type:

cuda.tile.Tile

nvmath.tile.linalg.lu_solve_using_factor(LU, B, pivots=None)#

Return X solving \(A X = B\) with existing packed LU factors.

Parameters:
  • LU (cuda.tile.Tile) – Packed LU factors with shape (M, M) or (B, M, M).

  • B (cuda.tile.Tile) – Right-hand sides with shape (M, K) or (B, M, K).

  • pivots (cuda.tile.Tile, optional) – Partial-pivot indices with shape (M,) or (B, M). Omit this argument for a no-pivot factorization.

Returns:

Solution with the same shape as B.

Return type:

cuda.tile.Tile

nvmath.tile.linalg.tridiagonal_solve(dl, d, du, B)#

Solve a no-pivot tridiagonal system and return (X, info).

Parameters:
  • dl (cuda.tile.Tile) – Subdiagonal values with padded shape (N,).

  • d (cuda.tile.Tile) – Diagonal values with shape (N,).

  • du (cuda.tile.Tile) – Superdiagonal values with padded shape (N,).

  • B (cuda.tile.Tile) – Right-hand side with shape (N,) or (N, K).

Returns:

(X, info), where X has the shape of B and info has scalar shape ().

Return type:

tuple[cuda.tile.Tile, cuda.tile.Tile]

nvmath.tile.linalg.eig_hermitian_tridiagonal(d, e, V0=None, *, job)#

Solve a real symmetric tridiagonal eigensystem.

Parameters:
  • d (cuda.tile.Tile) – Diagonal values with shape (N,).

  • e (cuda.tile.Tile) – Off-diagonal values with padded shape (N,). The final element is padding.

  • V0 (cuda.tile.Tile, optional) – Input basis with shape (N, N). Required only when job is JobMode.MULTIPLY_VECTORS.

  • job (JobMode) – Whether to compute no vectors, all vectors, or vectors multiplied into V0.

Returns:

(w, info) for no vectors, otherwise (w, V, info). Eigenvalues w have shape (N,), vectors V have shape (N, N), and info has scalar shape ().

Return type:

tuple[cuda.tile.Tile, …]

nvmath.tile.linalg.eig_hermitian(A, fill_mode, *, job)#

Solve a real symmetric eigensystem using the selected triangle.

Parameters:
  • A (cuda.tile.Tile) – Symmetric matrix with shape (N, N) or batched matrices with shape (B, N, N).

  • fill_mode (FillMode) – Triangle of A containing the meaningful values.

  • job (JobMode) – Whether to compute no eigenvectors or all eigenvectors.

Returns:

(w, info) for no vectors or (w, V, info) for all vectors. Eigenvalues w have shape (N,) or (B, N), vectors V have the shape of A, and info has shape () or (B,).

Return type:

tuple[cuda.tile.Tile, …]

nvmath.tile.linalg.generalized_to_standard_eig_hermitian(A, B, eig_type, fill_mode)#

Reduce a generalized Hermitian-definite eigenproblem to a standard one.

Parameters:
  • A (cuda.tile.Tile) – Problem matrix with shape (N, N) or (Batch, N, N).

  • B (cuda.tile.Tile) – Cholesky factor of the definite matrix, with the same shape as A and produced using the same fill_mode.

  • eig_type (int) – LAPACK problem type: 1 for \(A x = w B x\), 2 for \(A B x = w x\), or 3 for \(B A x = w x\).

  • fill_mode (FillMode) – Triangle containing the Cholesky factor in B.

Returns:

Reduced standard-problem matrix with the same shape as A.

Return type:

cuda.tile.Tile

nvmath.tile.linalg.generalized_eig_hermitian(A, B, eig_type, fill_mode, *, job)#

Solve a generalized Hermitian-definite eigenproblem.

Parameters:
  • A (cuda.tile.Tile) – Problem matrix with shape (N, N) or (Batch, N, N).

  • B (cuda.tile.Tile) – Definite matrix with the same shape as A. The routine factors it internally.

  • eig_type (int) – LAPACK problem type accepted by generalized_to_standard_eig_hermitian().

  • fill_mode (FillMode) – Triangle of A and B containing meaningful data.

  • job (JobMode) – Whether to compute no eigenvectors or all eigenvectors.

Returns:

(w, info) for no vectors or (w, V, info) for all vectors. Eigenvalues w have shape (N,) or (Batch, N), vectors V have the shape of A, and info has shape () or (Batch,).

Return type:

tuple[cuda.tile.Tile, …]

nvmath.tile.linalg.svd(A, jobu, jobvt)#

Compute singular values and optionally left and right singular vectors.

Let \(R = \min(M, N)\). All returned values preserve an optional leading batch extent.

Parameters:
  • A (cuda.tile.Tile) – Matrix with shape (M, N) or batched matrices with shape (B, M, N).

  • jobu (JobMode) – Whether to compute no, thin, or all left singular vectors.

  • jobvt (JobMode) – Whether to compute no, thin, or all right singular vectors.

Returns:

(S, info) when both jobs request no vectors. Otherwise, (U, S, Vt, info). S has shape (R,) or (B, R). U has matrix shape (M, R) or (M, M) for thin or all vectors, while Vt has (R, N) or (N, N). info has shape () or (B,).

Return type:

tuple[cuda.tile.Tile, …]

nvmath.tile.linalg.svd_bidiagonal(d, e, jobu, jobvt, U0=None, Vt0=None)#

SVD of a real upper-bidiagonal matrix given d, e.

Parameters:
  • d (cuda.tile.Tile) – Diagonal values with shape (N,).

  • e (cuda.tile.Tile) – Superdiagonal values with padded shape (N,). The final element is padding.

  • jobu (JobMode) – Left singular-vector job.

  • jobvt (JobMode) – Right singular-vector job.

  • U0 (cuda.tile.Tile, optional) – Left input basis with shape (N, N). Required for paired JobMode.MULTIPLY_VECTORS jobs.

  • Vt0 (cuda.tile.Tile, optional) – Right input basis with shape (N, N). Required for paired JobMode.MULTIPLY_VECTORS jobs.

Returns:

(S, info) for paired no-vector jobs. Otherwise, (U, S, Vt, info). S has shape (N,), U and Vt have shape (N, N), and info has scalar shape ().

Return type:

tuple[cuda.tile.Tile, …]

nvmath.tile.linalg.least_squares_solve(A, B, trans='no_trans')#

Solve \(\min_X \lVert \operatorname{op}(A) X - B \rVert_2\) for the selected transpose mode.

Underdetermined systems use the minimum-norm solution.

Parameters:
  • A (cuda.tile.Tile) – Coefficient matrix with shape (M, N) or (B, M, N).

  • B (cuda.tile.Tile) – Right-hand sides with shape (M, K) for \(\operatorname{op}(A) = A\) or (N, K) for \(\operatorname{op}(A) = A^T\), plus an optional common leading batch extent.

  • trans (TransposeMode, optional) – Whether \(\operatorname{op}(A)\) is \(A\) or \(A^T\), defaulting to TransposeMode.NO_TRANS.

Returns:

Reusable (factors, tau, X). factors has the shape of A, tau has shape (R,) or (B, R) for \(R = \min(M, N)\), and X has matrix shape (N, K) without transpose or (M, K) with transpose, preserving any batch extent.

Return type:

tuple[cuda.tile.Tile, cuda.tile.Tile, cuda.tile.Tile]

nvmath.tile.linalg.triangular_solve(A, B, fill_mode, diag, side, trans='no_trans')#

Solve a triangular system with the selected structural choices.

Parameters:
  • A (cuda.tile.Tile) – Triangular matrix with shape (O, O) or (B, O, O).

  • B (cuda.tile.Tile) – Right-hand sides with shape (O, K) for a left solve or (K, O) for a right solve, plus an optional common batch extent.

  • fill_mode (FillMode) – Triangle of A containing meaningful values.

  • diag (DiagMode) – Whether the diagonal is implicit unit or explicitly stored.

  • side (SideMode) – Solve \(\operatorname{op}(A) X = B\) from the left or \(X \operatorname{op}(A) = B\) from the right.

  • trans (TransposeMode, optional) – Whether \(\operatorname{op}(A)\) is \(A\) or \(A^T\), defaulting to TransposeMode.NO_TRANS.

Returns:

Solution with the same shape as B.

Return type:

cuda.tile.Tile

nvmath.tile.linalg.tril(A)#

Return the inclusive lower triangle over the final two dimensions.

Parameters:

A (cuda.tile.Tile) – Matrix with shape (M, N) or batched matrices with shape (B, M, N).

Returns:

Lower-triangular values with the same shape and type as A.

Return type:

cuda.tile.Tile

nvmath.tile.linalg.triu(A)#

Return the inclusive upper triangle over the final two dimensions.

Parameters:

A (cuda.tile.Tile) – Matrix with shape (M, N) or batched matrices with shape (B, M, N).

Returns:

Upper-triangular values with the same shape and type as A.

Return type:

cuda.tile.Tile