Singular Value Decomposition for General Matrices#

Background#

GESVD (GEneral Singular Value Decomposition) computes the singular values and, optionally, the left and/or right singular vectors of batched general matrices using the QR algorithm.

\[A = U \Sigma V^H\]

where:

  • \(A\) is an \(M \times N\) general matrix,

  • \(\Sigma\) is an \(M \times N\) matrix that is zero except for its \(\min(M, N)\) diagonal elements, which contain the singular values of \(A\) in descending order (singular values are always real and non-negative),

  • \(U\) is an \(M \times M\) unitary matrix containing the left singular vectors of \(A\), and

  • \(V^H\) is an \(N \times N\) unitary matrix representing the conjugate transpose of \(V\), containing the right singular vectors of \(A\).

Device API#

The cuSolverDx gesvd device functions are as follows (see Execution Methods):

// Compute singular values only
__device__ void execute(data_type* A, precision_type* S, data_type* workspace, status_type* info);

// Compute singular values only with runtime leading dimension
__device__ void execute(data_type* A, const unsigned int lda, precision_type* S,
                        data_type* workspace, status_type* info);

// Compute singular values and the left/right singular vectors
__device__ void execute(data_type* A, precision_type* S, data_type* U, data_type* VH,
                        data_type* workspace, status_type* info);

// Compute singular values and the left/right singular vectors with runtime leading dimensions
__device__ void execute(data_type* A, const unsigned int lda, precision_type* S,
                        data_type* U, const unsigned int ldu,
                        data_type* VH, const unsigned int ldvh,
                        data_type* workspace, status_type* info);

Parameters:

  • A: Batched \(M \times N\) input matrix. Leading dimension is \(\mathrm{lda} \geq M\) for column-major layout, or \(\mathrm{lda} \geq N\) for row-major layout.

  • S: Array of size \(\min(M, N)\) per batch. On exit, contains the singular values of \(A\) in descending order. Since singular values are always real and non-negative, S is of type precision_type (the underlying real type) rather than data_type.

  • U: Batched left singular vectors matrix. Storage size depends on the jobu configuration (see Storage requirements for U and VH).

  • VH: Batched right singular vectors matrix (\(V^H\)). Storage size depends on the jobvt configuration (see Storage requirements for U and VH).

  • workspace: Temporary buffer preallocated by the user. The required size can be obtained via workspace_size.

  • info: Status code for each batch (see Return Status).

Note

The gesvd function returns \(V^H\) (the conjugate transpose of \(V\)) rather than \(V\) itself, consistent with standard LAPACK conventions.

Important

cuSolverDx gesvd supports both tall-skinny (\(M \geq N\)) and short-wide (\(M < N\)) matrices.

Job Options#

The Job operator controls whether singular vectors are computed and how they are stored.

Left singular vectors (jobu):

  • job::no_vectors: U is ignored; no left singular vectors are computed.

  • job::all_vectors: All M columns of left singular vectors are returned in U.

  • job::some_vectors: The first \(\min(M, N)\) columns of left singular vectors are returned in U.

  • job::overwrite_vectors: The first \(\min(M, N)\) columns of left singular vectors overwrite A.

Right singular vectors (jobvt):

  • job::no_vectors: VH is ignored; no right singular vectors are computed.

  • job::all_vectors: All N rows of right singular vectors (as \(V^H\)) are returned in VH.

  • job::some_vectors: The first \(\min(M, N)\) rows of right singular vectors (as \(V^H\)) are returned in VH.

  • job::overwrite_vectors: The first \(\min(M, N)\) rows of right singular vectors (as \(V^H\)) overwrite A.

Table 1 Storage requirements for U and VH#

Job Configuration

Size of U

Size of VH

no_vectors

Not required

Not required

all_vectors

\(M \times M\) (leading dimension \(\mathrm{ldu} \geq M\))

\(N \times N\) (leading dimension \(\mathrm{ldvh} \geq N\))

some_vectors

\(M \times \min(M, N)\) (leading dimension \(\mathrm{ldu} \geq M\) if column-major, \(\mathrm{ldu} \geq \min(M, N)\) if row-major)

\(\min(M, N) \times N\) (leading dimension \(\mathrm{ldvh} \geq \min(M, N)\) if column-major, \(\mathrm{ldvh} \geq N\) if row-major)

overwrite_vectors

Not required (output U is written to A, using the Arrangement and Leading Dimension specified for A)

Not required (output VH is written to A, using the Arrangement and Leading Dimension specified for A)

Return Status#

The function returns a status code info for each batch:

  • info = 0: The function completed successfully.

  • info > 0: The algorithm did not converge; specifically, info off-diagonal elements failed to converge to zero.

Supported Configurations#

  1. Arrangement: A, U, and VH can independently use column-major or row-major layouts. Use the Arrangement<Arr, Urr, VHrr> operator, where Arr, Urr, and VHrr specify the layouts for A, U, and VH respectively (see Arrangement operator).

  2. Leading Dimensions: Leading dimensions for A, U, and VH can be set independently using the LeadingDimension<LDA, LDU, LDVH> operator (see LeadingDimension operator).

  3. Job Options: The Job operator accepts job::no_vectors, job::all_vectors, job::some_vectors, or job::overwrite_vectors for left and right singular vectors (see Job operator):

    • job::multiply_vectors is not supported for gesvd; using it results in a compile-time error.

    • Both jobu and jobvt cannot be set to job::overwrite_vectors simultaneously; this results in a compile-time error.

    • If job::no_vectors is selected and U/VH pointers are provided, they are ignored.