cuDSS Data Types#

Opaque Data Structures#

`cudssHandle_t`#

struct cudssHandle_t#: The structure holds the cuDSS library context (device properties, system information, execution controls like cudaStream_t, etc.).

The handle must be initialized with cudssCreate() or cudssCreateMg() prior to calling any cuDSS API which takes it as an argument. The handle must be destroyed to free up resources after using cuDSS with cudssDestroy().

`cudssMatrix_t`#

struct cudssMatrix_t#: The structure is a lightweight wrapper around standard dense/sparse matrix parameters and does not own any data arrays. Matrix objects are used to pass matrix of the linear system, as well as solution and right-hand side (even if these are in fact vectors). Currently, cuDSS matrix objects can have either one of the two underlying matrix formats: dense and 3-array CSR (sparse). Additionally, they can represent non-uniform batches of matrices or distributed (in the MGMN mode).

Matrix objects should be created via cudssMatrixCreateDn() (for dense matrices) or cudssMatrixCreateBatchDn() (for a batch of dense matrices) or cudssMatrixCreateCsr() (for sparse matrices in CSR format) or cudssMatrixCreateBatchCsr() (for a batch of sparse matrices in CSR format). After use, matrix objects should be destroyed via cudssMatrixDestroy(). For distributed matrices, one can additionally call cudssMatrixSetDistributionRow1d() to define how the matrix is distributed.

Matrix objects can be modified after creation via cudssMatrixSetValues() and cudssMatrixSetCsrPointers() (and similar APIs for batches).

Information can be retrieved from a matrix object by calling cudssMatrixGetFormat() followed by either cudssMatrixGetDn() or cudssMatrixGetCsr() depending on the format returned.

`cudssData_t`#

struct cudssData_t#: The structure holds internal data (e.g., factors related data structures) as well as pointers to user-provided data. A single object of this type should be associated with solving a specific linear system. If multiple systems with the same datatype(!) are solved consecutively the object can be re-used (all necessary internal buffers will be re-created per necessity).

Note: by default, the library allocates device memory required for performing LU factorization and storing the LU factors internally. All data buffers of this kind are kept inside the data object. To change this default behavior, one can set a cudssDeviceMemHandler_t which will then be used for allocating device memory inside the solver.

The object should be created via cudssDataCreate() and destroyed via cudssDataDestroy().

During execution of any of the stages cudssExecute(), configuration settings of the solver are read from cudssConfig_t and thus affect the execution and internal data stored in the cudssData_t object.

Data parameters can be updated or retrieved by calling cudssDataSet() or cudssDataGet() respectively.

`cudssConfig_t`#

struct cudssConfig_t#: The structure stores configuration settings for the solver. This object is a lightweight (host-side) wrapper around common solver settings. While it can be re-used for solving different linear systems, it is recommended to have one per linear system.

The object should be created via cudssConfigCreate() and destroyed via cudssConfigDestroy().

During execution of any of the stages cudssExecute(), configuration settings of the solver are read from cudssConfig_t and thus affect the execution.

Configuration settings can be updated or retrieved by calling cudssConfigSet() or cudssConfigGet() respectively. Note: certain settings need to be set before a corresponding solver stage is executed (e.g., reordering algorithm must be set prior to the phase CUDSS_PHASE_ANALYSIS).

Non-opaque Data Structures#

`cudssDeviceMemHandler_t`#

struct cudssDeviceMemHandler_t#

This structure holds information about the user-provided, stream-ordered device memory pool (mempool).
The object can be created by setting the struct members described below.
Once created, a device memory handler can be set for the cuDSS library handle via cudssSetDeviceMemHandler().
Once set for the cuDSS library handle, information about the set device memory handler can be retrieved via cudssGetDeviceMemHandler().

void *ctx#: A pointer to the user-owned mempool/context object.

int device_alloc(

void *ctx,

void **ptr,

size_t size,

cudaStream_t stream

)#: A function pointer to the user-provided routine for allocating device memory of size on stream.

The allocated memory should be made accessible to the current device (or more precisely, to the current CUDA context bound to the library handle).

This interface supports any stream-ordered memory allocator ctx. Upon success, the allocated memory can be immediately used on the given stream by any operations enqueued/ordered on the same stream after this call.

It is the caller’s responsibility to ensure a proper stream order is established.

The allocated memory should be at least 256-byte aligned.

Param ctx:: [in] A pointer to the user-owned mempool object.
Param ptr:: [out] On success, a pointer to the allocated buffer.
Param size:: [in] The amount of memory in bytes to be allocated.
Param stream:: [in] The CUDA stream on which the memory is allocated (and the stream order is established)
Returns:: [out] The error status (as int) of the invocation. Must return 0 on success and any nonzero integer otherwise.

int device_free( void *ctx, void *ptr, size_t size, cudaStream_t stream )#: A function pointer to the user-provided routine for deallocating device memory of size on stream.

This interface supports any stream-ordered memory allocator. Upon success, any subsequent accesses (of the memory pointed to by the pointer ptr) ordered after this call are undefined behaviors.

It is the caller’s responsibility to ensure a proper stream order is established.

If the arguments ctx and size are not the same as those passed to device_alloc to allocate the memory pointed to by ptr, the behavior is undefined.

The argument stream need not be identical to the one used for allocating ptr, as long as the stream order is correctly established. The behavior is undefined if this assumption is not held.

Param ctx:: [in] A pointer to the user-owned mempool object.
Param ptr:: [in] The pointer to the allocated buffer.
Param size:: [in] The size of the allocated memory.
Param stream:: [in] The CUDA stream on which the memory is deallocated (and the stream order is established).
Returns:: [out] The error status (as int) of the invocation. Must return 0 on success and any nonzero integer otherwise.

char name[CUDSS_ALLOCATOR_NAME_LEN]

The name of the provided mempool (must not exceed 64 characters).

`cudssPinnedMemHandler_t`#

struct cudssPinnedMemHandler_t#

This structure holds information about the user-provided pinned (page-locked) host memory pool (mempool).
It mirrors cudssDeviceMemHandler_t, but for pinned host memory.
The object can be created by setting the struct members described below.
Once created, a pinned host memory handler can be set for the cuDSS library handle via cudssSetPinnedMemHandler().
Once set for the cuDSS library handle, information about the set pinned host memory handler can be retrieved via cudssGetPinnedMemHandler().

void *ctx#: A pointer to the user-owned mempool/context object.

Any allocation properties (for example, the flags passed to cudaHostAlloc() / cudaHostRegister(), or a huge-page / parallel pre-fault policy) are carried by ctx and are thus fixed for a given handler instance. The allocation routine may still dispatch internally based on size (for example, a registered huge-page path for large requests versus a cudaHostAlloc() path for small ones).

int pinned_alloc(

void *ctx,

void **ptr,

size_t size,

cudaStream_t stream

)#: A function pointer to the user-provided routine for allocating pinned host memory of size bytes.

The stream is the CUDA stream the data object is bound to (see cudssSetStream()). Synchronous host allocators may ignore it; stream-aware allocators may use it to establish ordering with work enqueued by cuDSS.

This routine must return pinned host memory, for example from cudaHostAlloc() or from ordinary host memory that this routine registers with cudaHostRegister() before returning. cuDSS treats the returned pointer as already pinned and does not call cudaHostRegister()/cudaHostUnregister() for it; it only tracks the buffer internally so the same size value can be passed back to pinned_free.

If the allocator uses a threading runtime internally for parallel pre-faulting before cudaHostRegister(), make sure it is compatible with any threading runtime used by a cuDSS threading layer in the same process.

Param ctx:: [in] A pointer to the user-owned mempool object.
Param ptr:: [out] On success, a pointer to the allocated buffer.
Param size:: [in] The amount of memory in bytes to be allocated.
Param stream:: [in] The CUDA stream the data object is bound to (see cudssSetStream()).
Returns:: [out] The error status (as int) of the invocation. Must return 0 on success and any nonzero integer otherwise.

int pinned_free( void *ctx, void *ptr, size_t size, cudaStream_t stream )#: A function pointer to the user-provided routine for deallocating the pinned host memory of size bytes pointed to by ptr.

The size argument is the same value that was passed to pinned_alloc(), which makes size-aware releases (for example, cudaHostUnregister() followed by munmap()) possible.

If the arguments ctx and size are not the same as those passed to pinned_alloc to allocate the memory pointed to by ptr, the behavior is undefined.

The argument stream need not be identical to the one used for allocating ptr, as long as the stream order is correctly established. The behavior is undefined if this assumption is not held.

Param ctx:: [in] A pointer to the user-owned mempool object.
Param ptr:: [in] The pointer to the allocated buffer.
Param size:: [in] The size of the allocated memory.
Param stream:: [in] The CUDA stream on which the memory is deallocated.
Returns:: [out] The error status (as int) of the invocation. Must return 0 on success and any nonzero integer otherwise.

char name[CUDSS_ALLOCATOR_NAME_LEN]

The name of the provided mempool (must not exceed 64 characters).

Double-double values (`CUDSS_R_64F_MP2`)#

Note

Double-double support is experimental. The data type and the CUDSS_R_64F_MP2 value will stay, but the algorithms behind it (double-double arithmetic, pivoting, refinement) may change between releases, and with them the numerical results.

A value of type CUDSS_R_64F_MP2 is an extended-precision (double-double) floating-point number stored as the unevaluated sum of two double components hi + lo, with hi first in memory and the pair 16-byte aligned.
The pair must be normalized: hi is the double nearest to hi + lo (in double arithmetic, hi + lo == hi), so |lo| is at most half a unit in the last place of hi and the two components do not overlap. cuDSS uses the values exactly as provided, without renormalizing them, and its double-double arithmetic and comparisons rely on this form; unnormalized input such as hi = 1.0, lo = 1.0 produces incorrect results. A double value x is represented as (x, 0.0); the sum of two doubles a + b with |a| >= |b| is normalized as hi = a + b, lo = b - (hi - a).
cuDSS does not define a struct for it in the public header; applications declare a matching type themselves, for example:

typedef struct {
    _Alignas(2 * sizeof(double)) double hi; /* high-order part */
    double lo;                              /* low-order part */
} cudss_fp64mp2_t;

Logger Types#

`cudssLoggerCallback_t`#

type cudssLoggerCallback_t#: A callback function pointer type for logging.

typedef void(*cudssLoggerCallback_t)(
    int logLevel,
    const char* functionName,
    const char* message
);

Parameters:

logLevel - The log level (0-5, see cuDSS Logging Features)
functionName - Name of the API function that generated the log
message - The log message

Enumerators#

`cudaDataType_t`#

enum cudaDataType_t#

The CUDA data type enumerator (from library_types.h) specifies the data type of values, offsets and indices used in cuDSS matrix objects and API calls.

cuDSS accepts the cudaDataType_t values CUDA_R_32F, CUDA_R_64F, CUDA_C_32F, CUDA_C_64F (matrix values) and CUDA_R_32I, CUDA_R_64I (indices and offsets), and defines the following additional values and aliases:

enumerator CUDSS_R_32F#: Alias of CUDA_R_32F: 32-bit real single-precision floating-point (float)

enumerator CUDSS_R_64F#: Alias of CUDA_R_64F: 64-bit real double-precision floating-point (double)

enumerator CUDSS_C_32F#: Alias of CUDA_C_32F: 32-bit complex single-precision floating-point (cuComplex)

enumerator CUDSS_C_64F#: Alias of CUDA_C_64F: 64-bit complex double-precision floating-point (cuDoubleComplex)

enumerator CUDSS_R_64F_MP2#: cuDSS-specific value (experimental): 128-bit real double-double floating-point, stored as the unevaluated sum of two normalized double values (hi + lo); see Double-double values (CUDSS_R_64F_MP2). CUDSS_R_64F_64F is a deprecated alias of this value.

enumerator CUDSS_R_32I#: Alias of CUDA_R_32I: 32-bit signed integer (int32_t). Used for index and offset arrays.

enumerator CUDSS_R_64I#: Alias of CUDA_R_64I: 64-bit signed integer (int64_t). Used for index and offset arrays. It does not lift the limit of \(2^{31}\) on the number of rows and columns.

enumerator CUDSS_DATA_TYPE_UNSET#: cuDSS-specific sentinel value indicating an uninitialized or unset data type. Not a valid value for user-facing API calls.

type cudssDataType_t#: Deprecated alias of cudaDataType_t, kept so that code written against cuDSS 0.8.0 keeps compiling. Using it emits a compiler deprecation warning; it will be removed in a future release.

`cudssStatus_t`#

enum cudssStatus_t#

The enumerator specifies possible status values (on the host) which can be returned from calls to cuDSS routines.

Note: device side failures are returned via CUDSS_DATA_INFO from cudssDataParam_t.

enumerator CUDSS_STATUS_SUCCESS#: The operation completed successfully.

enumerator CUDSS_STATUS_NOT_INITIALIZED#: One of the input operands was not properly initialized prior to the call to a cuDSS routine. This can usually be one of the opaque objects like cudssHandle_t, cudssData_t or others.

enumerator CUDSS_STATUS_ALLOC_FAILED#: A resource allocation failed. This is usually caused by a device or host allocation failure, and can also report a CUDA memory allocation error observed at a public API boundary.

enumerator CUDSS_STATUS_INVALID_VALUE#: An incorrect value or parameter was passed to the function (a negative vector size, or a NULL pointer for a must-have buffer, for example)

enumerator CUDSS_STATUS_NOT_SUPPORTED#: An unsupported (but otherwise reasonable) parameter was passed to the function

enumerator CUDSS_STATUS_EXECUTION_FAILED#: The GPU program failed to execute, or a non-allocation CUDA error was pending at a public API boundary.

enumerator CUDSS_STATUS_INTERNAL_ERROR#: An internal cuDSS operation failed, including an unexpected C++ exception.

enumerator CUDSS_STATUS_IR_FAILED#: Iterative refinement failed to meet the requested convergence tolerance. In this case, the iterations were performed and the refined solution is still accessible.

`cudssConfigParam_t`#

enum cudssConfigParam_t#

The enumerator specifies possible names of solver configuration settings. For each setting there is a matching type to be used in cudssConfigSet() or cudssConfigGet().

enumerator CUDSS_CONFIG_REORDERING_ALG#

Algorithm for the reordering phase. Supported options are:

CUDSS_REORDERING_ALG_DEFAULT - let the library choose (typically nested dissection).
CUDSS_REORDERING_ALG_NESTED_DISSECTION - a customized nested dissection algorithm based on METIS.
CUDSS_REORDERING_ALG_BTF_COLAMD - a custom combination of block triangular reordering and COLAMD algorithms which can be used together with global pivoting to increase solution pivots. When this option is used for reordering, cuDSS uses an appropriate custom factorization algorithm (without the need to change the factorization setting CUDSS_CONFIG_FACTORIZATION_ALG).
CUDSS_REORDERING_ALG_COLAMD - similar to CUDSS_REORDERING_ALG_BTF_COLAMD in that it implies using a special factorization algorithm which is tailored for a block-triangular representation but uses a trivial block structure.
CUDSS_REORDERING_ALG_AMD - an approximate minimum degree (AMD) reordering.
CUDSS_REORDERING_ALG_NONE - uses natural (identity) order for the internal ordering when no user permutation is supplied.

Note: CUDSS_REORDERING_ALG_BTF_COLAMD and CUDSS_REORDERING_ALG_COLAMD are only supported for general (non-symmetric or non-hermitian) matrices.

Note: CUDSS_REORDERING_ALG_BTF_COLAMD uses an upper bound on the number of non-zero entries in the factors. If this bound appears to be not sufficient during the factorization phase, a runtime device error is returned (which can be checked by synchronizing the stream and calling cudssDataGet() with CUDSS_DATA_INFO which will output the device error). In order to set a non-default upper bound, one should call cudssConfigSet() with CUDSS_CONFIG_MAX_LU_NNZ setting.

Note: CUDSS_REORDERING_ALG_AMD does not support the matrix index type int64_t.

Associated parameter type: cudssReorderingAlg_t

Default value: CUDSS_REORDERING_ALG_DEFAULT

enumerator CUDSS_CONFIG_FACTORIZATION_ALG#

Algorithm for the factorization phase. Supported options are:

CUDSS_FACTORIZATION_ALG_DEFAULT - the default factorization algorithm. This option chooses the best fitting factorization algorithm based on the choice of the reordering algorithm and sparsity structure produced by it. It is not recommended to use other options.
CUDSS_FACTORIZATION_ALG_MULTIBLOCK - multiblock factorization algorithm. Starting with cudss 0.8.0, this option is supported only for real double data type. Suggested for use with extremely sparse matrices.
CUDSS_FACTORIZATION_ALG_GENERAL - general factorization algorithm.

Associated parameter type: cudssFactorizationAlg_t

Default value: CUDSS_FACTORIZATION_ALG_DEFAULT

enumerator CUDSS_CONFIG_SOLVE_ALG#

Algorithm for the solving phase. Supported options are:

CUDSS_SOLVE_ALG_DEFAULT - the default solve algorithm.
CUDSS_SOLVE_ALG_GENERAL - triangular solve only (forward/back substitution using existing L/U factors, without modifying them).

Associated parameter type: cudssSolveAlg_t

Default value: CUDSS_SOLVE_ALG_DEFAULT, currently equivalent to CUDSS_SOLVE_ALG_GENERAL.

enumerator CUDSS_CONFIG_PIVOT_EPSILON_ALG#

Algorithm for pivot epsilon calculation. When a pivot has magnitude smaller than the pivot epsilon threshold (see the description of CUDSS_CONFIG_PIVOT_EPSILON), it is replaced by an appropriately signed value derived from the epsilon; the chosen algorithm controls how that value is computed.

Supported options are:

CUDSS_PIVOT_EPSILON_ALG_DEFAULT - Default behavior pre-selected by the library, currently equivalent to CUDSS_PIVOT_EPSILON_ALG_STATIC.
CUDSS_PIVOT_EPSILON_ALG_SCALED - Pivots with magnitude smaller than epsilon will be replaced by the appropriately signed epsilon scaled by the maximum element in the corresponding row and column of the original matrix.
CUDSS_PIVOT_EPSILON_ALG_STATIC - Pivots with magnitude smaller than epsilon will be replaced by the appropriately signed epsilon (no scaling).

Note: CUDSS_PIVOT_EPSILON_ALG_SCALED is not supported when CUDSS_CONFIG_REORDERING_ALG is set to CUDSS_REORDERING_ALG_BTF_COLAMD or CUDSS_REORDERING_ALG_COLAMD.

Associated parameter type: cudssPivotEpsilonAlg_t

Default value: CUDSS_PIVOT_EPSILON_ALG_DEFAULT

enumerator CUDSS_CONFIG_MATCHING_ALG#

Matching is an optional preprocessing step which computes a permutation to put larger values on the diagonal which often improves the accuracy of the solution. This permutation is then combined with the permutation from the reordering. Set to CUDSS_MATCHING_ALG_NONE to disable matching; for any other value, matching is enabled. While matching often reduces the number of perturbed pivots and improves accuracy of the solution (especially for ill-conditioned and badly scaled matrices) there are no guarantees that the accuracy will improve. Enabling matching brings an overhead during the analysis step and changes the non-zero pattern of the factors and therefore, can slow down factorization and solve.

For matrix types other than CUDSS_MTYPE_GENERAL (CUDSS_MTYPE_SYMMETRIC, CUDSS_MTYPE_HERMITIAN, CUDSS_MTYPE_SPD, CUDSS_MTYPE_HPD), matching preserves the matrix symmetry by default. Use CUDSS_CONFIG_FORCE_NONSYMMETRIC_MATCHING to force the non-symmetric matching path for these matrix types.

When CUDSS_PIVOT_BUNCH_KAUFMAN is selected, symmetry-preserving matching also derives 2x2 pivot candidates and keeps each candidate’s columns consecutive during nested-dissection reordering. This does not force a 2x2 numerical pivot; the Bunch-Kaufman test still selects 1x1 or 2x2 pivots from the reordered matrix. This combination requires CUDSS_REORDERING_ALG_NESTED_DISSECTION, a single process (which may use multiple GPUs), and no user permutation or user nested-dissection partition tree. Forcing non-symmetric matching in this combination is not supported.

Note: matching routines require their internal workspace size to fit into the limit of 32-bit integers. The workspace limit depend on the choice of the matching algorithm. The biggest requirement among them is to have 10 * nrows + nnz < INT_MAX (nrows and nnz for the input matrix without accounting for symmetry).

Supported options are:

CUDSS_MATCHING_ALG_NONE - matching is disabled.
CUDSS_MATCHING_ALG_MAX_DIAG_COUNT - this option is based on job = 1 from MC64 algorithm (HSL). Computes a column permutation of the matrix so that the permuted matrix has as many entries on its diagonal as possible. The values on the diagonal are of arbitrary size. Note: this option does not use matrix values and thus can actually make the accuracy worse. It is not recommended to use this algorithm for matching unless there is a justified need.
CUDSS_MATCHING_ALG_MAX_MIN_DIAG - this option is based on job = 2 from MC64 algorithm (HSL). Computes a column permutation of the matrix so that the smallest value on the diagonal of the permuted matrix is maximized.
CUDSS_MATCHING_ALG_MAX_MIN_DIAG_ALT - this option is based on job = 3 from MC64 algorithm (HSL). Computes a column permutation of the matrix so that the smallest value on the diagonal of the permuted matrix is maximized. This algorithm is different from the one used for CUDSS_MATCHING_ALG_MAX_MIN_DIAG.
CUDSS_MATCHING_ALG_MAX_DIAG_SUM - this option is based on job = 4 from MC64 algorithm (HSL). Computes a column permutation of the matrix so that the sum of the diagonal entries of the permuted matrix is maximized.
CUDSS_MATCHING_ALG_MAX_DIAG_PRODUCT - this option is based on job = 5 from MC64 algorithm (HSL). Computes a column permutation of the matrix so that the product of the diagonal entries of the permuted matrix is maximized. In addition, this algorithm computes the row/col scaling vectors which can further improve accuracy of the solution.
CUDSS_MATCHING_ALG_AUTO - the library selects the alg to apply. For CUDSS_MTYPE_SPD and CUDSS_MTYPE_HPD matrices this resolves to CUDSS_MATCHING_ALG_NONE; for all other matrix types it resolves to CUDSS_MATCHING_ALG_MAX_DIAG_PRODUCT (matching with scaling, the most robust option). Since CUDSS_MATCHING_ALG_MAX_DIAG_PRODUCT requires matrix values, CUDSS_MATCHING_ALG_AUTO falls back to CUDSS_MATCHING_ALG_NONE when values are not provided during analysis, rather than failing.

Note: one of the options, CUDSS_MATCHING_ALG_MAX_DIAG_PRODUCT, computes scaling vectors in addition to the matching permutation. This option is considered to be the most impactful and it is recommended to use this option if accuracy of the solution needs to be improved. However, this option requires matrix values to be present during the analysis step.

Matching is not supported with:

non-uniform batches;
uniform-batch settings (CUDSS_CONFIG_UBATCH_SIZE greater than 1 or CUDSS_CONFIG_UBATCH_INDEX other than -1);
CUDSS_REORDERING_ALG_BTF_COLAMD or CUDSS_REORDERING_ALG_COLAMD;
Schur complement mode;
CUDSS_PIVOT_EPSILON_ALG_SCALED;
more than \(2^{30}\) nonzeros with 32-bit matrix indices.

During reordering, CUDSS_MATCHING_ALG_AUTO disables matching for these combinations instead of failing; an explicitly enabled matching algorithm returns CUDSS_STATUS_NOT_SUPPORTED. Later configuration changes do not trigger AUTO fallback and can return CUDSS_STATUS_NOT_SUPPORTED. Rerun analysis after changing options that affect matching compatibility.

With an effective CUDSS_PIVOT_BUNCH_KAUFMAN pivot type, CUDSS_MATCHING_ALG_AUTO disables matching when it is incompatible with the Bunch-Kaufman matching requirements above. See CUDSS_PIVOT_AUTO for pivot-selection fallback rules.

Distributed inputs have additional requirements when matching runs. When row ranges do not overlap, they must cover the matrix without gaps in ascending rank order. With 32-bit matrix offsets, the sum of local nonzero counts must fit in INT32_MAX. Violations return CUDSS_STATUS_NOT_SUPPORTED, including when matching was selected by CUDSS_MATCHING_ALG_AUTO, without falling back to no matching.

For matrices with type CUDSS_R_64F_MP2, value-dependent matching algorithms evaluate magnitudes in double precision after summing the hi and lo components. Consequently, matching can ignore value differences that are representable in double-double precision but not in double precision, which can affect the selected permutation in edge cases. Scaling factors are also computed in double precision and are therefore less accurate than scaling factors computed in full double-double precision. Scaling vectors returned through CUDSS_DATA_SCALE_ROW and CUDSS_DATA_SCALE_COL use the double-double layout (Double-double values (CUDSS_R_64F_MP2)), with the double-precision scaling value stored in hi and lo set to zero.

Associated parameter type: cudssMatchingAlg_t

Default value: CUDSS_MATCHING_ALG_AUTO

enumerator CUDSS_CONFIG_FORCE_NONSYMMETRIC_MATCHING#

Controls whether matching for matrix types other than CUDSS_MTYPE_GENERAL (CUDSS_MTYPE_SYMMETRIC, CUDSS_MTYPE_HERMITIAN, CUDSS_MTYPE_SPD, CUDSS_MTYPE_HPD) should preserve symmetry or use the non-symmetric matching path. Set to 0 to preserve symmetry when matching is enabled for these matrix types. Set to 1 to force non-symmetric matching for these matrix types. This setting has no effect when matching is disabled. With CUDSS_MATCHING_ALG_AUTO, matching is disabled for SPD/HPD matrices and enabled for symmetric/Hermitian indefinite matrices when values are available during analysis, subject to the compatibility restrictions in CUDSS_CONFIG_MATCHING_ALG. Setting it to 1 together with CUDSS_PIVOT_BUNCH_KAUFMAN while matching is explicitly enabled (a non-AUTO CUDSS_CONFIG_MATCHING_ALG value) is not supported; if both come from their AUTO defaults, matching falls back to disabled instead of failing.

Associated parameter type: int

Default value: 0 (preserve symmetry)

enumerator CUDSS_CONFIG_SOLVE_MODE#

Modifier on the system matrix used during the solve phase.

Value	Solved system	Notes
`CUDSS_SOLVE_MODE_NORMAL`	\(A x = b\)	Default.
`CUDSS_SOLVE_MODE_TRANSPOSE`	\(A^T x = b\)	No effect for `CUDSS_MTYPE_SYMMETRIC` and `CUDSS_MTYPE_SPD` because \(A = A^T\).
`CUDSS_SOLVE_MODE_CONJUGATE_TRANSPOSE`	\(A^H x = b\)	Supported for GENERAL complex matrices. No effect for complex `CUDSS_MTYPE_HERMITIAN` and `CUDSS_MTYPE_HPD` because \(A = A^H\).

Associated parameter type: cudssSolveMode_t

Default value: CUDSS_SOLVE_MODE_NORMAL.

For matrix types CUDSS_MTYPE_SYMMETRIC and CUDSS_MTYPE_SPD the transposed solve is a no-op (\(A = A^T\)).

For complex matrix types CUDSS_MTYPE_HERMITIAN and CUDSS_MTYPE_HPD, the conjugate-transposed solve is a no-op (\(A = A^H\)).

Transpose solve returns CUDSS_STATUS_NOT_SUPPORTED in the following cases:

matrix types CUDSS_MTYPE_HERMITIAN and CUDSS_MTYPE_HPD because they satisfy \(A = A^H\), not necessarily \(A = A^T\);
together with the Schur complement (a non-zero CUDSS_CONFIG_SCHUR_MODE);
together with BTF / COLAMD reordering (CUDSS_REORDERING_ALG_BTF_COLAMD or CUDSS_REORDERING_ALG_COLAMD);
for non-uniform matrix batches (batchCount > 0 matrix objects).

Conjugate-transpose solve returns CUDSS_STATUS_NOT_SUPPORTED for real data types and for non-GENERAL matrix types except CUDSS_MTYPE_HERMITIAN and CUDSS_MTYPE_HPD. It otherwise shares the same Schur complement, BTF / COLAMD, and non-uniform matrix-batch restrictions as transpose solve.

enumerator CUDSS_CONFIG_IR_N_STEPS#

Maximum number of steps during the iterative refinement. Refer to CUDSS_CONFIG_IR_TOL for the details.

Note: In-place iterative refinement is not supported in deterministic mode, for non-uniform batch solves, or in MGMN mode unless both the right-hand side and solution are distributed matrices. It is supported for uniform batches, including with CUDSS_CONFIG_UBATCH_INDEX and with CUDSS_DATA_UBATCH_MASK.

Note: for a uniform batch the convergence check is evaluated on a single system – the first system selected by the current selection – and the resulting decision to stop, together with the reported number of steps actually taken, applies to the whole batch. Entries other than that one do not take part in the CUDSS_CONFIG_IR_TOL comparison.

Associated parameter type: int

Default value: 0

enumerator CUDSS_CONFIG_IR_TOL#

Iterative refinement tolerance.

When users set both CUDSS_CONFIG_IR_N_STEPS > 0 and CUDSS_CONFIG_IR_TOL > 0, the solver performs iterative refinement with convergence checking.

In each iteration, the relative residual norm is computed as \(||r||_2 / ||b||_2\) where \(r = b - Ax\).

The refinement terminates early if the residual norm is less than the specified tolerance.

If CUDSS_CONFIG_IR_TOL = 0 and CUDSS_CONFIG_IR_N_STEPS > 0, exactly CUDSS_CONFIG_IR_N_STEPS iterations are performed without convergence checking.

The actual number of iterations performed can be retrieved via CUDSS_DATA_IR_N_STEPS. If the tolerance is not met after all iterations, CUDSS_STATUS_IR_FAILED is returned.

Associated parameter type: double

Default value: 0.0 (tolerance checking disabled)

enumerator CUDSS_CONFIG_PIVOT_TYPE#

Type of pivoting

The exact meaning of this parameter depends on the choice of the reordering algorithm which can be changed through CUDSS_CONFIG_REORDERING_ALG. For CUDSS_REORDERING_ALG_BTF_COLAMD and CUDSS_REORDERING_ALG_COLAMD the parameter refers to the global pivoting, while for CUDSS_REORDERING_ALG_NESTED_DISSECTION and others it refers to the partial pivoting procedure (1x1 pivots, or 1x1 and 2x2 pivots with CUDSS_PIVOT_BUNCH_KAUFMAN).

Note that in the latter case, if the matrix type is symmetric but not positive-definite, then the pivoting is also symmetric and the pivot is searched for within the diagonal of the block (by default via Bunch-Kaufman, see CUDSS_PIVOT_BUNCH_KAUFMAN), if only the pivot type is not equal to CUDSS_PIVOT_NONE.

cuDSS resolves the effective pivot type during the reordering phase. Changing this parameter after reordering does not change the analyzed solver data; the new value takes effect on the next cudssExecute() call that includes CUDSS_PHASE_REORDERING.

For more details, see the description of the pivoting strategies.

Associated parameter type: cudssPivotType_t

Default value: CUDSS_PIVOT_AUTO.

enumerator CUDSS_CONFIG_PIVOT_THRESHOLD#

Pivoting threshold \(p_{threshold}\) which is used to determine if diagonal element is subject to pivoting and will be swapped with the maximum element in the row (or column) depending on the type of pivoting. The diagonal element will be swapped if: \(p_{threshold} \cdot max_{(sub)row \, or \, col} |a_{ij}| \geq |a_{ii}|\)

For more details, see the description of the pivoting strategies.

Note: for the global pivoting strategies this parameter is only supported when the reordering algorithm is set to CUDSS_REORDERING_ALG_BTF_COLAMD or CUDSS_REORDERING_ALG_COLAMD.

Note: when CUDSS_PIVOT_BUNCH_KAUFMAN is selected, this parameter overrides the Bunch-Kaufman \(\alpha\) constant that decides between a 1x1 and a 2x2 pivot. The value must be finite and in the range \(0 < \alpha \le 1\); otherwise cudssExecute() returns CUDSS_STATUS_INVALID_VALUE. The default Bunch-Kaufman value \(\alpha = (1 + \sqrt{17}) / 8 \approx 0.64\) is used when this parameter has not been set explicitly. Explicitly setting the parameter to 1.0 uses \(\alpha = 1\).

Associated parameter type: double

Default value: 1.0.

enumerator CUDSS_CONFIG_PIVOT_EPSILON#

Pivoting epsilon.

By default, this is the absolute value to test and replace small diagonal elements encountered during numerical factorization.

In case CUDSS_CONFIG_PIVOT_EPSILON_ALG is set to CUDSS_PIVOT_EPSILON_ALG_SCALED, this value will be additionally scaled (see more details in the description of CUDSS_CONFIG_PIVOT_EPSILON_ALG).

Associated parameter type: double

Default value: 1e-5 for single precision and 1e-13 for double precision.

enumerator CUDSS_CONFIG_MAX_LU_NNZ#

Upper limit on the number of nonzero entries in LU factors. This is only relevant for non-symmetric matrices and reordering algorithm set to CUDSS_REORDERING_ALG_BTF_COLAMD or CUDSS_REORDERING_ALG_COLAMD. If the number of non-zero entries in L and U exceeds the set limit, a runtime error occurs. See also the note for CUDSS_REORDERING_ALG_BTF_COLAMD in the table entry for CUDSS_CONFIG_REORDERING_ALG.

Associated parameter type: int64_t

Default value: -1 (then the parameter value is ignored and cuDSS uses an estimate of 100 nnz where nnz is the number of non-zero entries in the input matrix).

enumerator CUDSS_CONFIG_HYBRID_MEMORY_MODE#

Memory mode: 0 (default = device-only) or 1 (hybrid = host/device).

Note: Hybrid host/device memory mode should be enabled before the analysis phase (cudssExecute() with CUDSS_PHASE_ANALYSIS). If the decision to use the hybrid mode is done after the analysis phase, the hybrid memory mode should be enabled and analysis phase must be re-done (which is sub-optimal).

Note: Unlike the hybrid host/device execution mode (see CUDSS_CONFIG_HYBRID_EXECUTE_MODE) which controls where compute kernels are executed, the hybrid memory mode only allows cuDSS to keep a part of the factor values (internal data) on the host and always uses the GPU for factorization and solve.

For more details, see Hybrid host/device memory mode feature.

Associated parameter type: int

Default value: 0 (disabled).

Currently not supported when CUDSS_REORDERING_ALG_BTF_COLAMD or CUDSS_REORDERING_ALG_COLAMD is used for reordering, or, when CUDSS_FACTORIZATION_ALG_MULTIBLOCK is used for the factorization.

enumerator CUDSS_CONFIG_HYBRID_DEVICE_MEMORY_LIMIT#

User-defined device memory limit (number of bytes) for the hybrid memory mode.

This setting only affects execution when the hybrid host/device memory mode is enabled. For more details, see Hybrid host/device memory mode feature.

Note: In case of multiple devices (cudssCreateMg()), this value must be set for each device separately by calling cudaSetDevice() prior to cudssConfigSet().

Associated parameter type: int64_t

Default value: -1 (use the internal default heuristic).

enumerator CUDSS_CONFIG_USE_CUDA_REGISTER_MEMORY#

A flag to enable or disable usage of cudaHostRegister() by cuDSS hybrid host/device memory mode. Since the hybrid memory mode of cuDSS uses host memory to store the factors, it can use cudaHostRegister() (if the HW supports it) to speedup associated host-to-device and device-to-host memory transfers. However, registering host memory has limitations and in some cases might lead to slowdowns. If the flag is set to 0, cuDSS will not attempt to use cudaHostRegister() even if the HW supports it. If a cudssPinnedMemHandler_t is attached with cudssSetPinnedMemHandler(), registration of routed host allocations is controlled by the handler instead of this flag. Therefore, a pinned host memory handler is not supported when this flag is set to 0. This setting only affects execution when the hybrid memory mode is enabled. For more details, see Hybrid host/device memory mode feature.

Associated parameter type: int

Default value: 1 (use cudaHostRegister() if the device supports it)

enumerator CUDSS_CONFIG_HOST_NTHREADS#

Number of threads to be used by cuDSS in MT mode. This setting only affects execution when the multi-threaded mode is enabled.

Associated parameter type: int

Default value: -1 (use number of threads returned by cudssGetMaxThreads())

enumerator CUDSS_CONFIG_HYBRID_EXECUTE_MODE#

Execute mode: 0 (default = device-only) or 1 (hybrid = host/device). Hybrid execute mode allows cuDSS to perform calculations on both GPU and CPU. Currently it is used to speed up execution parts with low parallelization capacity.

Note: Reordering part of the analysis step is performed on CPU regardless of execute mode value.

Note: Hybrid execute mode should be enabled before the analysis phase (cudssExecute() with CUDSS_PHASE_ANALYSIS).

Note: Unlike the hybrid execution mode which controls where compute kernels are executed (and allows greater flexibility in placement of the input data), the hybrid host/device memory mode CUDSS_CONFIG_HYBRID_MEMORY_MODE only allows cuDSS to keep part of the factor values (internal data) on the host while only the GPU is used for factorization and solve.

Note: The input matrix can be passed with host memory pointers regardless of this setting. If hybrid execute mode is enabled, right hand side and solution can be host memory pointers as well (single-process runs with nrhs = 1 only).

For more details, see the Hybrid host/device execute mode feature.

Associated parameter type: int

Note: For nrhs > 1, the analysis phase must be run with nrhs > 1, i.e. the right-hand side passed to cudssExecute() with CUDSS_PHASE_ANALYSIS must have nrhs > 1; otherwise a later solve with nrhs > 1 can return CUDSS_STATUS_NOT_SUPPORTED. The host/device placement of the factorization and solve kernels is decided during analysis and takes nrhs from that right-hand side. The nrhs of subsequent solves does not need to match.

Note: It is strongly recommended to enable the threading layer (cudssSetThreadingLayer()) when hybrid execute mode is active, as all analysis step computations are enhanced by multi-threading. Expect significant performance improvements, especially for small and medium-sized matrices.

Default value: 0 (disabled). Currently not supported when CUDSS_CONFIG_HYBRID_MEMORY_MODE is used. Also not supported when CUDSS_CONFIG_DETERMINISTIC_MODE is enabled, or, when batchCount is greater than 1, or when right-hand side and/or solution have a non-trivial leading dimension.

enumerator CUDSS_CONFIG_ND_NLEVELS#

Minimum number of levels for the nested dissection reordering. The value for this parameter should be a positive integer. In MGMN mode and MG mode, cuDSS automatically increases the effective number of levels for an internally generated partition tree, if needed, to satisfy \(2^{n_{levels}-1} \geq \max(n_{proc}, n_{device})\), where \(n_{proc}\) is the number of processes and \(n_{device}\) is the number of devices. The configuration getter continues to return the configured value. When CUDSS_DATA_USER_ND_PARTITION_TREE is provided, the configured number of levels must already provide at least one leaf per process or device; cuDSS does not enlarge a user-provided tree. This setting only works when reordering algorithm is CUDSS_REORDERING_ALG_NESTED_DISSECTION (which is also CUDSS_REORDERING_ALG_DEFAULT).

Note: This is considered an advanced performance knob and it is recommended to use a non-default value only when optimizing for performance. Typical values to try should not be too far from the default value, e.g. from the range 8 - 11.

Associated parameter type: int

Default value: 10.

enumerator CUDSS_CONFIG_ND_UBFACTOR#

Controls the balance of the nested dissection partition tree. The value represents the allowed percentage of imbalance between the two children at each level of the partition tree. Possible values are from 1 to 100.

Note: Smaller values produce a more balanced partition tree. For MGMN mode and MG mode, it is recommended to use smaller values in the range 2 to 5.

Associated parameter type: int

Default value: 20.

enumerator CUDSS_CONFIG_EMULATION_MANTISSA_BIT_COUNT#

Enables floating-point emulation in cuDSS and controls the working precision in bits of mantissa. The value is the number of significand bits that the emulation must preserve. The original input data type (e.g. CUDSS_R_64F or CUDSS_C_64F) and the format of the produced factors are not changed; only the internal kernels switch to an emulated path.

When this setting is 0 (default), emulation is disabled and cuDSS uses the regular floating-point kernels. When set to a positive integer \(b\), cuDSS uses emulated kernels that reproduce the result with at least \(b\) bits of mantissa precision. For example, to recover a near full double-precision result (FP64 mantissa is 52 bits), values in the range of 53 to 55 are typical; smaller values trade accuracy for performance.

Note: this is a performance/accuracy tuning knob. Larger values produce more work, while smaller values are faster but less accurate. Some of the accuracy lost to a smaller value can typically be recovered by enabling iterative refinement via CUDSS_CONFIG_IR_N_STEPS (and optionally CUDSS_CONFIG_IR_TOL).

Note: emulation kernels are used only during the factorization phase. Other phases (analysis, solve, iterative refinement) continue to use the regular floating-point kernels.

Note: this setting is ignored on GPUs with compute capability 8.0 or earlier and on GPUs with compute capability 9.0.

Note: emulation is currently supported for double precision real (CUDSS_R_64F) and double precision complex (CUDSS_C_64F) data types.

Note: emulation is not supported with CUDSS_PIVOT_BUNCH_KAUFMAN when superpanels are formed.

Associated parameter type: int

Default value: 0 (emulation disabled).

enumerator CUDSS_CONFIG_UBATCH_SIZE#

The number of matrices in a uniform batch of systems to be processed by cuDSS. Uniform batch of matrices is defined as a batch of matrices with the same non-zero pattern but potentially different values. To create a uniform batch of matrices (unlike a non-uniform batch) one can simply use the usual cudssMatrixCreateCsr() or cudssMatrixCreateDn() with the only change: as a pointer to the matrix values one should provide a buffer which holds values for all matrices in the batch. Thus, it should have nnz * CUDSS_CONFIG_UBATCH_SIZE elements for sparse matrices (csr_values only) and nrows * ncols * CUDSS_CONFIG_UBATCH_SIZE elements for dense matrices.

Note: a single system can be viewed as a uniform batch with CUDSS_CONFIG_UBATCH_SIZE set to 1.

Combining this parameter with matrices created by the non-uniform batch APIs returns CUDSS_STATUS_NOT_SUPPORTED.

There are two ways how cuDSS can process a uniform batch based on the value of the CUDSS_CONFIG_UBATCH_INDEX: either factorizing (or solving) all matrices at once or just one at a time.

Benchmark factorization and solve separately when choosing a batch size: their performance crossover points can differ with the matrix and GPU.

For details, see the description of CUDSS_CONFIG_UBATCH_INDEX.

Note: CUDSS_CONFIG_UBATCH_SIZE must be set before calling CUDSS_PHASE_ANALYSIS.

Associated parameter type: int

Default value: 1.

Currently not supported when either CUDSS_CONFIG_HYBRID_MEMORY_MODE (see here) or CUDSS_CONFIG_HYBRID_EXECUTE_MODE (see here) are enabled, or MGMN mode is used, or with CUDSS_REORDERING_ALG_BTF_COLAMD and CUDSS_REORDERING_ALG_COLAMD reordering algorithms.

enumerator CUDSS_CONFIG_UBATCH_INDEX#

-1 or a 0-based index of matrix in a uniform batch which will be processed during factorization or solve phase. Special value of -1 can be used to tell cuDSS to process all matrices in the uniform batch at once.

The value must be less than CUDSS_CONFIG_UBATCH_SIZE. It can be set after CUDSS_PHASE_FACTORIZATION, which is how a whole-batch factorization is followed by solves of individual systems.

Combining this parameter with matrices created by the non-uniform batch APIs returns CUDSS_STATUS_NOT_SUPPORTED.

When CUDSS_CONFIG_UBATCH_INDEX is set, cuDSS treats the input sparse matrix, right-hand side and solution as regular matrices with just one set of values. This allows a uniform batch to be processed one system at a time while re-using the result of the analysis and keeping the factors of all the matrices in a single cudssData_t object.

The two phases differ in which descriptors cuDSS offsets by CUDSS_CONFIG_UBATCH_INDEX:

Descriptor	`CUDSS_PHASE_FACTORIZATION`	`CUDSS_PHASE_SOLVE`
Sparse matrix	Not offset. The caller points the descriptor at item `k`.	Not used.
Factors	Written to slot `k`.	Read from slot `k` (internal).
Right-hand side, solution	Not offset; a single system.	Not offset; a single system.

Note: because the factorization does not offset the matrix values, leaving the matrix descriptor at the base storage of the batch factorizes item 0 into slot k, and the following solve then returns item 0’s answer with a success status. To factorize batch item k, point the matrix descriptor at item k first, for example with cudssMatrixSetValues().

A solve may only cover systems that have factors. A factorization performed with CUDSS_CONFIG_UBATCH_INDEX set to k gives factors to item k alone, so a subsequent solve with CUDSS_CONFIG_UBATCH_INDEX reset to -1 is rejected with CUDSS_STATUS_NOT_SUPPORTED: the remaining items were never factorized. Factorize the whole batch, or keep the solve narrowed to the items that were.

Associated parameter type: int

Default value: -1

enumerator CUDSS_CONFIG_USE_SUPERPANELS#

Use superpanel optimization: 1 (default = enabled) or 0 (disabled).

Note: superpanel optimization should be enabled before the analysis phase (cudssExecute() with CUDSS_PHASE_ANALYSIS). If the decision to use the superpanel optimization mode is done after the analysis phase, the superpanel optimization should be enabled and analysis phase must be re-done which is sub-optimal).

Associated parameter type: int

Default value: 1 (enabled).

enumerator CUDSS_CONFIG_DEVICE_COUNT#

Device count in case of multiple device (see cudssCreateMg())

Note: Maximum supported device count is 16. Set this parameter before CUDSS_CONFIG_DEVICE_INDICES.

See CUDSS_CONFIG_DEVICE_INDICES for how this parameter is matched against multi-GPU handles (device_count > 1).

Associated parameter type: int

Default value: 1

enumerator CUDSS_CONFIG_DEVICE_INDICES#

A list of device indices as an integer array of length CUDSS_CONFIG_DEVICE_COUNT.

Note: Device indices must be unique and in the range 0 to cudaGetDeviceCount() - 1. Setting this parameter initializes the CUDA runtime (via cudaGetDeviceCount) and can return CUDSS_STATUS_EXECUTION_FAILED if that call fails. CUDSS_CONFIG_DEVICE_COUNT must be set before this parameter; for cudssConfigSet(), sizeInBytes must equal device_count * sizeof(int); for cudssConfigGet(), sizeInBytes must be at least that large. For multi-GPU handles (device_count > 1 from cudssCreateMg()), if either this parameter or CUDSS_CONFIG_DEVICE_COUNT is set, the resulting count and ordered indices must match the handle. Single-device handles do not enforce that match. If neither of them is set, cudssExecute() uses the handle topology. A NULL value is invalid for cudssConfigSet() (unlike cudssCreateMg(), where NULL selects 0..device_count-1).

Associated parameter type: int * (array of CUDSS_CONFIG_DEVICE_COUNT elements).

Default value: identity 0, 1, ..., device_count - 1 (after cudssConfigCreate()).

enumerator CUDSS_CONFIG_SCHUR_MODE#

Schur complement mode: 0 (default = disabled) or 1 (enabled).

For more details regarding the Schur complement feature, see Schur complement feature.

Associated parameter type: int

Default value: 0 (disabled).

Currently not supported when CUDSS_REORDERING_ALG_BTF_COLAMD or CUDSS_REORDERING_ALG_COLAMD is used for reordering, when MGMN mode or multi-GPU mode is used, or, when CUDSS_FACTORIZATION_ALG_MULTIBLOCK is used for the factorization. It is also not supported for uniform and non-uniform batches, or when matching is enabled.

enumerator CUDSS_CONFIG_DETERMINISTIC_MODE#

Enable deterministic mode. In deterministic mode, cuDSS is guaranteed to provide the bit-wise identical result at every run when executed on GPUs with the same architecture and the same number of SMs (assuming the input data and solver settings are also bit-wise identical).

Note: in deterministic mode cuDSS uses a different set of kernels which often might be slower than the kernels used in the default mode.

Associated parameter type: int

Default value: 0 (disabled).

Currently the feature is supported only for single-gpu, single rhs, and with both hybrid host/device memory mode (CUDSS_CONFIG_HYBRID_MEMORY_MODE) and hybrid host/device execute mode (CUDSS_CONFIG_HYBRID_EXECUTE_MODE) disabled. Uniform batches are not supported either: CUDSS_CONFIG_UBATCH_SIZE greater than 1 together with deterministic mode returns CUDSS_STATUS_NOT_SUPPORTED. Also, this feature does not support iterative refinement (CUDSS_CONFIG_IR_N_STEPS greater than 0) or the reordering algorithms CUDSS_REORDERING_ALG_BTF_COLAMD and CUDSS_REORDERING_ALG_COLAMD. The deterministic-mode setting used during analysis is part of the analyzed state. After analysis completes, changing this setting requires running analysis again; otherwise factorization, refactorization, or solve returns CUDSS_STATUS_INVALID_VALUE. Bunch-Kaufman pivoting does not support deterministic mode when superpanels are formed.

enumerator CUDSS_CONFIG_DOWNCAST_FACTORS#

Controls whether cuDSS stores the double-precision factors in single (reduced) precision after factorization, releasing the full-precision factors. When disabled (the default) the factors are kept in full precision.

Supported values:

CUDSS_DOWNCAST_FACTORS_MODE_DISABLED (default): disabled.
CUDSS_DOWNCAST_FACTORS_MODE_KEEP_BLOCK_DIAGONAL: store downcast factors and release the full-precision factors, retaining the full-precision diagonal block of every supernode.
CUDSS_DOWNCAST_FACTORS_MODE_KEEP_STRICT_DIAGONAL: as above but retain only the full-precision strict diagonal (one value per column). With Bunch-Kaufman pivoting, the off-diagonal of each 2x2 diagonal block is also retained in full precision for mixed solves.

Numerical factorization is performed in the matrix value precision. Iterative refinement is also computed in the matrix value precision. The supported downcast conversions are CUDSS_R_64F to CUDSS_R_32F and CUDSS_C_64F to CUDSS_C_32F.

It is possible to perform the entire solve phase in the downcast factor precision by providing right-hand side and solution matrices in the single-precision counterpart of the matrix value type. The right-hand side and solution value types must match. Transpose and conjugate-transpose solves are supported when the right-hand side and solution use the matrix value precision (mixed-precision solve), but not when they use the downcast factor precision.

For example, with a CUDSS_R_64F matrix and downcast factors enabled:

CUDSS_R_32F right-hand side and solution matrices run the entire solve in single precision over the downcast factors.
CUDSS_R_64F right-hand side and solution matrices run a mixed-precision solve with double-precision arithmetic, single-precision off-diagonal factor sources, and retained double-precision diagonal factor sources.

The corresponding complex value types are CUDSS_C_64F and CUDSS_C_32F. Any other right-hand side and solution value type returns CUDSS_STATUS_NOT_SUPPORTED at the solve phase.

CUDSS_CONFIG_DOWNCAST_FACTORS must be set before the factorization phase and must remain unchanged through every solve that reuses those factors.

Associated parameter type: cudssDowncastFactorsMode_t

Default value: CUDSS_DOWNCAST_FACTORS_MODE_DISABLED.

Downcast factors are not supported in the following configurations:

With CUDSS_REORDERING_ALG_BTF_COLAMD or CUDSS_REORDERING_ALG_COLAMD.
When CUDSS_CONFIG_DETERMINISTIC_MODE is enabled.
When CUDSS_CONFIG_SCHUR_MODE is enabled.
For transpose or conjugate-transpose solves when the right-hand side and solution use the downcast factor precision.

These unsupported configurations return CUDSS_STATUS_NOT_SUPPORTED.

enumerator CUDSS_CONFIG_MEMORY_OPTIMIZATION_SETTING#

Selects which memory optimizations cuDSS may apply. Each optimization trades some capability, or some other resource, for a lower memory footprint; none of them is applied unless it is requested here.

The value is a bitwise OR of the flags in cudssMemoryOptimizationFlags_t, which describes what each flag does and what it costs. The flags are independent and may be combined in any way; CUDSS_MEMORY_OPTIMIZATION_DEFAULT (zero) requests none of them. Setting a value with any bit that does not correspond to a defined flag returns CUDSS_STATUS_INVALID_VALUE and leaves the configuration unchanged.

The values reported by CUDSS_DATA_MEMORY_ESTIMATES are not affected by this setting: they describe the requirement before any optimization releases memory.

The setting takes effect for the factorization or refactorization phase during which it is set; setting it afterwards has no effect on a factorization that has already run.

Associated parameter type: cudssMemoryOptimizationFlags_t

Default value: CUDSS_MEMORY_OPTIMIZATION_DEFAULT.

`cudssDataParam_t`#

enum cudssDataParam_t#

The enumerator specifies possible parameter names which can set or get in the cudssData_t object. For each parameter name there is an associated type to be used in cudssDataSet() or cudssDataGet(). Each parameter name is marked with “in”, “out”, or “in, out” depending on whether a parameter can be only set, only retrieved, or both.

enumerator CUDSS_DATA_INFO#

By default, this parameter can be used to detect device-side asynchronous errors (primarily, during factorization and solve). However, in the special case, when in hybrid host/device execute mode, factorization is done on the host at least partially, this parameter can be used to retrieve host-side errors. In case execution is successful, the value is 0.

Note: the error returned via CUDSS_DATA_INFO is independent from the host side status of type cudssStatus_t returned by all cuDSS routines, including cudssExecute(). One of the noticeable use cases is when a matrix of the system is passed with an mtype for positive-definite matrices but it appears to have non-positive minors (at least numerically). In this case, calling cudssDataGet() with CUDSS_DATA_INFO after the factorization phase will return the 1-based index of the first encountered non-positive minor. Then one can set the device error back to zero via cudssDataSet(), and either change the matrix type or adjust the matrix values (to make the matrix positive-definite) and call the factorization phase again.

Note that the returned index is 1-based and is for the reordered matrix. To get the corresponding original index, first map it through the permutation returned by cudssDataGet() for CUDSS_DATA_PERM_REORDER_ROW. If symmetry-preserving matching is enabled, map that result through CUDSS_DATA_PERM_MATCHING as well; in 0-based notation the combined mapping is matching[reordering[index]].

Another case is CUDSS_PIVOT_BUNCH_KAUFMAN: if the diagonal solve encounters 2x2 pivot metadata which is internally inconsistent, the solve phase returns CUDSS_STATUS_INTERNAL_ERROR and sets CUDSS_DATA_INFO to 1. Unlike the non-positive minor above this value is not an index, and it indicates a failure inside cuDSS rather than a property of the input matrix.

Direction: in, out

Memory: host

Associated parameter type: int

enumerator CUDSS_DATA_LU_NNZ#

Number of non-zero entries in LU factors.

Note: in case batchCount > 1 (non-uniform batch) cudssDataGet() returns accumulated number over all matrices in the batch.

Direction: out

Memory: host

Associated parameter type: int64_t

enumerator CUDSS_DATA_NPIVOTS#

Number of perturbed 1x1 pivots during factorization. Stable, unperturbed 1x1 pivots and 2x2 pivot blocks are not counted.

Direction: out

Memory: host

Associated parameter type: same as for the indices of the sparse matrix of the system

enumerator CUDSS_DATA_INERTIA#

Positive and negative indices of inertia for the system matrix A (two integer values). Valid only for symmetric/Hermitian non positive-definite matrix types.

Note: in case batchCount > 1 (non-uniform batch) cudssDataGet() returns the accumulated number over all matrices in the batch.

Direction: out

Memory: host

Associated parameter type: same as for the indices of the sparse matrix of the system

enumerator CUDSS_DATA_PERM_REORDER_ROW#

Row permutation P produced by the reordering algorithm. If matching is enabled, this permutation applies to the matrix after CUDSS_DATA_PERM_MATCHING has been applied; it does not include the matching permutation.

For reordering algorithms other than CUDSS_REORDERING_ALG_BTF_COLAMD and CUDSS_REORDERING_ALG_COLAMD, the row and column reordering permutations are identical.

Note: using this parameter in cudssDataGet() in case batchCount > 1 is not supported.

Direction: out

Memory: host or device

Associated parameter type: same as for the indices of the sparse matrix of the system

enumerator CUDSS_DATA_PERM_REORDER_COL#

Column permutation Q produced by the reordering algorithm. If matching is enabled, this permutation applies to the matrix after CUDSS_DATA_PERM_MATCHING has been applied; it does not include the matching permutation.

For reordering algorithms other than CUDSS_REORDERING_ALG_BTF_COLAMD and CUDSS_REORDERING_ALG_COLAMD, the row and column reordering permutations are identical.

Note: using this parameter in cudssDataGet() in case batchCount > 1 is not supported.

Direction: out

Memory: host or device

Associated parameter type: same as for the indices of the sparse matrix of the system

enumerator CUDSS_DATA_PERM_ROW#

Final row permutation P (includes effects of both reordering and pivoting) which is applied to the original right-hand side of the system in the form \(b_{new} = b_{old} \circ P\)

Note: using this parameter in cudssDataGet() in case batchCount > 1 is not supported.

Direction: out

Memory: host or device

Associated parameter type: same as for the indices of the sparse matrix of the system

Currently supported only when CUDSS_REORDERING_ALG_BTF_COLAMD or CUDSS_REORDERING_ALG_COLAMD is used for reordering.

enumerator CUDSS_DATA_PERM_COL#

Final column permutation Q (includes effects of both reordering and pivoting) which is applied to transform the solution of the permuted system into the original solution \(x_{old} = x_{new} \circ Q^{-1}\)

Note: using this parameter in cudssDataGet() in case batchCount > 1 is not supported.

Direction: out

Memory: host or device

Associated parameter type: same as for the indices of the sparse matrix of the system

Currently supported only when CUDSS_REORDERING_ALG_BTF_COLAMD or CUDSS_REORDERING_ALG_COLAMD is used for reordering.

enumerator CUDSS_DATA_PERM_MATCHING#

Matching permutation. For non-symmetric matching, this is a column permutation Q such that A[:,Q] is reordered and then factorized. For symmetry-preserving matching, this is a symmetric permutation applied to both rows and columns.

Direction: out

Memory: host or device

Associated parameter type: same as for the indices of the sparse matrix of the system

enumerator CUDSS_DATA_DIAG#

Diagonal of the factorized matrix. The diagonal is returned in the original matrix order, taking into account all permutations applied during matching, reordering, and pivoting.

Note: when matching with scaling is enabled, the returned diagonal values retain that scaling.

Note: with Bunch-Kaufman pivoting, only the diagonal entries of D are returned; off-diagonal entries of 2x2 pivot blocks are not included.

Note: when a Schur complement is requested (CUDSS_CONFIG_SCHUR_MODE is set to 1), the rows/columns belonging to the Schur complement are not factorized, so no diagonal values exist for them. The returned buffer still has n entries, and the entries corresponding to the Schur complement are left unmodified. They are the rows/columns for which the caller set CUDSS_DATA_USER_SCHUR_INDICES to 1.

For a diagonal query using neither CUDSS_REORDERING_ALG_BTF_COLAMD nor CUDSS_REORDERING_ALG_COLAMD, the CUDA device that is current when cudssDataGet() is called must belong to the cuDSS handle; otherwise CUDSS_STATUS_INVALID_VALUE is returned.

With a Schur complement, a device output buffer must additionally belong to that current CUDA device; otherwise CUDSS_STATUS_NOT_SUPPORTED is returned. This output-buffer restriction applies only when a Schur complement is requested: without one, the whole buffer is written and it may be host memory or memory on any device.

In MGMN mode, this query is collective. Every process must configure non-null host and device communicators with the same participating processes and consistent ranks before analysis, and keep them unchanged through the query. All processes must perform the same operation: either query the required size or retrieve the value using that size. The output is written only on rank 0.

Note: in case batchCount > 1 (non-uniform batch) cudssDataGet() returns the accumulated values over all matrices in the batch.

Direction: out

Memory: host or device

Associated parameter type: same as for the values of the sparse matrix of the system

enumerator CUDSS_DATA_SCALE_ROW#

Row scaling the factorized matrix (corresponding to the rows of the original matrix)

Direction: out

Memory: host or device

Associated parameter type: the real-valued counterpart of the sparse matrix value type (float, double, or double-double, see Double-double values (CUDSS_R_64F_MP2))

Only supported when matching is enabled (i.e. CUDSS_CONFIG_MATCHING_ALG is not CUDSS_MATCHING_ALG_NONE) and the chosen matching algorithm computes the scaling. For symmetry-preserving matching, users should not assume that row and column scaling vectors are different.

enumerator CUDSS_DATA_SCALE_COL#

Column scaling the factorized matrix (corresponding to the columns of the original matrix).

Direction: out

Memory: host or device

Associated parameter type: the real-valued counterpart of the sparse matrix value type (float, double, or double-double, see Double-double values (CUDSS_R_64F_MP2)).

enumerator CUDSS_DATA_USER_PERM#

User permutation to be used instead of running the reordering algorithms. The user permutation can be disabled by providing a NULL with zero size with cudssDataSet(). The provided buffer should be an integer array of size n, where n is the number of rows/columns of the system matrix. The integer type must match the one used for the system matrix. The values should represent a valid permutation vector for the set \(\{ index base, ..., n - 1 + index base \}\)

Note: after cudssDataSet() is called with CUDSS_DATA_USER_PERM, the data are copied into an internal buffer so that the user buffer can be deallocated or re-used.

Note: using this parameter in cudssDataGet() in case batchCount > 1 is not supported.

Direction: in, out

Memory: host or device

Associated parameter type: same as for the indices of the sparse matrix of the system

Currently not supported when CUDSS_REORDERING_ALG_BTF_COLAMD or CUDSS_REORDERING_ALG_COLAMD is used for reordering.

Also not supported together with matching (i.e. CUDSS_CONFIG_MATCHING_ALG other than CUDSS_MATCHING_ALG_NONE), except for the CUDSS_PIVOT_BUNCH_KAUFMAN symmetry-preserving matching path: matching computes its own row/column permutation, which cannot be composed with a user-supplied one. With CUDSS_MATCHING_ALG_AUTO, cuDSS resolves matching to CUDSS_MATCHING_ALG_NONE instead of erroring; an explicitly requested matching algorithm returns CUDSS_STATUS_NOT_SUPPORTED.

enumerator CUDSS_DATA_ND_PARTITION_TREE#

Nested Dissection (ND) partition tree information, also known as a separator tree or separator sizes, which are computed during the reordering phase and used for improving parallelization. The number of elements in this array is \(2^{n_{levels}} - 1\) with \(n_{levels}\) from CUDSS_CONFIG_ND_NLEVELS. It can be used in combination with CUDSS_DATA_USER_PERM and CUDSS_DATA_USER_ND_PARTITION_TREE for a matrix with the same sparsity structure. This avoids regenerating the same reordering information while retaining the same performance for the factorization and solve phases.

See the Nested Dissection Partition Tree section for more details.

Direction: out

Memory: host or device

Associated parameter type: same as for the indices of the sparse matrix of the system

enumerator CUDSS_DATA_USER_ND_PARTITION_TREE#

User provided nested dissection (ND) partition tree information, which is used instead of running the reordering algorithm (therefore, saving runtime). It must be used in combination with CUDSS_DATA_USER_PERM to have an effect.

See CUDSS_DATA_ND_PARTITION_TREE for the size and usage information. The user ND partition tree can be disabled by providing a NULL with zero size with cudssDataSet().

In MGMN mode and MG mode, the configured CUDSS_CONFIG_ND_NLEVELS must provide at least one leaf per process or device. cuDSS does not enlarge a user-provided partition tree.

Note: While it is possible to construct this array outside cuDSS (e.g., extracting the so-called sizes array from METIS / ParMETIS), we recommend to only use this feature by extracting the ND partition tree from cuDSS, followed by passing it back in subsequent calls.

Direction: in, out

Memory: host or device

Associated parameter type: same as for the indices of the sparse matrix of the system.

enumerator CUDSS_DATA_MEMORY_ESTIMATES#

Memory estimates (in bytes) for host and device memory required for the chosen memory mode. The chosen memory mode is defined as the memory mode detected during the last executed analysis phase for the current cudssData_t object and the state of the corresponding cudssConfigParam_t object during the call.

See cudssConfigParam_t for more details.

Note: the returned memory estimate uses the input matrix, right-hand side, and solution descriptors and analysis-dependent settings captured during reordering or symbolic factorization. In particular, the estimate uses the number of right-hand sides, leading dimensions, batch or distributed formats, and user-provided RHS and solution buffer placement when the hybrid execute mode is enabled. For non-batch matrices, it also distinguishes an in-place solve, where the right-hand side and solution descriptors use the same value buffer, from an out-of-place solve. Analysis must be run again if these captured inputs change, including when switching between in-place and out-of-place solves. Changes to settings evaluated when the estimate is queried, such as iterative refinement or factor downcasting, are reflected by the next query and do not require another analysis.

Note: for uniform batches (CUDSS_CONFIG_UBATCH_SIZE greater than 1) cuDSS chooses between two internal workspace layouts for the factorization, and the estimate reflects the layout that the factorization is expected to use for the configuration as it stands at the time of the call. If CUDSS_DATA_UBATCH_MASK is set, or CUDSS_CONFIG_UBATCH_SIZE is changed, after the estimate has been queried, the estimate may no longer describe the layout that will be used and should be queried again. cuDSS may also fall back to the smaller layout if the larger one cannot be allocated, in which case the factorization uses less memory than estimated.

Values returned in the output array at position: * 0 - permanent device memory * 1 - peak device memory * 2 - permanent host memory * 3 - peak host memory * 4 - minimum device memory required after analysis for hybrid factorization and solve * 5 - maximum host memory over analysis, factorization, and solve in hybrid memory mode * 6 - how far the analysis phase progressed when the estimate was produced (see below) * 7, … 15 - reserved for future use

Slots 4 and 5 model the hybrid memory path even when CUDSS_CONFIG_HYBRID_MEMORY_MODE is disabled.

This query can be done once the non-zero count of the factors is known, and will return status CUDSS_STATUS_NOT_SUPPORTED if it cannot be processed.

The value at position 6 says how complete the estimate is, as one of cudssMemoryEstimateState_t.

Note: on multi-GPU (cudssCreateMg()) and multi-process runs the per-device factor counts only become available later in the analysis phase, so the query stays unavailable for longer than on a single-GPU run.

Note: in the multi-process (MGMN) mode this query is collective, including size probes with sizeInBytes == 0, and must be called by all ranks when inspecting an estimate after a failed cudssExecute(). Every rank must request the same parameter, use the same query form, and provide valid local arguments. The same applies to CUDSS_DATA_HYBRID_DEVICE_MEMORY_MIN.

Note: In case of multiple devices (cudssCreateMg()) memory estimate will be done for the calling device only. Thus, cudaSetDevice() must be called prior to cudssDataGet() to get the memory estimate for the corresponding device.

Direction: out

Memory: host

Associated parameter type: int64_t[16]

enumerator CUDSS_DATA_HYBRID_DEVICE_MEMORY_MIN#

Minimal amount of device memory (number of bytes) required in the hybrid memory mode. This query must be done after the analysis phase and will return status CUDSS_STATUS_NOT_SUPPORTED if it cannot be processed. Unlike CUDSS_DATA_MEMORY_ESTIMATES, this query is refused until the analysis phase has completed.

The returned value uses right-hand side and solution shapes. When the hybrid execute mode is enabled, it also uses user-provided RHS and solution buffer placement. For non-batch matrices, it also distinguishes an in-place solve, where the right-hand side and solution descriptors use the same value buffer, from an out-of-place solve. This information is captured during reordering or symbolic factorization. Analysis must be run again if these captured inputs change, including when switching between in-place and out-of-place solves. Changes to settings evaluated when the estimate is queried, such as iterative refinement or factor downcasting, are reflected by the next query and do not require another analysis. The query models the hybrid memory path even when CUDSS_CONFIG_HYBRID_MEMORY_MODE is disabled.

Note: in the multi-process (MGMN) mode this query is collective, including size probes with sizeInBytes == 0. Every rank must request the same parameter, use the same query form, and provide valid local arguments.

Note: In case of multiple devices (cudssCreateMg()), minimal memory will be returned for the calling device only. Thus, cudaSetDevice() must be called prior to cudssDataGet() to get the minimal memory for the corresponding device.

Direction: out

Memory: host

Associated parameter type: int64_t

For more details, see the Hybrid host/device memory mode feature.

enumerator CUDSS_DATA_FLOPS#

Floating-point operation count for the factorization phase.

Note: the value is available after the CUDSS_PHASE_ANALYSIS step.

Direction: out

Memory: host

Associated parameter type: int64_t

enumerator CUDSS_DATA_COMM_DEVICE#

Device communicator for MGMN mode. The device and host communicators must agree in size and rank ordering. The actual type of the communicator must match the device communication backend of the communication layer which must be set via calling cudssSetCommLayer(). This communicator is used for communication involving device memory buffers.

Note: cudssDataSet() stores the communicator pointer itself (the communicator is not copied), so it must remain valid until the parameter is set again or the cudssData_t object is destroyed.

Direction: in

Memory: host

Associated parameter type: void*

See MGMN mode for further details.

enumerator CUDSS_DATA_COMM_HOST#

Host communicator for MGMN mode. The actual type of the communicator must match the host communication backend of the communication layer which must be set via calling cudssSetCommLayer(). This communicator is used for communication involving host memory buffers.

Note: cudssDataSet() stores the communicator pointer itself (the communicator is not copied), so it must remain valid until the parameter is set again or the cudssData_t object is destroyed.

Direction: in

Memory: host

Associated parameter type: void*

See MGMN mode for further details.

enumerator CUDSS_DATA_NSUPERPANELS#

Number of superpanels in the matrix.

Direction: out

Memory: host

Associated parameter type: same as for the indices of the sparse matrix of the system

enumerator CUDSS_DATA_USER_SCHUR_INDICES#

User-provided Schur complement indices. The provided buffer should be an integer array of size n, where n is the number of rows (columns) of the system matrix, and the integer type should match the one used for the system matrix. The values should be equal to 1 for the rows/columns which are part of the Schur complement and 0 for the rest.

Note: after cudssDataSet() is called with CUDSS_DATA_USER_SCHUR_INDICES, the data are copied into an internal buffer so that the user buffer can be deallocated or re-used.

Direction: in

Memory: host or device

Associated parameter type: same as for the indices of the sparse matrix of the system

enumerator CUDSS_DATA_SCHUR_SHAPE#

Shape of the Schur complement matrix as a triplet (nrows, ncols, sparse nnz). The last entry is only relevant when the Schur complement needs to be exported as a sparse matrix. By querying with cudssDataGet() with CUDSS_DATA_SCHUR_SHAPE, users can get the necessary allocation sizes for the Schur complement buffers which can then be used to create a cudssMatrix_t object and passed back to cuDSS to fill in the Schur complement data via cudssDataGet() with CUDSS_DATA_SCHUR_MATRIX.

Direction: out

Memory: host

Associated parameter type: int64_t[3]

enumerator CUDSS_DATA_SCHUR_MATRIX#

Schur complement matrix as a cudssMatrix_t object.

Direction: out

Memory: host (with underlying buffers on host or device)

Associated parameter type: cudssMatrix_t

Note: the data type for matrix values, offsets and indices (if it is a sparse matrix) must be same as for the values, offsets and indices of the system matrix respectively.

Note: Schur complement matrix cannot be a distributed matrix. Also, Schur complement cannot be computed for matrix batches currently. For the dense Schur complement, the layout must be column-major. For sparse Schur complement, the matrix type must be CUDSS_MTYPE_GENERAL and the matrix view must be CUDSS_MVIEW_FULL.

enumerator CUDSS_DATA_USER_HOST_INTERRUPT#

User-provided host interrupt pointer. Setting this pointer to a non-NULL value enables user interruption of cuDSS routines on the host. Specifically, in the calls to cudssExecute() cuDSS will perform checks on the host for the value of the pointer. If it is non-zero, execution stops and cuDSS returns early (before the phase execution completes) with the status CUDSS_STATUS_EXECUTION_FAILED. To resume after an interruption, set the value at that location back to 0 and call again. To remove the (small) overhead of checking for the host interruption, the interruption can be completely disabled by setting CUDSS_DATA_USER_HOST_INTERRUPT to NULL.

Note: unlike the array parameters (e.g. CUDSS_DATA_USER_PERM), the pointer itself is stored and the pointed-to value is not copied by cudssDataSet(). The location must remain valid until the parameter is set again or the cudssData_t object is destroyed.

Note: since most of device kernels inside cuDSS are launched asynchronously, execution on the GPU may continue some time after the host execution has been interrupted.

Direction: in

Memory: host

Default value: NULL (user interruption disabled)

Associated parameter type: pointer to a 4-byte integer (e.g. int32_t or int).

Note: When setting the flag from another thread, the user must use an atomic store with release semantics so the library observes the write. A plain write to a non-atomic object from another thread is a data race and undefined behavior.

Note: std::atomic<int> (C++) or _Atomic(int)* (C) may be used when the object representation is a single 4-byte integer; the library does not guarantee correct behavior for all atomic types.

Note: The host interruption feature is not supported when MGMN mode or MG mode is used. Also, reordering algorithms CUDSS_REORDERING_ALG_BTF_COLAMD and CUDSS_REORDERING_ALG_COLAMD do not support the user interrupt currently.

enumerator CUDSS_DATA_IR_N_STEPS#

Actual number of iterative refinement steps performed

Direction: out

Memory: host

Associated parameter type: int

This parameter stores the actual number of iterative refinement iterations performed during the solve phase.

The value can be less than CUDSS_CONFIG_IR_N_STEPS if user-specified convergence tolerance is reached. The value is 0 if iterative refinement is disabled or if the initial solution already satisfies the tolerance.

enumerator CUDSS_DATA_UBATCH_MASK#

An array specifying the subset of problems to be processed in the uniform batch case, at factorization and at solve alike. It holds one int per problem in the batch, so its size must be CUDSS_CONFIG_UBATCH_SIZE * sizeof(int); a non-zero element includes the problem at that index in the operation, a zero element excludes it. Not supported for non-uniform matrix batches.

Note: CUDSS_DATA_UBATCH_MASK can be used as an alternative to CUDSS_CONFIG_UBATCH_INDEX to select multiple specific problems from the batch in a single call. The two are mutually exclusive.

Note: CUDSS_DATA_UBATCH_MASK applies to the factorization and the solve. At the solve the selected systems are solved as usual and every other entry of the solution is left exactly as the caller supplied it, neither solved nor zeroed. Clear the mask, by calling cudssDataSet() with a NULL value and a size of 0, to solve the whole batch. A mask that selects no system at all is accepted and solves nothing.

Note: a solve may only select systems that have factors. cuDSS rejects the solve with CUDSS_STATUS_NOT_SUPPORTED when the mask selects a system that no factorization since the last CUDSS_PHASE_ANALYSIS covered.

Note: the mask must be set before CUDSS_PHASE_ANALYSIS or before the first factorization, as its internal device-side storage is allocated at that point. Introducing a mask after a factorization that ran without one is not supported. This applies to a mask intended only for the solve as well: set one (an all-ones mask selects the whole batch) before the analysis, then narrow it when the solve is issued.

Note: the problems excluded by the mask keep the factors computed for them by a previous factorization; only the selected problems are refreshed. A mask that selects no problems at all is accepted and refreshes nothing, leaving all previously computed factors intact.

Note: a solve without a mask covers the whole batch and therefore requires every problem to have been factorized. cuDSS accumulates the masks of all factorizations performed since the last CUDSS_PHASE_ANALYSIS, so factorizing the batch in several masked chunks and then solving once is supported. A solve that would cover a problem no factorization ever selected returns CUDSS_STATUS_NOT_SUPPORTED, since there are no factors to solve it with; setting a mask on the solve narrows both what is solved and what is therefore required.

Note: when CUDSS_CONFIG_DOWNCAST_FACTORS is enabled, the mask must select a single contiguous range of problems, and the value reported by CUDSS_DATA_MEMORY_ESTIMATES depends on the mask that is set when the estimate is queried. The transient full-precision factors are sized for the selected range, so querying with a narrow selection and then factorizing a wider one under-reports the peak. Query the estimate with the widest selection you intend to use.

Note: the mask can be retrieved with cudssDataGet() once the data object has been used in a call to cudssExecute().

Direction: in, out

Memory: host

Associated parameter type: int * (array of CUDSS_CONFIG_UBATCH_SIZE elements).

Default value: NULL (no mask; all problems selected).

Not supported when CUDSS_CONFIG_HYBRID_MEMORY_MODE or CUDSS_CONFIG_HYBRID_EXECUTE_MODE is enabled, in MG mode or MGMN mode, or with CUDSS_REORDERING_ALG_BTF_COLAMD or CUDSS_REORDERING_ALG_COLAMD.

`cudssPhase_t`#

enum cudssPhase_t#

The enumerator specifies the solver phases to be performed in the main cuDSS routine cudssExecute(). Phases can be combined with the binary OR operator (|) to specify multiple phases at once: CUDSS_PHASE_FACTORIZATION | CUDSS_PHASE_SOLVE.

enumerator CUDSS_PHASE_REORDERING#: Reordering

enumerator CUDSS_PHASE_SYMBOLIC_FACTORIZATION#

Symbolic factorization

Note: it is not allowed to call symbolic factorization twice without calling the reordering phase in-between.

enumerator CUDSS_PHASE_ANALYSIS#: Reordering and symbolic factorization combined

enumerator CUDSS_PHASE_FACTORIZATION#: Numerical factorization

enumerator CUDSS_PHASE_REFACTORIZATION#

Numerical re-factorization.

Note: For now it is only used if the reordering algorithm is set to CUDSS_REORDERING_ALG_BTF_COLAMD or CUDSS_REORDERING_ALG_COLAMD. Otherwise, it is the same as FACTORIZATION phase.

Note: this phase returns CUDSS_STATUS_NOT_SUPPORTED if CUDSS_MEMORY_OPTIMIZATION_SOLVE_ONLY was set for the preceding factorization.

enumerator CUDSS_PHASE_SOLVE_FWD_PERM#

Applying reordering permutation to the right hand side before the forward solve (forward substitution).

Note: solve sub-phases are not supported when CUDSS_REORDERING_ALG_BTF_COLAMD or CUDSS_REORDERING_ALG_COLAMD are used for reordering.

enumerator CUDSS_PHASE_SOLVE_FWD#

Forward substitution sub-step of the solving phase.

Note: this phase includes the local permutation due to the partial pivoting. To remove this effect, if undesired, one can use a combination of reordering and final permutation (see CUDSS_DATA_PERM_REORDER_ROW and CUDSS_DATA_PERM_ROW).

Note: solve sub-phases are not supported when CUDSS_REORDERING_ALG_BTF_COLAMD or CUDSS_REORDERING_ALG_COLAMD are used for reordering.

enumerator CUDSS_PHASE_SOLVE_DIAG#

Diagonal solve sub-step of the solving phase (only needed for symmetric/Hermitian matrix types).

Note: solve sub-phases are not supported when CUDSS_REORDERING_ALG_BTF_COLAMD or CUDSS_REORDERING_ALG_COLAMD are used for reordering.

enumerator CUDSS_PHASE_SOLVE_BWD#

Backward substitution sub-step of the solving phase.

Note: solve sub-phases are not supported when CUDSS_REORDERING_ALG_BTF_COLAMD or CUDSS_REORDERING_ALG_COLAMD are used for reordering.

enumerator CUDSS_PHASE_SOLVE_BWD_PERM#

Applying inverse reordering permutation to the intermediate solution after the backward solve (backward substitution). If matching (and scaling) is enabled, this phase also includes applying the inverse matching permutation and inverse scaling (as the matching permutation and scalings were used to modify the matrix before the factorization).

Note: solve sub-phases are not supported when CUDSS_REORDERING_ALG_BTF_COLAMD or CUDSS_REORDERING_ALG_COLAMD are used for reordering.

enumerator CUDSS_PHASE_SOLVE_REFINEMENT#

Iterative refinement (see configuration settings CUDSS_CONFIG_IR_N_STEPS and CUDSS_CONFIG_IR_TOL).

Note: solve sub-phases are not supported when CUDSS_REORDERING_ALG_BTF_COLAMD or CUDSS_REORDERING_ALG_COLAMD are used for reordering.

enumerator CUDSS_PHASE_SOLVE#

Full solving phase, combining all of the above (forward permutation + forward solve + diagonal solve + backward solve + backward permutation) and (optional) iterative refinement.

Note: combining the solve sub-phases should be preferred to calling sub-phases separately.

Note: Changing the sparse matrix for this phase is allowed but with restrictions, see limitations for cudssExecute().

`cudssMatrixFormat_t`#

enum cudssMatrixFormat_t#

The enumerator specifies the underlying matrix formats inside a cuDSS matrix object.

enumerator CUDSS_MFORMAT_DENSE#: Matrix is dense (applied to a single matrix and a batch equally)

enumerator CUDSS_MFORMAT_CSR#

Matrix is in CSR format (applies to a single matrix and a batch equally)

Note: Only 3-array CSR is supported.

enumerator CUDSS_MFORMAT_BATCH#: Matrix object represents a batch of matrices

enumerator CUDSS_MFORMAT_DISTRIBUTED#: Matrix object represents a distributed matrix (used in MGMN mode)

Note: The format flags can be combined. E.g., creating a cudssMatrix_t via cudssMatrixCreateBatchCsr() would set both CUDSS_MFORMAT_CSR and CUDSS_MFORMAT_BATCH flags. One can check for a mixture of flags via bit-wise operations, e.g. CUDSS_MFORMAT_CSR | CUDSS_MFORMAT_BATCH for a batch of CSR matrices.

`cudssMatrixType_t`#

enum cudssMatrixType_t#

The enumerator specifies available matrix types for sparse matrices. Matrix type should be used to describe the properties of the underlying matrix storage. Matrix type affects the decision about what type of factorization is computed by the solver. E.g, when matrix type is one of the positive-definite types, checks for singular values on the diagonal is not done.

enumerator CUDSS_MTYPE_GENERAL#: General matrix [default] LDU factorization will be computed with optional local or global pivoting

enumerator CUDSS_MTYPE_SYMMETRIC#: Real or complex symmetric matrix. LDL^T factorization will be computed with optional local pivoting

enumerator CUDSS_MTYPE_HERMITIAN#: Complex Hermitian matrix. LDL^H factorization will be computed with optional local pivoting

enumerator CUDSS_MTYPE_SPD#

Symmetric positive-definite matrix Cholesky factorization will be computed with optional local pivoting

Note: if the matrix passed with this matrix type appears to have zero minors (at least numerically), one can get the 1-based index of the first encountered non-positive minor by calling cudssDataGet() with CUDSS_DATA_INFO. As this would be a device-side error, the call to cudssExecute() may still return CUDSS_STATUS_SUCCESS.

enumerator CUDSS_MTYPE_HPD#

Hermitian positive-definite matrix Complex Cholesky factorization will be computed with optional local pivoting

Note: if the matrix passed with this matrix type appears to have zero minors (at least numerically), one can get the 1-based index of the first non-positive minor by calling cudssDataGet() with CUDSS_DATA_INFO. As this would be a device-side error, the call to cudssExecute() may still return CUDSS_STATUS_SUCCESS.

`cudssMatrixViewType_t`#

enum cudssMatrixViewType_t#

The enumerator specifies available matrix view types for sparse matrices. Matrix view defines how the matrix is treated by the main cuDSS routine cudssExecute(). E.g., to provide only upper-triangle data for a symmetric matrix one can use as CUDSS_MTYPE_SYMMETRIC as matrix type combined with CUDSS_MVIEW_UPPER as the matrix view. If the accompanying matrix type is CUDSS_MTYPE_GENERAL, the matrix view is ignored.

enumerator CUDSS_MVIEW_FULL#: Full matrix [default]

enumerator CUDSS_MVIEW_LOWER#: Lower-triangular matrix (including the diagonal). All values above the main diagonal will be ignored.

enumerator CUDSS_MVIEW_UPPER#: Upper-triangular matrix (including the diagonal). All values below the main diagonal will be ignored.

`cudssIndexBase_t`#

enum cudssIndexBase_t#

The enumerator specifies indexing base (0 or 1) for sparse matrix indices (row start/end offsets and column indices). Once set for a sparse matrix, cudssExecute() will use the indexing base from the input sparse matrix for all index-related data (e.g. output from cudssDataGet() called with CUDSS_DATA_PERM_REORDER_ROW or CUDSS_DATA_PERM_REORDER_COL).

enumerator CUDSS_BASE_ZERO#: Zero-based indexing [default]

enumerator CUDSS_BASE_ONE#: One-based indexing

`cudssLayout_t`#

enum cudssLayout_t#

The enumerator specifies dense matrix layout.

enumerator CUDSS_LAYOUT_COL_MAJOR#: Column-major layout [default]

enumerator CUDSS_LAYOUT_ROW_MAJOR#: Row-major layout. Currently not supported.

`cudssReorderingAlg_t`#

enum cudssReorderingAlg_t#

The enumerator specifies the reordering algorithm for the analysis phase.

Used with CUDSS_CONFIG_REORDERING_ALG.

enumerator CUDSS_REORDERING_ALG_DEFAULT#: The default algorithm for reordering (equivalent to CUDSS_REORDERING_ALG_NESTED_DISSECTION).

enumerator CUDSS_REORDERING_ALG_BTF_COLAMD#: Block triangular form (BTF) combined with COLAMD. Supports global pivoting.

enumerator CUDSS_REORDERING_ALG_COLAMD#: COLAMD with trivial block structure. Supports global pivoting.

enumerator CUDSS_REORDERING_ALG_AMD#: Approximate minimum degree (AMD) reordering.

enumerator CUDSS_REORDERING_ALG_NESTED_DISSECTION#: Nested dissection algorithm based on METIS.

enumerator CUDSS_REORDERING_ALG_NONE#: Uses natural (identity) order for the internal ordering when no user permutation is supplied.

`cudssFactorizationAlg_t`#

enum cudssFactorizationAlg_t#

The enumerator specifies the factorization algorithm for the numerical factorization phase.

Used with CUDSS_CONFIG_FACTORIZATION_ALG.

enumerator CUDSS_FACTORIZATION_ALG_DEFAULT#: The default factorization algorithm (equivalent to CUDSS_FACTORIZATION_ALG_GENERAL, recommended).

enumerator CUDSS_FACTORIZATION_ALG_GENERAL#: General factorization algorithm.

enumerator CUDSS_FACTORIZATION_ALG_MULTIBLOCK#: Multiblock factorization algorithm. Supported only for real double data type (CUDSS_R_64F). Suggested for use with extremely sparse matrices.

`cudssSolveAlg_t`#

enum cudssSolveAlg_t#

The enumerator specifies the algorithm for the solve phase.

Used with CUDSS_CONFIG_SOLVE_ALG.

enumerator CUDSS_SOLVE_ALG_DEFAULT#: Let the library choose the solve algorithm (recommended).

enumerator CUDSS_SOLVE_ALG_GENERAL#: Triangular solve only: forward/back substitution using the existing L/U factors without modifying them.

`cudssSolveMode_t`#

enum cudssSolveMode_t#

The enumerator specifies which system matrix modifier is used in the solve phase.

Used with CUDSS_CONFIG_SOLVE_MODE.

enumerator CUDSS_SOLVE_MODE_NORMAL#: Solve \(A x = b\).

enumerator CUDSS_SOLVE_MODE_TRANSPOSE#: Solve \(A^T x = b\).

enumerator CUDSS_SOLVE_MODE_CONJUGATE_TRANSPOSE#: Solve \(A^H x = b\).

`cudssMemoryEstimateState_t`#

enum cudssMemoryEstimateState_t#

The enumerator specifies how complete a memory estimate is, i.e. how far the analysis phase had progressed when it was produced.

Returned at position 6 of CUDSS_DATA_MEMORY_ESTIMATES.

enumerator CUDSS_MEMORY_ESTIMATE_STATE_NONE#: Internal state before the non-zero count of the factors is known; it is never returned at position 6. A query in this state returns CUDSS_STATUS_NOT_INITIALIZED before the first cudssExecute() call on the data object, and after one that failed early enough to leave it unusable; otherwise it returns CUDSS_STATUS_NOT_SUPPORTED. In particular, an estimate is not available after the reordering phase alone: until the fill-in is known any number would exclude the factors, which dominate the requirement.

enumerator CUDSS_MEMORY_ESTIMATE_STATE_FILL_KNOWN#: Positions 0-3 include the factors, but the estimate is preliminary because later analysis steps have not sized every workspace. When a cudssData_t object is reused, it can also include allocation capacity retained from its preceding analysis. Do not interpret this value as either a lower or an upper bound. On multi-GPU (cudssCreateMg()) and multi-process runs this state is only reached once the per-device split exists, and positions 0-3 then hold that per-device requirement. Positions 4-5 (the hybrid memory mode minimums) are reported as 0 below CUDSS_MEMORY_ESTIMATE_STATE_FINAL.

enumerator CUDSS_MEMORY_ESTIMATE_STATE_FINAL#: The analysis phase completed and positions 0-5 are the final estimate.

`cudssDowncastFactorsMode_t`#

enum cudssDowncastFactorsMode_t#

Selects how cuDSS keeps the numeric factors after factorization. By default (CUDSS_DOWNCAST_FACTORS_MODE_DISABLED) the factors are kept in full precision. The KEEP modes instead store the factors in single (reduced) precision and release the full-precision factors.

Used with CUDSS_CONFIG_DOWNCAST_FACTORS.

enumerator CUDSS_DOWNCAST_FACTORS_MODE_DISABLED#: Keep the factors in full precision (no downcasting).

enumerator CUDSS_DOWNCAST_FACTORS_MODE_KEEP_BLOCK_DIAGONAL#: Store the factors in reduced (single) precision and release the full-precision factors, retaining the full-precision diagonal block of every supernode.

enumerator CUDSS_DOWNCAST_FACTORS_MODE_KEEP_STRICT_DIAGONAL#: Store the factors in reduced (single) precision and release the full-precision factors, retaining only the full-precision strict diagonal (one value per column). With Bunch-Kaufman pivoting, the off-diagonal of each 2x2 diagonal block is also retained in full precision for mixed solves.

`cudssMemoryOptimizationFlags_t`#

enum cudssMemoryOptimizationFlags_t#

Flags naming the memory optimizations cuDSS may apply. The flags are independent and are combined with a bitwise OR; they are not positions on a scale, and new flags may be added without changing the meaning of the existing ones.

Used with CUDSS_CONFIG_MEMORY_OPTIMIZATION_SETTING.

enumerator CUDSS_MEMORY_OPTIMIZATION_DEFAULT#: No additional memory optimizations are applied.

enumerator CUDSS_MEMORY_OPTIMIZATION_SOLVE_ONLY#

Declares that the factors will not be recomputed, so the state which only a further factorization would read is released at the end of the factorization phase instead of being held through the solve. Solve phase performance may be slower.

A subsequent CUDSS_PHASE_FACTORIZATION or CUDSS_PHASE_REFACTORIZATION on the same cudssData_t returns CUDSS_STATUS_NOT_SUPPORTED; running CUDSS_PHASE_ANALYSIS again makes factorization legal once more. Any number of solves may still be performed. The refusal depends on whether the state was actually released, not on the current setting, so clearing the flag afterwards does not lift it.

Unsupported configurations are:

A partial uniform-batch factorization using CUDSS_DATA_UBATCH_MASK or CUDSS_CONFIG_UBATCH_INDEX, because the state released after the first pass is what the remaining passes would need.
CUDSS_REORDERING_ALG_BTF_COLAMD or CUDSS_REORDERING_ALG_COLAMD reordering.

These configurations return CUDSS_STATUS_NOT_SUPPORTED from cudssExecute().

Eligible single-GPU, single-process, in-core symmetric-family configurations may release additional solve state. After it has been released, changing the configuration to require an incompatible solve path returns CUDSS_STATUS_NOT_SUPPORTED.

`cudssPivotEpsilonAlg_t`#

enum cudssPivotEpsilonAlg_t#

The enumerator specifies the algorithm for pivot epsilon calculation when replacing small pivots during numerical factorization.

Used with CUDSS_CONFIG_PIVOT_EPSILON_ALG.

enumerator CUDSS_PIVOT_EPSILON_ALG_DEFAULT#: The default pivot epsilon algorithm (same as CUDSS_PIVOT_EPSILON_ALG_STATIC, recommended).

enumerator CUDSS_PIVOT_EPSILON_ALG_SCALED#: Pivots below the threshold are replaced by the appropriately signed epsilon scaled by the maximum element in the corresponding row and column of the original matrix. Not supported with CUDSS_REORDERING_ALG_BTF_COLAMD or CUDSS_REORDERING_ALG_COLAMD.

enumerator CUDSS_PIVOT_EPSILON_ALG_STATIC#: Pivots below the threshold are replaced by the appropriately signed epsilon (no scaling). Recommended for most cases.

`cudssMatchingAlg_t`#

enum cudssMatchingAlg_t#

The enumerator specifies the matching algorithm.

Used with CUDSS_CONFIG_MATCHING_ALG. Except for CUDSS_MATCHING_ALG_NONE and CUDSS_MATCHING_ALG_AUTO, all options are based on the MC64 algorithm (HSL).

For matrix types other than CUDSS_MTYPE_GENERAL (CUDSS_MTYPE_SYMMETRIC, CUDSS_MTYPE_HERMITIAN, CUDSS_MTYPE_SPD, CUDSS_MTYPE_HPD), matching preserves symmetry by default.

enumerator CUDSS_MATCHING_ALG_NONE#: Matching is disabled.

enumerator CUDSS_MATCHING_ALG_AUTO#: Same as CUDSS_MATCHING_ALG_MAX_DIAG_PRODUCT (the most robust option). Matching with scaling; requires matrix values during analysis.

enumerator CUDSS_MATCHING_ALG_MAX_DIAG_COUNT#: Column permutation to maximize the number of diagonal entries (values arbitrary). MC64 JOB=1. Does not use matrix values; not recommended unless justified.

enumerator CUDSS_MATCHING_ALG_MAX_MIN_DIAG#: Column permutation to maximize the smallest value on the diagonal. MC64 JOB=2.

enumerator CUDSS_MATCHING_ALG_MAX_MIN_DIAG_ALT#: Alternate algorithm to maximize the smallest value on the diagonal. MC64 JOB=3. May differ in performance from CUDSS_MATCHING_ALG_MAX_MIN_DIAG.

enumerator CUDSS_MATCHING_ALG_MAX_DIAG_SUM#: Column permutation to maximize the sum of diagonal entries. MC64 JOB=4.

enumerator CUDSS_MATCHING_ALG_MAX_DIAG_PRODUCT#: Column permutation to maximize the product of diagonal entries; also computes row/column scaling so that nonzero diagonal entries are 1 in absolute value and off-diagonal entries are ≤ 1. MC64 JOB=5. Most impactful for accuracy; requires matrix values during analysis.

`cudssPivotType_t`#

enum cudssPivotType_t#

The enumerator specifies the pivoting strategy for numerical factorization.

enumerator CUDSS_PIVOT_AUTO#

Automatic pivoting strategy [default]. Automatically selects the appropriate pivot type based on the reordering algorithm and matrix type:

For CUDSS_REORDERING_ALG_NESTED_DISSECTION, CUDSS_REORDERING_ALG_AMD, and CUDSS_REORDERING_ALG_NONE:
- Symmetric/Hermitian indefinite matrices: Uses CUDSS_PIVOT_BUNCH_KAUFMAN whenever it is supported for the requested configuration (see below); otherwise falls back to CUDSS_PIVOT_DIAGONAL
- General matrices: Uses CUDSS_PIVOT_LOCAL_BLOCK
- SPD/HPD matrices: Uses CUDSS_PIVOT_DIAGONAL (numerical pivoting is effectively disabled, since the matrix type indicates no pivoting is needed)
For CUDSS_REORDERING_ALG_BTF_COLAMD and CUDSS_REORDERING_ALG_COLAMD: Uses CUDSS_PIVOT_GLOBAL_COL

Unlike an explicit CUDSS_PIVOT_BUNCH_KAUFMAN request, AUTO never fails due to a rejected combination in Symmetric/Hermitian indefinite matrices: it falls back to CUDSS_PIVOT_DIAGONAL instead of selecting CUDSS_PIVOT_BUNCH_KAUFMAN whenever the requested configuration includes batching (CUDSS_DATA_NSUPERPANELS-style non-uniform batching, or uniform batching combined with CUDSS_CONFIG_HYBRID_MEMORY_MODE/ CUDSS_CONFIG_HYBRID_EXECUTE_MODE), an explicitly requested matching algorithm that would not use the Bunch-Kaufman symmetry-preserving matching path (see CUDSS_PIVOT_BUNCH_KAUFMAN), or CUDSS_CONFIG_DETERMINISTIC_MODE/CUDSS_CONFIG_EMULATION_MANTISSA_BIT_COUNT combined with superpanels enabled. See CUDSS_PIVOT_BUNCH_KAUFMAN for the full list of restrictions that still apply when it is requested explicitly.

enumerator CUDSS_PIVOT_NONE#: No pivoting. Used only with reordering algorithms CUDSS_REORDERING_ALG_NESTED_DISSECTION and CUDSS_REORDERING_ALG_AMD (MetisND and AMD).

enumerator CUDSS_PIVOT_GLOBAL_COL#: Global column pivoting. Used only with reordering algorithms CUDSS_REORDERING_ALG_BTF_COLAMD and CUDSS_REORDERING_ALG_COLAMD. Searches for pivot elements within entire columns of the sub-matrix.

enumerator CUDSS_PIVOT_GLOBAL_ROW#: Global row pivoting. Used only with reordering algorithms CUDSS_REORDERING_ALG_BTF_COLAMD and CUDSS_REORDERING_ALG_COLAMD. Searches for pivot elements within entire rows of the sub-matrix.

enumerator CUDSS_PIVOT_DIAGONAL#: Diagonal pivoting. Used only with reordering algorithms CUDSS_REORDERING_ALG_NESTED_DISSECTION and CUDSS_REORDERING_ALG_AMD (MetisND and AMD). For symmetric/Hermitian indefinite matrices, searches for pivot elements only within the diagonal of the supernode.

enumerator CUDSS_PIVOT_LOCAL_BLOCK#: Complete block pivoting. Used only with reordering algorithms CUDSS_REORDERING_ALG_NESTED_DISSECTION and CUDSS_REORDERING_ALG_AMD (MetisND and AMD). For general matrices, searches for pivot elements within the entire diagonal block of the supernode.

enumerator CUDSS_PIVOT_BUNCH_KAUFMAN#

Bunch-Kaufman pivoting for symmetric/Hermitian indefinite matrices. Unlike CUDSS_PIVOT_DIAGONAL, which only searches the diagonal and perturbs tiny pivots, Bunch-Kaufman additionally forms stable 2x2 diagonal pivot blocks, which improves accuracy on indefinite matrices where a 1x1 diagonal pivot would be unstable. The 1x1-vs-2x2 selection threshold is controlled by CUDSS_CONFIG_PIVOT_THRESHOLD (see its description).

Solves using this pivoting strategy are synchronous and cannot be stream-captured.

At each numerical pivot step, Bunch-Kaufman applies this test inside the current local active block. For pivot position \(j\), let \(\lambda = \max_{i>j} |a_{ij}|\), and let \(r\) be the row where that maximum is attained. If \(|a_{jj}| \ge \alpha \lambda\), cuDSS uses a 1x1 pivot at \(j\). Otherwise it computes \(\sigma = \max_{k \ge j, k \ne r} |a_{rk}|\). If \(|a_{jj}| \sigma \ge \alpha \lambda^2\), cuDSS again uses a 1x1 pivot at \(j\); else if \(|a_{rr}| \ge \alpha \sigma\), it interchanges \(r\) with \(j\) and uses a 1x1 pivot. In the remaining case, it interchanges \(r\) next to \(j\) and forms a 2x2 pivot block.

The number of perturbed 1x1 pivots can be queried after factorization via CUDSS_DATA_NPIVOTS.

The following combinations are currently rejected with CUDSS_STATUS_NOT_SUPPORTED:

matrix types other than CUDSS_MTYPE_SYMMETRIC and CUDSS_MTYPE_HERMITIAN;
reordering algorithms other than CUDSS_REORDERING_ALG_NESTED_DISSECTION, CUDSS_REORDERING_ALG_AMD (MetisND and AMD), or CUDSS_REORDERING_ALG_NONE;
non-uniform batches;
hybrid memory or hybrid execute mode with CUDSS_CONFIG_UBATCH_SIZE greater than 1;
deterministic mode when superpanels are formed;
fp64 emulation (CUDSS_CONFIG_EMULATION_MANTISSA_BIT_COUNT > 0) when superpanels are formed.

Hybrid execute mode disables superpanel creation, so a request that combines hybrid execute mode and CUDSS_CONFIG_USE_SUPERPANELS is not rejected by the superpanel restriction unless CUDSS_CONFIG_UBATCH_SIZE is greater than 1. Bunch-Kaufman supports all placements supported by hybrid execute mode.

When matching is enabled, Bunch-Kaufman additionally requires symmetry-preserving matching, CUDSS_REORDERING_ALG_NESTED_DISSECTION, a single process, and no user permutation or user nested-dissection partition tree. A single process may use multiple GPUs.

Communication Layer (Distributed Interface) Types#

`cudssDistributedInterface_t`#

struct cudssDistributedInterface_t#

This struct defines all communication primitives which need to be implemented in (any) implementation of cuDSS communication layer, see for more details communication layer and MGMN mode.

Dual-backend structure: Each communication primitive must be implemented twice — once for device (GPU) memory buffers (function pointer name ending in Device) and once for host (CPU) memory buffers (function pointer name ending in Host). Both variants have identical signatures. cuDSS selects the appropriate variant based on the memory location of the buffers at call time. For example, rank query has two function pointers: cudssCommRankDevice and cudssCommRankHost.

Note: the comm argument to all functions is interpreted by the implementation based on the underlying communication backend. E.g., for OpenMPI it should be treated as an MPI_Comm.

Note: functions with a stream argument must be stream-ordered. For GPU-aware backends like OpenMPI, this implies performing an explicit cudaStreamSynchronize() inside the implementation.

int cudssCommRankDevice(void *comm, int *rank)#
int cudssCommRankHost(void *comm, int *rank)#: Function pointers to a routine which returns the rank of the process in a communicator.

Param comm:: [in] A pointer to the communicator.
Param rank:: [out] Rank of the calling process in the communicator.
Returns:: [out] The error status (as int) of the invocation. Must return 0 on success and any nonzero integer otherwise.

int cudssCommSizeDevice(void *comm, int *size)#
int cudssCommSizeHost(void *comm, int *size)#: Function pointers to a routine which returns the number of processes in a communicator.

Param comm:: [in] A pointer to the communicator.
Param size:: [out] Number of processes in the communicator.
Returns:: [out] The error status (as int) of the invocation. Must return 0 on success and any nonzero integer otherwise.

int cudssSendDevice( const void *buffer, int count, cudaDataType_t datatype, int dest, int tag, void *comm, cudaStream_t stream )#
int cudssSendHost( const void *buffer, int count, cudaDataType_t datatype, int dest, int tag, void *comm, cudaStream_t stream )#: Function pointers to a routine which performs a blocking send.

Param buffer:: [in] Initial address of the send buffer.
Param count:: [in] Number of elements (of type datatype) to be sent.
Param datatype:: [in] cuDSS datatype of elements to be sent.
Param dest:: [in] Rank of the receiving process (destination).
Param tag:: [in] Message tag.
Param comm:: [in] A pointer to the communicator.
Param stream:: [in] CUDA stream.
Returns:: [out] The error status (as int) of the invocation. Must return 0 on success and any nonzero integer otherwise.

int cudssRecvDevice( void *buffer, int count, cudaDataType_t datatype, int source, int tag, void *comm, cudaStream_t stream )#
int cudssRecvHost( void *buffer, int count, cudaDataType_t datatype, int source, int tag, void *comm, cudaStream_t stream )#: Function pointers to a routine which performs a blocking receive for a message.

Param buffer:: [out] Initial address of the receive buffer.
Param count:: [in] Number of elements (of type datatype) to be received.
Param datatype:: [in] cuDSS datatype of elements to be received.
Param source:: [in] Rank of the sending process (source).
Param tag:: [in] Message tag.
Param comm:: [in] A pointer to the communicator.
Param stream:: [in] CUDA stream.
Returns:: [out] The error status (as int) of the invocation. Must return 0 on success and any nonzero integer otherwise.

int cudssBcastDevice( void *buffer, int count, cudaDataType_t datatype, int root, void *comm, cudaStream_t stream )#
int cudssBcastHost( void *buffer, int count, cudaDataType_t datatype, int root, void *comm, cudaStream_t stream )#: Function pointers to a routine which performs a broadcast from the root process

to all other processes of the communicator.

Param buffer:: [in/out] Address of the buffer (send on root, receive on all other processes).
Param count:: [in] Number of elements (of type datatype) to be broadcast.
Param datatype:: [in] cuDSS datatype of elements to be broadcast.
Param root:: [in] Rank of the sending process (source).
Param comm:: [in] A pointer to the communicator.
Param stream:: [in] CUDA stream.
Returns:: [out] The error status (as int) of the invocation. Must return 0 on success and any nonzero integer otherwise.

int cudssReduceDevice( const void *sendbuf, void *recvbuf, int count, cudaDataType_t datatype, cudssOpType_t op, int root, void *comm, cudaStream_t stream )#
int cudssReduceHost( const void *sendbuf, void *recvbuf, int count, cudaDataType_t datatype, cudssOpType_t op, int root, void *comm, cudaStream_t stream )#: Function pointers to a routine which performs a reduction and stores the result on the root process.

Param sendbuf:: [in] Address of the send buffer.
Param recvbuf:: [out] Address of the receive buffer (valid only at root).
Param count:: [in] Number of elements (of type datatype) to be reduced.
Param datatype:: [in] cuDSS datatype of elements to be reduced.
Param op:: [in] Type of the reduction operation to be performed, see cudssOpType_t for supported values.
Param root:: [in] Rank of the root process (destination of the result).
Param comm:: [in] A pointer to the communicator.
Param stream:: [in] CUDA stream.
Returns:: [out] The error status (as int) of the invocation. Must return 0 on success and any nonzero integer otherwise.

int cudssAllreduceDevice( const void *sendbuf, void *recvbuf, int count, cudaDataType_t datatype, cudssOpType_t op, void *comm, cudaStream_t stream )#
int cudssAllreduceHost( const void *sendbuf, void *recvbuf, int count, cudaDataType_t datatype, cudssOpType_t op, void *comm, cudaStream_t stream )#: Function pointers to a routine which performs a reduction and distributes the result back to all processes.

Param sendbuf:: [in] Address of the send buffer.
Param recvbuf:: [out] Address of the receive buffer.
Param count:: [in] Number of elements (of type datatype) to be reduced.
Param datatype:: [in] cuDSS datatype of elements to be reduced.
Param op:: [in] Type of the reduction operation to be performed, see cudssOpType_t for supported values.
Param comm:: [in] A pointer to the communicator.
Param stream:: [in] CUDA stream.
Returns:: [out] The error status (as int) of the invocation. Must return 0 on success and any nonzero integer otherwise.

int cudssScattervDevice( const void *sendbuf, const int *sendcounts, const int *displs, cudaDataType_t sendtype, void *recvbuf, int recvcount, cudaDataType_t recvtype, int root, void *comm, cudaStream_t stream )#
int cudssScattervHost( const void *sendbuf, const int *sendcounts, const int *displs, cudaDataType_t sendtype, void *recvbuf, int recvcount, cudaDataType_t recvtype, int root, void *comm, cudaStream_t stream )#: Function pointers to a routine which scatters a buffer in parts to all processes in a communicator.

Param sendbuf:: [in] Address of the send buffer.
Param sendcounts:: [in] Non-negative integer array (of length communicator size) specifying the number of elements to send to each rank.
Param displs:: [in] An array of integers of length communicator size. Entry i specifies the displacement (relative to sendbuf) from which to take the outgoing data to process i.
Param sendtype:: [in] cuDSS datatype of elements to be sent.
Param recvbuf:: [out] Address of the receive buffer.
Param recvcount:: [in] Number of elements in receive buffer (non-negative integer).
Param recvtype:: [in] cuDSS datatype of elements to be received.
Param root:: [in] Rank of the sending process (source).
Param comm:: [in] A pointer to the communicator.
Param stream:: [in] CUDA stream.
Returns:: [out] The error status (as int) of the invocation. Must return 0 on success and any nonzero integer otherwise.

int cudssCommSplitDevice( const void *comm, int color, int key, void *newcomm )#
int cudssCommSplitHost( const void *comm, int color, int key, void *newcomm )#: Function pointers to a routine which creates new communicators based on colors and keys.

Param comm:: [in] A pointer to the communicator to be split.
Param color:: [in] Control of the subset assignment. Processes with the same color are grouped together.
Param key:: [in] Control of the rank assignment. Processes in the new communicator are ordered based on the keys.
Param newcomm:: [out] A pointer to the new communicator defined w.r.t to colors.
Returns:: [out] The error status (as int) of the invocation. Must return 0 on success and any nonzero integer otherwise.

int cudssCommFreeDevice(void *comm)#
int cudssCommFreeHost(void *comm)#: Function pointers to a routine which deallocates resources of a communicator.

Param comm:: [in] A pointer to the communicator to be freed.
Returns:: [out] The error status (as int) of the invocation. Must return 0 on success and any nonzero integer otherwise.

int cudssDistributedGetProperty( libraryPropertyType propertyType, int *value )#: A function pointer to a routine which returns a versioning property of the communication layer library.

Used by cuDSS to verify compatibility of a loaded communication layer.

Param propertyType:: [in] Requested property type (e.g. MAJOR_VERSION, MINOR_VERSION, PATCH_LEVEL).
Param value:: [out] Value of the requested property.
Returns:: [out] The error status (as int) of the invocation. Must return 0 on success and any nonzero integer otherwise.

`cudssOpType_t`#

enum cudssOpType_t#

The enumerator specifies reduction operation to be used when calling

communication layer APIs cudssReduceDevice(), cudssReduceHost(),

cudssAllreduceDevice() or cudssAllreduceHost().

enumerator CUDSS_SUM#: Reduced elements are added together.

enumerator CUDSS_MAX#: Maximum element is found among the reduced elements.

enumerator CUDSS_MIN#: Minimum element is found among the reduced elements.

Threading Layer Types#

`cudssThreadingInterface_t`#

struct cudssThreadingInterface_t#

This struct defines all threading primitives which need to be implemented

in (any) implementation of cuDSS threading layer, see for more details

threading layer and MT mode.

int cudssGetMaxThreads(void)#: A function pointer to a routine which returns the maximum number of threads on the CPU that can be used by cuDSS for parallel execution.

Returns:: [out] The maximum number of threads on the CPU that can be used by cuDSS for parallel execution.

void cudssParallelFor( int nthr_requested, int ntasks, void *ctx, cudss_thr_func_t f )#: A function pointer to a routine which opens a parallel for section with the requested number of threads and call cudss_thr_func_t (see cudss_threading_interface.h for details) f in the for loop with ntasks number of iterations.

Param nthr_requested:

[in] Requested number of threads for the parallel section.

Param ntasks:

[in] Number of tasks in the parallel for loop.

Param ctx:

[in] A pointer to the input data for cudss_thr_func_t f

which defines parallel units of work, also called tasks:

void cudss_thr_func_t(int task, void *ctx)#

Parameters:

task – [in] The task number (or iteration count of the parallel loop).
ctx – [in] A pointer to the input data.

int cudssThreadingGetProperty( libraryPropertyType propertyType, int *value )#: A function pointer to a routine which returns a versioning property of the threading layer library. Used by cuDSS to verify compatibility of a loaded threading layer.

Param propertyType:: [in] Requested property type (e.g. MAJOR_VERSION, MINOR_VERSION, PATCH_LEVEL).
Param value:: [out] Value of the requested property.
Returns:: [out] The error status (as int) of the invocation. Must return 0 on success and any nonzero integer otherwise.

cuDSS Data Types#

Opaque Data Structures#

cudssHandle_t#

cudssMatrix_t#

cudssData_t#

cudssConfig_t#

Non-opaque Data Structures#

cudssDeviceMemHandler_t#

cudssPinnedMemHandler_t#

Double-double values (CUDSS_R_64F_MP2)#

Logger Types#

cudssLoggerCallback_t#

Enumerators#

cudaDataType_t#

cudssStatus_t#

cudssConfigParam_t#

cudssDataParam_t#

cudssPhase_t#

cudssMatrixFormat_t#

cudssMatrixType_t#

cudssMatrixViewType_t#

cudssIndexBase_t#

cudssLayout_t#

cudssReorderingAlg_t#

cudssFactorizationAlg_t#

cudssSolveAlg_t#

cudssSolveMode_t#

cudssMemoryEstimateState_t#

cudssDowncastFactorsMode_t#

cudssMemoryOptimizationFlags_t#

cudssPivotEpsilonAlg_t#

cudssMatchingAlg_t#

cudssPivotType_t#

Communication Layer (Distributed Interface) Types#

cudssDistributedInterface_t#

cudssOpType_t#

Threading Layer Types#

cudssThreadingInterface_t#

`cudssHandle_t`#

`cudssMatrix_t`#

`cudssData_t`#

`cudssConfig_t`#

`cudssDeviceMemHandler_t`#

`cudssPinnedMemHandler_t`#

Double-double values (`CUDSS_R_64F_MP2`)#

`cudssLoggerCallback_t`#

`cudaDataType_t`#

`cudssStatus_t`#

`cudssConfigParam_t`#

`cudssDataParam_t`#

`cudssPhase_t`#

`cudssMatrixFormat_t`#

`cudssMatrixType_t`#

`cudssMatrixViewType_t`#

`cudssIndexBase_t`#

`cudssLayout_t`#

`cudssReorderingAlg_t`#

`cudssFactorizationAlg_t`#

`cudssSolveAlg_t`#

`cudssSolveMode_t`#

`cudssMemoryEstimateState_t`#

`cudssDowncastFactorsMode_t`#

`cudssMemoryOptimizationFlags_t`#

`cudssPivotEpsilonAlg_t`#

`cudssMatchingAlg_t`#

`cudssPivotType_t`#

`cudssDistributedInterface_t`#

`cudssOpType_t`#

`cudssThreadingInterface_t`#