cuSPARSELt Functions¶

Library Management Functions¶

`cusparseLtInit`¶

cusparseStatus_t
cusparseLtInit(cusparseLtHandle_t* handle)

The function initializes the cuSPARSELt library handle (cusparseLtHandle_t) which holds the cuSPARSELt library context. It allocates light hardware resources on the host, and must be called prior to making any other cuSPARSELt library calls. Calling any cusparseLt function which uses cusparseLtHandle_t without a previous call of cusparseLtInit() will return an error.

The cuSPARSELt library context is tied to the current CUDA device. To use the library on multiple devices, one cuSPARSELt handle should be created for each device.

Parameter	Memory	In/Out	Description
`handle`	Host	OUT	cuSPARSELt library handle

See cusparseStatus_t for the description of the return status.

`cusparseLtDestroy`¶

cusparseStatus_t
cusparseLtDestroy(const cusparseLtHandle_t* handle)

The function releases hardware resources used by the cuSPARSELt library. This function is the last call with a particular handle to the cuSPARSELt library.

Calling any cusparseLt function which uses cusparseLtHandle_t after cusparseLtDestroy() will return an error.

Parameter	Memory	In/Out	Description
`handle`	Host	IN	cuSPARSELt library handle

See cusparseStatus_t for the description of the return status.

`cusparseLtGetVersion`¶

cusparseStatus_t
cusparseLtGetVersion(const cusparseLtHandle_t* handle,
                     int*                      version)

This function returns the version number of the cuSPARSELt library.

Parameter	Memory	In/Out	Description
`handle`	Host	IN	cuSPARSELt library handle
`version`	Host	OUT	The version number of the library

See cusparseStatus_t for the description of the return status.

`cusparseLtGetProperty`¶

cusparseStatus_t
cusparseLtGetProperty(libraryPropertyType propertyType,
                      int*                value)

The function returns the value of the requested property. Refer to libraryPropertyType for supported types.

Parameter	Memory	In/Out	Description
`propertyType`	Host	IN	Requested property
`value`	Host	OUT	Value of the requested property

libraryPropertyType (defined in library_types.h):

Value	Meaning
`MAJOR_VERSION`	Enumerator to query the major version
`MINOR_VERSION`	Enumerator to query the minor version
`PATCH_LEVEL`	Number to identify the patch level

See cusparseStatus_t for the description of the return status.

Matrix Descriptor Functions¶

`cusparseLtDenseDescriptorInit`¶

cusparseStatus_t
cusparseLtDenseDescriptorInit(const cusparseLtHandle_t*  handle,
                              cusparseLtMatDescriptor_t* matDescr,
                              int64_t                    rows,
                              int64_t                    cols,
                              int64_t                    ld,
                              uint32_t                   alignment,
                              cudaDataType               valueType,
                              cusparseOrder_t            order)

The function initializes the descriptor of a dense matrix.

Parameter	Memory	In/Out	Description	Possible Values
`handle`	Host	IN	cuSPARSELt library handle
`matDescr`	Host	OUT	Dense matrix description
`rows`	Host	IN	Number of rows
`cols`	Host	IN	Number of columns
`ld`	Host	IN	Leading dimension	≥ rows if column-major, ≥ cols if row-major
`alignment`	Host	IN	Memory alignment in bytes	Multiple of 16
`valueType`	Host	IN	Data type of the matrix	`CUDA_R_32F`, `CUDA_R_16F`, `CUDA_R_16BF`, `CUDA_R_8I`, `CUDA_R_8F_E4M3`, `CUDA_R_8F_E5M2`
`order`	Host	IN	Memory layout	`CUSPARSE_ORDER_COL`, `CUSPARSE_ORDER_ROW`

Constrains:

rows, cols, and ld must be a multiple of

16 if valueType is CUDA_R_8I, CUDA_R_8F_E4M3 or CUDA_R_8F_E5M2

8 if valueType is CUDA_R_16F or CUDA_R_16BF

4 if valueType is CUDA_R_32F

The total size of the matrix cannot exceed:

$2^{32}-1$ elements for CUDA_R_8I, CUDA_R_8F_E4M3 or CUDA_R_8F_E5M2

$2^{31}-1$ elements for CUDA_R_16F or CUDA_R_16BF

$2^{30}-1$ elements for CUDA_R_32F

See cusparseStatus_t for the description of the return status.

`cusparseLtStructuredDescriptorInit`¶

cusparseStatus_t
cusparseLtStructuredDescriptorInit(const cusparseLtHandle_t*  handle,
                                   cusparseLtMatDescriptor_t* matDescr,
                                   int64_t                    rows,
                                   int64_t                    cols,
                                   int64_t                    ld,
                                   uint32_t                   alignment,
                                   cudaDataType               valueType,
                                   cusparseOrder_t            order,
                                   cusparseLtSparsity_t       sparsity)

The function initializes the descriptor of a structured matrix.

Parameter	Memory	In/Out	Description	Possible Values
`handle`	Host	IN	cuSPARSELt library handle
`matDescr`	Host	OUT	Sparse matrix description
`rows`	Host	IN	Number of rows
`cols`	Host	IN	Number of columns
`ld`	Host	IN	Leading dimension	≥ rows if column-major, ≥ cols if row-major
`alignment`	Host	IN	Memory alignment in bytes	Multiple of 16
`valueType`	Host	IN	Data type of the matrix	`CUDA_R_32F`, `CUDA_R_16F`, `CUDA_R_16BF`, `CUDA_R_8I`, `CUDA_R_8F_E4M3`, `CUDA_R_8F_E5M2`
`order`	Host	IN	Memory layout	`CUSPARSE_ORDER_COL`, `CUSPARSE_ORDER_ROW`
`sparsity`	Host	IN	Matrix sparsity ratio	`CUSPARSELT_SPARSITY_50_PERCENT`

Constrains:

rows, cols, and ld must be a multiple of

32 if valueType is CUDA_R_8I, CUDA_R_8F_E4M3 or CUDA_R_8F_E5M2

16 if valueType is CUDA_R_16F, or CUDA_R_16BF

8 if valueType is CUDA_R_32F

The total size of the matrix cannot exceed:

$2^{32}-1$ elements for CUDA_R_8I, CUDA_R_8F_E4M3 or CUDA_R_8F_E5M2

$2^{31}-1$ elements for CUDA_R_16F or CUDA_R_16BF

$2^{30}-1$ elements for CUDA_R_32F

See cusparseStatus_t for the description of the return status.

`cusparseLtMatDescriptorDestroy`¶

cusparseStatus_t
cusparseLtMatDescriptorDestroy(const cusparseLtMatDescriptor_t* matDescr)

The function releases the resources used by an instance of a matrix descriptor. After this call, the matrix descriptor, the matmul descriptor, and the plan can no longer be used.

Parameter	Memory	In/Out	Description
`matDescr`	Host	IN	Matrix descriptor

See cusparseStatus_t for the description of the return status.

`cusparseLtMatDescSetAttribute`¶

cusparseStatus_t
cusparseLtMatDescSetAttribute(const cusparseLtHandle_t*    handle,
                              cusparseLtMatDescriptor_t*   matmulDescr,
                              cusparseLtMatDescAttribute_t matAttribute,
                              const void*                  data,
                              size_t                       dataSize)

The function sets the value of the specified attribute belonging to matrix descriptor such as number of batches and their stride.

Parameter	Memory	In/Out	Description	Possible Values
`handle`	Host	IN	cuSPARSELt library handle
`matmulDescr`	Host	OUT	Matrix descriptor
`matAttribute`	Host	IN	Attribute to set	`CUSPARSELT_MAT_NUM_BATCHES`, `CUSPARSELT_MAT_BATCH_STRIDE`
`data`	Host	IN	Pointer to the value to which the specified attribute will be set
`dataSize`	Host	IN	Size in bytes of the attribute value used for verification

See cusparseStatus_t for the description of the return status.

`cusparseLtMatDescGetAttribute`¶

cusparseStatus_t
cusparseLtMatDescGetAttribute(const cusparseLtHandle_t*        handle,
                              const cusparseLtMatDescriptor_t* matmulDescr,
                              cusparseLtMatDescAttribute_t     matAttribute,
                              void*                            data,
                              size_t                           dataSize)

The function gets the value of the specified attribute belonging to matrix descriptor such as number of batches and their stride.

Parameter	Memory	In/Out	Description	Possible Values
`handle`	Host	IN	cuSPARSELt library handle
`matmulDescr`	Host	IN	Matrix descriptor
`matAttribute`	Host	IN	Attribute to retrieve	`CUSPARSELT_MAT_NUM_BATCHES`, `CUSPARSELT_MAT_BATCH_STRIDE`
`data`	Host	OUT	Memory address containing the attribute value retrieved by this function
`dataSize`	Host	IN	Size in bytes of the attribute value used for verification

See cusparseStatus_t for the description of the return status.

Matmul Descriptor Functions¶

`cusparseLtMatmulDescriptorInit`¶

cusparseStatus_t
cusparseLtMatmulDescriptorInit(const cusparseLtHandle_t*        handle,
                               cusparseLtMatmulDescriptor_t*    matmulDescr,
                               cusparseOperation_t              opA,
                               cusparseOperation_t              opB,
                               const cusparseLtMatDescriptor_t* matA,
                               const cusparseLtMatDescriptor_t* matB,
                               const cusparseLtMatDescriptor_t* matC,
                               const cusparseLtMatDescriptor_t* matD,
                               cusparseComputeType              computeType)

The function initializes the matrix multiplication descriptor.

Parameter	Memory	In/Out	Description	Possible Values
`handle`	Host	IN	cuSPARSELt library handle
`matmulDescr`	Host	OUT	Matrix multiplication descriptor
`opA`	Host	IN	Operation applied to the matrix `A`	`CUSPARSE_OPERATION_NON_TRANSPOSE`, `CUSPARSE_OPERATION_TRANSPOSE`
`opB`	Host	IN	Operation applied to the matrix `B`	`CUSPARSE_OPERATION_NON_TRANSPOSE`, `CUSPARSE_OPERATION_TRANSPOSE`
`matA`	Host	IN	Structured or dense matrix descriptor `A`
`matB`	Host	IN	Structured or dense matrix descriptor `B`
`matC`	Host	IN	Dense matrix descriptor `C`
`matD`	Host	IN	Dense matrix descriptor `D`
`computeType`	Host	IN	Compute precision	`CUSPARSE_COMPUTE_32I`, `CUSPARSE_COMPUTE_32F`, `CUSPARSE_COMPUTE_16F`

The structured matrix descriptor can used for matA or matB but not both.

Constrains:

See cusparseLtMatmul() for the supported data types.

CUDA_R_8I, CUDA_R_8F_E4M3 and CUDA_R_8F_E5M2 data type only supports (the opposite if B is structured):

opA/opB = TN if the matrix orders are orderA/orderB = Col/Col

opA/opB = NT if the matrix orders are orderA/orderB = Row/Row

opA/opB = NN if the matrix orders are orderA/orderB = Row/Col

opA/opB = TT if the matrix orders are orderA/orderB = Col/Row

C and D must have the same leading dimension and memory layout (see cusparseOrder_t for different memory layouts).

See cusparseStatus_t for the description of the return status.

`cusparseLtMatmulDescSetAttribute`¶

cusparseStatus_t
cusparseLtMatmulDescSetAttribute(const cusparseLtHandle_t*       handle,
                                 cusparseLtMatmulDescriptor_t*   matmulDescr,
                                 cusparseLtMatmulDescAttribute_t matmulAttribute,
                                 const void*                     data,
                                 size_t                          dataSize)

The function sets the value of the specified attribute belonging to matrix descriptor such as activation function and bias.

Parameter	Memory	In/Out	Description
`handle`	Host	IN	cuSPARSELt library handle
`matmulDescr`	Host	OUT	Matrix descriptor
`matmulAttribute`	Host	IN	Attribute to set	`CUSPARSELT_MATMUL_ACTIVATION_RELU`, `CUSPARSELT_MATMUL_ACTIVATION_RELU_UPPERBOUND`, `CUSPARSELT_MATMUL_ACTIVATION_RELU_THRESHOLD`, `CUSPARSELT_MATMUL_ACTIVATION_GELU`, `CUSPARSELT_MATMUL_ACTIVATION_GELU_SCALING`, `CUSPARSELT_MATMUL_ALPHA_VECTOR_SCALING`, `CUSPARSELT_MATMUL_BETA_VECTOR_SCALING`, `CUSPARSELT_MATMUL_BIAS_POINTER`, `CUSPARSELT_MATMUL_BIAS_STRIDE`
`data`	Host	IN	Pointer to the value to which the specified attribute will be set
`dataSize`	Host	IN	Size in bytes of the attribute value used for verification

See cusparseStatus_t for the description of the return status.

`cusparseLtMatmulDescGetAttribute`¶

cusparseStatus_t
cusparseLtMatmulDescGetAttribute(const cusparseLtHandle_t*           handle,
                                 const cusparseLtMatmulDescriptor_t* matmulDescr,
                                 cusparseLtMatmulDescAttribute_t     matmulAttribute,
                                 void*                               data,
                                 size_t                              dataSize)

The function gets the value of the specified attribute belonging to matrix descriptor such as activation function and bias.

Parameter	Memory	In/Out	Description
`handle`	Host	IN	cuSPARSELt library handle
`matmulDescr`	Host	IN	Matrix descriptor
`matmulAttribute`	Host	IN	Attribute to retrieve	`CUSPARSELT_MATMUL_ACTIVATION_RELU`, `CUSPARSELT_MATMUL_ACTIVATION_RELU_UPPERBOUND`, `CUSPARSELT_MATMUL_ACTIVATION_RELU_THRESHOLD`, `CUSPARSELT_MATMUL_ACTIVATION_GELU`, `CUSPARSELT_MATMUL_ACTIVATION_GELU_SCALING`, `CUSPARSELT_MATMUL_ALPHA_VECTOR_SCALING`, `CUSPARSELT_MATMUL_BETA_VECTOR_SCALING`, `CUSPARSELT_MATMUL_BIAS_POINTER`, `CUSPARSELT_MATMUL_BIAS_STRIDE`
`data`	Host	OUT	Memory address containing the attribute value retrieved by this function
`dataSize`	Host	IN	Size in bytes of the attribute value used for verification

See cusparseStatus_t for the description of the return status.

Matmul Algorithm Functions¶

`cusparseLtMatmulAlgSelectionInit`¶

cusparseStatus_t
cusparseLtMatmulAlgSelectionInit(const cusparseLtHandle_t*           handle,
                                 cusparseLtMatmulAlgSelection_t*     algSelection,
                                 const cusparseLtMatmulDescriptor_t* matmulDescr,
                                 cusparseLtMatmulAlg_t               alg)

The function initializes the algorithm selection descriptor.

Parameter	Memory	In/Out	Description	Possible Values
`handle`	Host	IN	cuSPARSELt library handle
`algSelection`	Host	OUT	Algorithm selection descriptor
`matmulDescr`	Host	IN	Matrix multiplication descriptor
`alg`	Host	IN	Algorithm mode	`CUSPARSELT_MATMUL_ALG_DEFAULT`

See cusparseStatus_t for the description of the return status.

`cusparseLtMatmulAlgSetAttribute`¶

cusparseStatus_t
cusparseLtMatmulAlgSetAttribute(const cusparseLtHandle_t*       handle,
                                cusparseLtMatmulAlgSelection_t* algSelection,
                                cusparseLtMatmulAlgAttribute_t  attribute,
                                const void*                     data,
                                size_t                          dataSize)

The function sets the value of the specified attribute belonging to algorithm selection descriptor.

Parameter	Memory	In/Out	Description	Possible Values
`handle`	Host	IN	cuSPARSELt library handle
`algSelection`	Host	OUT	Algorithm selection descriptor
`attribute`	Host	IN	The attribute to set	`CUSPARSELT_MATMUL_ALG_CONFIG_ID`, `CUSPARSELT_MATMUL_ALG_CONFIG_MAX_ID`, `CUSPARSELT_MATMUL_SEARCH_ITERATIONS`, `CUSPARSELT_MATMUL_SPLIT_K`, `CUSPARSELT_MATMUL_SPLIT_K_MODE`, `CUSPARSELT_MATMUL_SPLIT_K_BUFFERS`
`data`	Host	IN	Pointer to the value to which the specified attribute will be set
`dataSize`	Host	IN	Size in bytes of the attribute value used for verification

See cusparseStatus_t for the description of the return status.

`cusparseLtMatmulAlgGetAttribute`¶

cusparseStatus_t
cusparseLtMatmulAlgGetAttribute(const cusparseLtHandle_t*             handle,
                                const cusparseLtMatmulAlgSelection_t* algSelection,
                                cusparseLtMatmulAlgAttribute_t        attribute,
                                void*                                 data,
                                size_t                                dataSize)

The function returns the value of the queried attribute belonging to algorithm selection descriptor.

Parameter	Memory	In/Out	Description	Possible Values
`handle`	Host	IN	cuSPARSELt library handle
`algSelection`	Host	IN	Algorithm selection descriptor
`attribute`	Host	IN	The attribute that will be retrieved by this function	`CUSPARSELT_MATMUL_ALG_CONFIG_ID`, `CUSPARSELT_MATMUL_ALG_CONFIG_MAX_ID`, `CUSPARSELT_MATMUL_SEARCH_ITERATIONS`, `CUSPARSELT_MATMUL_SPLIT_K`, `CUSPARSELT_MATMUL_SPLIT_K_MODE`, `CUSPARSELT_MATMUL_SPLIT_K_BUFFERS`
`data`	Host	OUT	Memory address containing the attribute value retrieved by this function
`dataSize`	Host	IN	Size in bytes of the attribute value used for verification

See cusparseStatus_t for the description of the return status.

Matmul Functions¶

`cusparseLtMatmulGetWorkspace`¶

cusparseStatus_t
cusparseLtMatmulGetWorkspace(const cusparseLtHandle_t*     handle,
                             const cusparseLtMatmulPlan_t* plan,
                             size_t*                       workspaceSize)

The function determines the required workspace size associated to the selected algorithm.

Parameter	Memory	In/Out	Description
`handle`	Host	IN	cuSPARSELt library handle
`plan`	Host	IN	Matrix multiplication plan
`workspaceSize`	Host	OUT	Workspace size in bytes

See cusparseStatus_t for the description of the return status.

`cusparseLtMatmulPlanInit`¶

cusparseStatus_t
cusparseLtMatmulPlanInit(const cusparseLtHandle_t*             handle,
                         cusparseLtMatmulPlan_t*               plan,
                         const cusparseLtMatmulDescriptor_t*   matmulDescr,
                         const cusparseLtMatmulAlgSelection_t* algSelection)

Parameter	Memory	In/Out	Description
`handle`	Host	IN	cuSPARSELt library handle
`plan`	Host	OUT	Matrix multiplication plan
`matmulDescr`	Host	IN	Matrix multiplication descriptor
`algSelection`	Host	IN	Algorithm selection descriptor

See cusparseStatus_t for the description of the return status.

`cusparseLtMatmulPlanDestroy`¶

cusparseStatus_t
cusparseLtMatmulPlanDestroy(const cusparseLtMatmulPlan_t* plan)

The function releases the resources used by an instance of the matrix multiplication plan. This function is the last call with a specific plan instance.

Calling any cusparseLt function which uses cusparseLtMatmulPlan_t after cusparseLtMatmulPlanDestroy() will return an error.

Parameter	Memory	In/Out	Description
`plan`	Host	IN	Matrix multiplication plan

See cusparseStatus_t for the description of the return status.

`cusparseLtMatmul`¶

cusparseStatus_t
cusparseLtMatmul(const cusparseLtHandle_t*     handle,
                 const cusparseLtMatmulPlan_t* plan,
                 const void*                   alpha,
                 const void*                   d_A,
                 const void*                   d_B,
                 const void*                   beta,
                 const void*                   d_C,
                 void*                         d_D,
                 void*                         workspace,
                 cudaStream_t*                 streams,
                 int32_t                       numStreams)

The function computes the matrix multiplication of matrices A and B to produce the the output matrix D, according to the following operation:

$D = Activation(\alpha op(A) \cdot op(B) + \beta op(C) + bias) \cdot scale$

where A, B, and C are input matrices, and $\alpha$ and $\beta$ are input scalars or vectors of scalars (device-side pointers).

Note: The function currently only supports the case where D has the same shape of C

Parameter	Memory	In/Out	Description
`handle`	Host	IN	cuSPARSELt library handle
`plan`	Host	IN	Matrix multiplication plan
`alpha`	Host	IN	$\alpha$ scalar/vector of scalars used for multiplication (`float` data type)
`d_A`	Device	IN	Pointer to the structured or dense matrix `A`
`d_B`	Device	IN	Pointer to the structured or dense matrix `B`
`beta`	Host	IN	$\beta$ scalar/vector of scalars used for multiplication (`float` data type). It can have a `NULL` value only if `CUSPARSELT_MATMUL_ALPHA_VECTOR_SCALING` is set without `CUSPARSELT_MATMUL_BETA_VECTOR_SCALING`
`d_C`	Device	IN	Pointer to the dense matrix `C`
`d_D`	Device	OUT	Pointer to the dense matrix `D`
`workspace`	Device	IN	Pointer to workspace
`streams`	Host	IN	Pointer to CUDA stream array for the computation
`numStreams`	Host	IN	Number of CUDA streams in `streams`

Data types Supported:

Input A/B	Input C	Output D	Compute
`CUDA_R_32F`	`CUDA_R_32F`		`CUSPARSE_COMPUTE_32F`
`CUDA_R_16F`	`CUDA_R_16F`		`CUSPARSE_COMPUTE_32F`
`CUDA_R_16F`	`CUDA_R_16F`		`CUSPARSE_COMPUTE_16F`
`CUDA_R_16BF`	`CUDA_R_16BF`		`CUSPARSE_COMPUTE_32F`
`CUDA_R_8I`	`CUDA_R_8I`		`CUSPARSE_COMPUTE_32I`
	`CUDA_R_32I`
	`CUDA_R_16F`
	`CUDA_R_16BF`
`CUDA_R_8F_E4M3`	`CUDA_R_16F`	`CUDA_R_8F_E4M3`	`CUSPARSE_COMPUTE_32F`
	`CUDA_R_16BF`	`CUDA_R_8F_E4M3`
	`CUDA_R_16F`
	`CUDA_R_16BF`
	`CUDA_R_32F`
`CUDA_R_8F_E5M2`	`CUDA_R_16F`	`CUDA_R_8F_E5M2`	`CUSPARSE_COMPUTE_32F`
	`CUDA_R_16BF`	`CUDA_R_8F_E5M2`
	`CUDA_R_16F`
	`CUDA_R_16BF`
	`CUDA_R_32F`

The structured matrix A or B (before the compression) must respect the following constrains depending on the operation applied on it:

For op = CUSPARSE_NON_TRANSPOSE
- CUDA_R_16F, CUDA_R_16BF, CUDA_R_8I, CUDA_R_8F_E4M3, CUDA_R_8F_E5M2 each row must have at least two non-zero values every four elements
- CUDA_R_32F each row must have at least one non-zero value every two elements
For op = CUSPARSE_TRANSPOSE
- CUDA_R_16F, CUDA_R_16BF, CUDA_R_8I, CUDA_R_8F_E4M3, CUDA_R_8F_E5M2 each column must have at least two non-zero values every four elements
- CUDA_R_32F each column must have at least one non-zero value every two elements

int8, e4m3 and e5m2 kernels should run at high SM clocks for maximizing the performance.

The correctness of the pruning result (matrix A/B) can be check with the function cusparseLtSpMMAPruneCheck().

Constrains:

All pointers must be aligned to 16 bytes

Properties

The routine requires no extra storage

The routine supports asynchronous execution with respect to streams[0]

Provides deterministic (bit-wise) results for each run

cusparseLtMatmul supports the following optimizations:

CUDA graph capture

Hardware Memory Compression

See cusparseStatus_t for the description of the return status.

`cusparseLtMatmulSearch`¶

cusparseStatus_t
cusparseLtMatmulSearch(const cusparseLtHandle_t* handle,
                       cusparseLtMatmulPlan_t*   plan,
                       const void*               alpha,
                       const void*               d_A,
                       const void*               d_B,
                       const void*               beta,
                       const void*               d_C,
                       void*                     d_D,
                       void*                     workspace,
                       cudaStream_t*             streams,
                       int32_t                   numStreams)

The function evaluates all available algorithms for the matrix multiplication and automatically updates the plan by selecting the fastest one. The functionality is intended to be used for auto-tuning purposes when the same operation is repeated multiple times over different inputs.

The function behavior is the same of cusparseLtMatmul().

The function is NOT asynchronous with respect to streams[0] (blocking call)

The number of iterations for the evaluation can be set by using cusparseLtMatmulAlgSetAttribute() with CUSPARSELT_MATMUL_SEARCH_ITERATIONS.

The selected algorithm id can be retrieved by using cusparseLtMatmulAlgGetAttribute() with CUSPARSELT_MATMUL_ALG_CONFIG_ID.

The function also searches for optimal combination of Split-K parameters. The selected values can be retrieved by using cusparseLtMatmulAlgGetAttribute().

Helper Functions¶

`cusparseLtSpMMAPrune`¶

cusparseStatus_t
cusparseLtSpMMAPrune(const cusparseLtHandle_t*           handle,
                     const cusparseLtMatmulDescriptor_t* matmulDescr,
                     const void*                         d_in,
                     void*                               d_out,
                     cusparseLtPruneAlg_t                pruneAlg,
                     cudaStream_t                        stream)

The function prunes a dense matrix d_in according to the specified algorithm pruneAlg.

Parameter	Memory	In/Out	Description	Possible Values
`handle`	Host	IN	cuSPARSELt library handle
`matmulDescr`	Host	IN	Matrix multiplication descriptor
`d_in`	Device	IN	Pointer to the dense matrix
`d_out`	Device	OUT	Pointer to the pruned matrix
`pruneAlg`	Device	IN	Pruning algorithm	`CUSPARSELT_PRUNE_SPMMA_TILE`, `CUSPARSELT_PRUNE_SPMMA_STRIP`
`stream`	Host	IN	CUDA stream for the computation

Properties

The routine requires no extra storage

The routine supports asynchronous execution with respect to stream

Provides deterministic (bit-wise) results for each run

cusparseLtSpMMAPrune() supports the following optimizations:

CUDA graph capture

Hardware Memory Compression

See cusparseStatus_t for the description of the return status.

`cusparseLtSpMMAPrune2 [DEPRECATED]`¶

cusparseStatus_t
cusparseLtSpMMAPrune2(const cusparseLtHandle_t*        handle,
                      const cusparseLtMatDescriptor_t* sparseMatDescr,
                      int                              isSparseA,
                      cusparseOperation_t              op,
                      const void*                      d_in,
                      void*                            d_out,
                      cusparseLtPruneAlg_t             pruneAlg,
                      cudaStream_t                     stream);

The function prunes a dense matrix d_in according to the specified algorithm pruneAlg.

Parameter	Memory	In/Out	Description	Possible Values
`handle`	Host	IN	cuSPARSELt library handle
`sparseMatDescr`	Host	IN	Structured (sparse) matrix descriptor
`isSparseA`	Host	IN	Specify if the structured (sparse) matrix is in the first position (`matA` or `matB`)	`0` false, true otherwise
`op`	Host	IN	Operation that will be applied to the structured (sparse) matrix in the multiplication	`CUSPARSE_OPERATION_NON_TRANSPOSE`, `CUSPARSE_OPERATION_TRANSPOSE`
`d_in`	Device	IN	Pointer to the dense matrix
`d_out`	Device	OUT	Pointer to the pruned matrix
`pruneAlg`	Device	IN	Pruning algorithm	`CUSPARSELT_PRUNE_SPMMA_TILE`, `CUSPARSELT_PRUNE_SPMMA_STRIP`
`stream`	Host	IN	CUDA stream for the computation

If CUSPARSELT_PRUNE_SPMMA_TILE is used, isSparseA and op are not relevant.

The function has the same properties of cusparseLtSpMMAPrune()

`cusparseLtSpMMAPruneCheck`¶

cusparseStatus_t
cusparseLtSpMMAPruneCheck(const cusparseLtHandle_t*           handle,
                          const cusparseLtMatmulDescriptor_t* matmulDescr,
                          const void*                         d_in,
                          int*                                d_valid,
                          cudaStream_t                        stream)

The function checks the correctness of the pruning structure for a given matrix.

Parameter	Memory	In/Out	Description
`handle`	Host	IN	cuSPARSELt library handle
`matmulDescr`	Host	IN	Matrix multiplication descriptor
`d_in`	Device	IN	Pointer to the matrix to check
`d_valid`	Device	OUT	Validation results (`0` correct, `1` wrong)
`stream`	Host	IN	CUDA stream for the computation

See cusparseStatus_t for the description of the return status.

`cusparseLtSpMMAPruneCheck2 [DEPRECATED]`¶

cusparseStatus_t
cusparseLtSpMMAPruneCheck2(const cusparseLtHandle_t*        handle,
                           const cusparseLtMatDescriptor_t* sparseMatDescr,
                           int                              isSparseA,
                           cusparseOperation_t              op,
                           const void*                      d_in,
                           int*                             d_valid,
                           cudaStream_t                     stream)

The function checks the correctness of the pruning structure for a given matrix.

Parameter	Memory	In/Out	Description	Possible Values
`handle`	Host	IN	cuSPARSELt library handle
`sparseMatDescr`	Host	IN	Structured (sparse) matrix descriptor
`isSparseA`	Host	IN	Specify if the structured (sparse) matrix is in the first position (`matA` or `matB`)	`0`: false, `!= 0`: true
`op`	Host	IN	Operation that will be applied to the structured (sparse) matrix in the multiplication	`CUSPARSE_OPERATION_NON_TRANSPOSE`, `CUSPARSE_OPERATION_TRANSPOSE`
`d_in`	Device	IN	Pointer to the matrix to check
`d_valid`	Device	OUT	Validation results (`0` correct, `1` wrong)
`stream`	Host	IN	CUDA stream for the computation

The function has the same properties of cusparseLtSpMMAPruneCheck()

`cusparseLtSpMMACompressedSize`¶

cusparseStatus_t
cusparseLtSpMMACompressedSize(const cusparseLtHandle_t*     handle,
                              const cusparseLtMatmulPlan_t* plan,
                              size_t*                       compressedSize,
                              size_t*                       compressBufferSize)

The function provides the size of the compressed matrix to be allocated before calling cusparseLtSpMMACompress().

Parameter	Memory	In/Out	Description
`handle`	Host	IN	cuSPARSELt library handle
`plan`	Host	IN	Matrix plan descriptor
`compressedSize`	Host	OUT	Size in bytes for the compressed matrix
`compressBufferSize`	Host	OUT	Size in bytes for the buffer needed for the matrix compression

See cusparseStatus_t for the description of the return status.

`cusparseLtSpMMACompressedSize2 [DEPRECATED]`¶

cusparseStatus_t
cusparseLtSpMMACompressedSize2(const cusparseLtHandle_t*        handle,
                               const cusparseLtMatDescriptor_t* sparseMatDescr,
                               size_t*                          compressedSize,
                               size_t*                          compressBufferSize)

The function provides the size of the compressed matrix to be allocated before calling cusparseLtSpMMACompress2(). It has to be called after cusparseLtMatmulPlanInit.

Parameter	Memory	In/Out	Description
`handle`	Host	IN	cuSPARSELt library handle
`sparseMatDescr`	Host	IN	Structured (sparse) matrix descriptor
`compressedSize`	Host	OUT	Size in bytes of the compressed matrix
`compressBufferSize`	Host	OUT	Size in bytes for the buffer needed for the matrix compression

The function has the same properties of cusparseLtSpMMACompressedSize()

`cusparseLtSpMMACompress`¶

cusparseStatus_t
cusparseLtSpMMACompress(const cusparseLtHandle_t*     handle,
                        const cusparseLtMatmulPlan_t* plan,
                        const void*                   d_dense,
                        void*                         d_compressed,
                        void*                         d_compressed_buffer,
                        cudaStream_t                  stream)

The function compresses a dense matrix d_dense. The compressed matrix is intended to be used as the first/second operand A/B in the cusparseLtMatmul() function.

Parameter	Memory	In/Out	Description
`handle`	Host	IN	cuSPARSELt library handle
`plan`	Host	IN	Matrix multiplication plan
`d_dense`	Device	IN	Pointer to the dense matrix
`d_compressed`	Device	OUT	Pointer to the compressed matrix
`d_compressed_buffer`	Device	OUT	Pointer to temporary buffer for the compression
`stream`	Host	IN	CUDA stream for the computation

Properties

The routine supports asynchronous execution with respect to stream

Provides deterministic (bit-wise) results for each run

cusparseLtSpMMACompress() supports the following optimizations:

CUDA graph capture

Hardware Memory Compression

See cusparseStatus_t for the description of the return status.

`cusparseLtSpMMACompress2 [DEPRECATED]`¶

cusparseStatus_t
cusparseLtSpMMACompress2(const cusparseLtHandle_t*        handle,
                         const cusparseLtMatDescriptor_t* sparseMatDescr,
                         int                              isSparseA,
                         cusparseOperation_t              op,
                         const void*                      d_dense,
                         void*                            d_compressed,
                         void*                            d_compressed_buffer,
                         cudaStream_t                     stream)

Parameter	Memory	In/Out	Description	Possible Values
`handle`	Host	IN	cuSPARSELt library handle
`sparseMatDescr`	Host	IN	Structured (sparse) matrix descriptor
`isSparseA`	Host	IN	Specify if the structured (sparse) matrix is in the first position (`matA` or `matB`)	`0` false, true otherwise
`op`	Host	IN	Operation that will be applied to the structured (sparse) matrix in the multiplication	`CUSPARSE_OPERATION_NON_TRANSPOSE`, `CUSPARSE_OPERATION_TRANSPOSE`
`d_dense`	Device	IN	Pointer to the dense matrix
`d_compressed`	Device	OUT	Pointer to the compressed matrix
`d_compressed_buffer`	Device	OUT	Pointer to temporary buffer for the compression
`stream`	Host	IN	CUDA stream for the computation

The function has the same properties of cusparseLtSpMMACompress()

cuSPARSELt Functions¶

Library Management Functions¶

cusparseLtInit¶

cusparseLtDestroy¶

cusparseLtGetVersion¶

cusparseLtGetProperty¶

Matrix Descriptor Functions¶

cusparseLtDenseDescriptorInit¶

cusparseLtStructuredDescriptorInit¶

cusparseLtMatDescriptorDestroy¶

cusparseLtMatDescSetAttribute¶

cusparseLtMatDescGetAttribute¶

Matmul Descriptor Functions¶

cusparseLtMatmulDescriptorInit¶

cusparseLtMatmulDescSetAttribute¶

cusparseLtMatmulDescGetAttribute¶

Matmul Algorithm Functions¶

cusparseLtMatmulAlgSelectionInit¶

cusparseLtMatmulAlgSetAttribute¶

cusparseLtMatmulAlgGetAttribute¶

Matmul Functions¶

cusparseLtMatmulGetWorkspace¶

cusparseLtMatmulPlanInit¶

cusparseLtMatmulPlanDestroy¶

cusparseLtMatmul¶

cusparseLtMatmulSearch¶

Helper Functions¶

cusparseLtSpMMAPrune¶

cusparseLtSpMMAPrune2 [DEPRECATED]¶

cusparseLtSpMMAPruneCheck¶

cusparseLtSpMMAPruneCheck2 [DEPRECATED]¶

cusparseLtSpMMACompressedSize¶

cusparseLtSpMMACompressedSize2 [DEPRECATED]¶

cusparseLtSpMMACompress¶

cusparseLtSpMMACompress2 [DEPRECATED]¶

`cusparseLtInit`¶

`cusparseLtDestroy`¶

`cusparseLtGetVersion`¶

`cusparseLtGetProperty`¶

`cusparseLtDenseDescriptorInit`¶

`cusparseLtStructuredDescriptorInit`¶

`cusparseLtMatDescriptorDestroy`¶

`cusparseLtMatDescSetAttribute`¶

`cusparseLtMatDescGetAttribute`¶

`cusparseLtMatmulDescriptorInit`¶

`cusparseLtMatmulDescSetAttribute`¶

`cusparseLtMatmulDescGetAttribute`¶

`cusparseLtMatmulAlgSelectionInit`¶

`cusparseLtMatmulAlgSetAttribute`¶

`cusparseLtMatmulAlgGetAttribute`¶

`cusparseLtMatmulGetWorkspace`¶

`cusparseLtMatmulPlanInit`¶

`cusparseLtMatmulPlanDestroy`¶

`cusparseLtMatmul`¶

`cusparseLtMatmulSearch`¶

`cusparseLtSpMMAPrune`¶

`cusparseLtSpMMAPrune2 [DEPRECATED]`¶

`cusparseLtSpMMAPruneCheck`¶

`cusparseLtSpMMAPruneCheck2 [DEPRECATED]`¶

`cusparseLtSpMMACompressedSize`¶

`cusparseLtSpMMACompressedSize2 [DEPRECATED]`¶

`cusparseLtSpMMACompress`¶

`cusparseLtSpMMACompress2 [DEPRECATED]`¶