cuTENSOR Data Types

cutensorDataType_t

enum cutensorDataType_t

This enum specifies the data precision. It is used when the data reference does not carry the type itself (e.g void *)

Values:

enumerator CUTENSOR_R_16F

real as a half

enumerator CUTENSOR_C_16F

complex as a pair of half numbers

enumerator CUTENSOR_R_16BF

real as a nv_bfloat16

enumerator CUTENSOR_C_16BF

complex as a pair of nv_bfloat16 numbers

enumerator CUTENSOR_R_32F

real as a float

enumerator CUTENSOR_C_32F

complex as a pair of float numbers

enumerator CUTENSOR_R_64F

real as a double

enumerator CUTENSOR_C_64F

complex as a pair of double numbers

enumerator CUTENSOR_R_4I

real as a signed 4-bit int

enumerator CUTENSOR_C_4I

complex as a pair of signed 4-bit int numbers

enumerator CUTENSOR_R_4U

real as a unsigned 4-bit int

enumerator CUTENSOR_C_4U

complex as a pair of unsigned 4-bit int numbers

enumerator CUTENSOR_R_8I

real as a signed 8-bit int

enumerator CUTENSOR_C_8I

complex as a pair of signed 8-bit int numbers

enumerator CUTENSOR_R_8U

real as a unsigned 8-bit int

enumerator CUTENSOR_C_8U

complex as a pair of unsigned 8-bit int numbers

enumerator CUTENSOR_R_16I

real as a signed 16-bit int

enumerator CUTENSOR_C_16I

complex as a pair of signed 16-bit int numbers

enumerator CUTENSOR_R_16U

real as a unsigned 16-bit int

enumerator CUTENSOR_C_16U

complex as a pair of unsigned 16-bit int numbers

enumerator CUTENSOR_R_32I

real as a signed 32-bit int

enumerator CUTENSOR_C_32I

complex as a pair of signed 32-bit int numbers

enumerator CUTENSOR_R_32U

real as a unsigned 32-bit int

enumerator CUTENSOR_C_32U

complex as a pair of unsigned 32-bit int numbers

enumerator CUTENSOR_R_64I

real as a signed 64-bit int

enumerator CUTENSOR_C_64I

complex as a pair of signed 64-bit int numbers

enumerator CUTENSOR_R_64U

real as a unsigned 64-bit int

enumerator CUTENSOR_C_64U

complex as a pair of unsigned 64-bit int numbers

cutensorComputeDescriptor_t

typedef struct cutensorComputeDescriptor *cutensorComputeDescriptor_t

Opaque structure representing a tensor descriptor.

CUTENSOR_EXTERN const cutensorComputeDescriptor_t CUTENSOR_COMPUTE_DESC_16F

floating-point: 5-bit exponent and 10-bit mantissa (aka half)

CUTENSOR_EXTERN const cutensorComputeDescriptor_t CUTENSOR_COMPUTE_DESC_16BF

floating-point: 8-bit exponent and 7-bit mantissa (aka bfloat)

CUTENSOR_EXTERN const cutensorComputeDescriptor_t CUTENSOR_COMPUTE_DESC_TF32

floating-point: 8-bit exponent and 10-bit mantissa (aka tensor-float-32)

CUTENSOR_EXTERN const cutensorComputeDescriptor_t CUTENSOR_COMPUTE_DESC_3XTF32

floating-point: More precise than TF32, but less precise than float

CUTENSOR_EXTERN const cutensorComputeDescriptor_t CUTENSOR_COMPUTE_DESC_32F

floating-point: 8-bit exponent and 23-bit mantissa (aka float)

CUTENSOR_EXTERN const cutensorComputeDescriptor_t CUTENSOR_COMPUTE_DESC_64F

floating-point: 11-bit exponent and 52-bit mantissa (aka double)

cutensorHandle_t

typedef struct cutensorHandle *cutensorHandle_t

Opaque structure holding cuTENSOR’s library context.

cutensorTensorDescriptor_t

typedef struct cutensorTensorDescriptor *cutensorTensorDescriptor_t

Opaque structure representing a tensor descriptor.

cutensorOperationDescriptor_t

typedef struct cutensorOperationDescriptor *cutensorOperationDescriptor_t

Opaque structure representing any type of problem descriptor (e.g., contraction, reduction, elementwise).

cutensorOperationDescriptorAttribute_t

enum cutensorOperationDescriptorAttribute_t

This enum lists all attributes of a cutensorOperationDescriptor_t that can be modified (see cutensorOperationDescriptorSetAttribute and cutensorOperationDescriptorGetAttribute).

Values:

enumerator CUTENSOR_OPERATION_DESCRIPTOR_TAG

int32_t: enables users to distinguish two identical problems w.r.t. the sw-managed plan-cache. (default value: 0)

enumerator CUTENSOR_OPERATION_DESCRIPTOR_SCALAR_TYPE

cutensorDataType_t: data type of the scaling factors

enumerator CUTENSOR_OPERATION_DESCRIPTOR_FLOPS

float: number of floating-point operations necessary to perform this operation (assuming all scalar are not equal to zero, unless otherwise specified)

enumerator CUTENSOR_OPERATION_DESCRIPTOR_MOVED_BYTES

float: minimal number of bytes transferred from/to global-memory (assuming all scalar are not equal to zero, unless otherwise specified)

enumerator CUTENSOR_OPERATION_DESCRIPTOR_PADDING_LEFT

uint32_t[] (of size descOut->numModes): Each entry i holds the number of padded values that should be padded to the left of the ith dimension

enumerator CUTENSOR_OPERATION_DESCRIPTOR_PADDING_RIGHT

uint32_t[] (of size descOut->numModes): Each entry i holds the number of padded values that should be padded to the right of the ith dimension

enumerator CUTENSOR_OPERATION_DESCRIPTOR_PADDING_VALUE

host-side pointer to element of the same type as the output tensor: Constant padding value

cutensorPlanPreference_t

typedef struct cutensorPlanPreference *cutensorPlanPreference_t

Opaque structure that narrows down the space of applicable algorithms/variants/kernels.

cutensorPlanPreferenceAttribute_t

enum cutensorPlanPreferenceAttribute_t

This enum lists all attributes of a cutensorPlanPreference_t object that can be modified.

Values:

enumerator CUTENSOR_PLAN_PREFERENCE_AUTOTUNE_MODE

cutensorAutotuneMode_t: Determines if recurrent executions of the plan (e.g., via cutensorContract, cutensorPermute) should autotune (i.e., try different kernels); see section “Plan Cache” for details.

enumerator CUTENSOR_PLAN_PREFERENCE_CACHE_MODE

cutensorCacheMode_t: Determines if the corresponding algorithm/kernel for this plan should be cached and it gives fine control over what is considered a cachehit.

enumerator CUTENSOR_PLAN_PREFERENCE_INCREMENTAL_COUNT

int32_t: Only applicable if CUTENSOR_PLAN_PREFERENCE_CACHE_MODE is set to CUTENSOR_AUTOTUNE_MODE_INCREMENTAL

enumerator CUTENSOR_PLAN_PREFERENCE_ALGO

cutensorAlgo_t: Fixes a certain cutensorAlgo_t

enumerator CUTENSOR_PLAN_PREFERENCE_KERNEL_RANK

int32_t: Fixes a kernel (a sub variant of an algo; e.g., kernel_rank==1 while algo == CUTENSOR_ALGO_TGETT would select the second-best GETT kernel/variant according to cuTENSOR’s performance model; kernel_rank==2 would select the third-best)

enumerator CUTENSOR_PLAN_PREFERENCE_JIT

cutensorJitMode_t: determines if just-in-time compilation is enabled or disabled (default: CUTENSOR_JIT_MODE_NONE)

cutensorPlan_t

typedef struct cutensorPlan *cutensorPlan_t

Opaque structure representing a plan (e.g, contraction, reduction, elementwise).

cutensorPlanAttribute_t

enum cutensorPlanAttribute_t

This enum lists all attributes of a cutensorPlan_t object that can be retrieved via cutensorPlanGetAttribute.

Values:

enumerator CUTENSOR_PLAN_REQUIRED_WORKSPACE

uint64_t: exact required workspace in bytes that is needed to execute the plan

cutensorAutotuneMode_t

enum cutensorAutotuneMode_t

This enum determines the mode w.r.t. cuTENSOR’s auto-tuning capability.

Values:

enumerator CUTENSOR_AUTOTUNE_MODE_NONE

Indicates no autotuning (default); in this case the cache will help to reduce the plan-creation overhead. In the case of a cachehit: the cached plan will be reused, otherwise the plancache will be neglected.

enumerator CUTENSOR_AUTOTUNE_MODE_INCREMENTAL

Indicates an incremental autotuning (i.e., each invocation of corresponding cutensorCreatePlan() will create a plan based on a different algorithm/kernel; the maximum number of kernels that will be tested is defined by the CUTENSOR_PLAN_PREFERENCE_INCREMENTAL_COUNT of cutensorPlanPreferenceAttribute_t). WARNING: If this autotuning mode is selected, then we cannot guarantee bit-wise identical results (since different algorithms could be executed).

cutensorJitMode_t

enum cutensorJitMode_t

This enum determines the mode w.r.t. cuTENSOR’s just-in-time compilation capability.

Values:

enumerator CUTENSOR_JIT_MODE_NONE

Indicates that no kernel will be just-in-time compiled.

enumerator CUTENSOR_JIT_MODE_DEFAULT

Indicates that the corresponding plan will try to compile a dedicated kernel for the given operation. Only supported for GPUs with compute capability >= 8.0 (Ampere or newer).

cutensorCacheMode_t

enum cutensorCacheMode_t

This enum defines what is considered a cache hit.

Values:

enumerator CUTENSOR_CACHE_MODE_NONE

Plan will not be cached.

enumerator CUTENSOR_CACHE_MODE_PEDANTIC

All parameters of the corresponding descriptor must be identical to the cached plan (default).

cutensorAlgo_t

enum cutensorAlgo_t

Allows users to specify the algorithm to be used for performing the desired tensor operation.

Values:

enumerator CUTENSOR_ALGO_DEFAULT_PATIENT

More time-consuming than CUTENSOR_DEFAULT, but typically provides a more accurate kernel selection.

enumerator CUTENSOR_ALGO_GETT

Choose the GETT algorithm (only applicable to contractions)

enumerator CUTENSOR_ALGO_TGETT

Transpose (A or B) + GETT (only applicable to contractions)

enumerator CUTENSOR_ALGO_TTGT

Transpose-Transpose-GEMM-Transpose (requires additional memory) (only applicable to contractions)

enumerator CUTENSOR_ALGO_DEFAULT

A performance model chooses the appropriate algorithm and kernel.

cutensorWorksizePreference_t

enum cutensorWorksizePreference_t

This enum gives users finer control over the suggested workspace.

This enum gives users finer control over the amount of workspace that is suggested by cutensorEstimateWorkspaceSize

Values:

enumerator CUTENSOR_WORKSPACE_MIN

Least memory requirement; at least one algorithm will be available.

enumerator CUTENSOR_WORKSPACE_DEFAULT

Aims to attain high performance while also reducing the workspace requirement.

enumerator CUTENSOR_WORKSPACE_MAX

Highest memory requirement; all algorithms will be available (choose this option if memory footprint is not a concern)

cutensorOperator_t

enum cutensorOperator_t

This enum captures all unary and binary element-wise operations supported by the cuTENSOR library.

Values:

enumerator CUTENSOR_OP_IDENTITY

Identity operator (i.e., elements are not changed)

enumerator CUTENSOR_OP_SQRT

Square root.

enumerator CUTENSOR_OP_RELU

Rectified linear unit.

enumerator CUTENSOR_OP_CONJ

Complex conjugate.

enumerator CUTENSOR_OP_RCP

Reciprocal.

enumerator CUTENSOR_OP_SIGMOID

y=1/(1+exp(-x))

enumerator CUTENSOR_OP_TANH

y=tanh(x)

enumerator CUTENSOR_OP_EXP

Exponentiation.

enumerator CUTENSOR_OP_LOG

Log (base e).

enumerator CUTENSOR_OP_ABS

Absolute value.

enumerator CUTENSOR_OP_NEG

Negation.

enumerator CUTENSOR_OP_SIN

Sine.

enumerator CUTENSOR_OP_COS

Cosine.

enumerator CUTENSOR_OP_TAN

Tangent.

enumerator CUTENSOR_OP_SINH

Hyperbolic sine.

enumerator CUTENSOR_OP_COSH

Hyperbolic cosine.

enumerator CUTENSOR_OP_ASIN

Inverse sine.

enumerator CUTENSOR_OP_ACOS

Inverse cosine.

enumerator CUTENSOR_OP_ATAN

Inverse tangent.

enumerator CUTENSOR_OP_ASINH

Inverse hyperbolic sine.

enumerator CUTENSOR_OP_ACOSH

Inverse hyperbolic cosine.

enumerator CUTENSOR_OP_ATANH

Inverse hyperbolic tangent.

enumerator CUTENSOR_OP_CEIL

Ceiling.

enumerator CUTENSOR_OP_FLOOR

Floor.

enumerator CUTENSOR_OP_MISH

Mish y=x*tanh(softplus(x)).

enumerator CUTENSOR_OP_SWISH

Swish y=x*sigmoid(x).

enumerator CUTENSOR_OP_SOFT_PLUS

Softplus y=log(exp(x)+1).

enumerator CUTENSOR_OP_SOFT_SIGN

Softsign y=x/(abs(x)+1).

enumerator CUTENSOR_OP_ADD

Addition of two elements.

enumerator CUTENSOR_OP_MUL

Multiplication of two elements.

enumerator CUTENSOR_OP_MAX

Maximum of two elements.

enumerator CUTENSOR_OP_MIN

Minimum of two elements.

enumerator CUTENSOR_OP_UNKNOWN

reserved for internal use only

cutensorStatus_t

enum cutensorStatus_t

cuTENSOR status type returns

The type is used for function status returns. All cuTENSOR library functions return their status, which can have the following values.

Values:

enumerator CUTENSOR_STATUS_SUCCESS

The operation completed successfully.

enumerator CUTENSOR_STATUS_NOT_INITIALIZED

The opaque data structure was not initialized.

enumerator CUTENSOR_STATUS_ALLOC_FAILED

Resource allocation failed inside the cuTENSOR library.

enumerator CUTENSOR_STATUS_INVALID_VALUE

An unsupported value or parameter was passed to the function (indicates an user error).

enumerator CUTENSOR_STATUS_ARCH_MISMATCH

Indicates that the device is either not ready, or the target architecture is not supported.

enumerator CUTENSOR_STATUS_MAPPING_ERROR

An access to GPU memory space failed, which is usually caused by a failure to bind a texture.

enumerator CUTENSOR_STATUS_EXECUTION_FAILED

The GPU program failed to execute. This is often caused by a launch failure of the kernel on the GPU, which can be caused by multiple reasons.

enumerator CUTENSOR_STATUS_INTERNAL_ERROR

An internal cuTENSOR error has occurred.

enumerator CUTENSOR_STATUS_NOT_SUPPORTED

The requested operation is not supported.

enumerator CUTENSOR_STATUS_LICENSE_ERROR

The functionality requested requires some license and an error was detected when trying to check the current licensing.

enumerator CUTENSOR_STATUS_CUBLAS_ERROR

A call to CUBLAS did not succeed.

enumerator CUTENSOR_STATUS_CUDA_ERROR

Some unknown CUDA error has occurred.

enumerator CUTENSOR_STATUS_INSUFFICIENT_WORKSPACE

The provided workspace was insufficient.

enumerator CUTENSOR_STATUS_INSUFFICIENT_DRIVER

Indicates that the driver version is insufficient.

enumerator CUTENSOR_STATUS_IO_ERROR

Indicates an error related to file I/O.

cudaDataType_t

enum cudaDataType_t

cudaDataType_t is an enumeration of the types supported by CUDA libraries. cuTENSOR supports real FP16, BF16, FP32 and FP64 as well as complex FP32 and FP64 input types.

Values:

enumerator CUDA_R_16F

16-bit real half precision floating-point type

enumerator CUDA_R_16BF

16-bit real BF16 floating-point type

enumerator CUDA_R_32F

32-bit real single precision floating-point type

enumerator CUDA_C_32F

32-bit complex single precision floating-point type (represented as pair of real and imaginary part)

enumerator CUDA_R_64F

64-bit real double precision floating-point type

enumerator CUDA_C_64F

64-bit complex double precision floating-point type (represented as pair of real and imaginary part)

cutensorLoggerCallback_t

typedef void (*cutensorLoggerCallback_t)(int32_t logLevel, const char *functionName, const char *message)

A function pointer type for logging.