cuTENSOR: A High-Performance CUDA Library For Tensor Primitives

Welcome to the cuTENSOR library documentation.

cuTENSOR is a high-performance CUDA library for tensor primitives.

Download: https://developer.nvidia.com/cutensor/downloads

Key Features

• Extensive mixed-precision support:

  • FP64 inputs with FP32 compute.

  • FP32 inputs with FP16, BF16, TF32, or 3XTF32 compute.

  • Complex-times-real operations.

  • Conjugate (without transpose) support.

• Just-in-time (JIT) compilation

• Support for up to 64-dimensional tensors.

• Arbitrary data layouts.

• Main computational routines:

  • Direct (i.e., transpose-free) tensor contractions.

  • Tensor reductions (including partial reductions).

  • Element-wise tensor operations:

    • Support for various activation functions.

    • Arbitrary tensor permutations.

    • Conversion between different data types.

    • Support for padding output tensors.

The documentation consists of three main components:

• A User Guide that introduces important basics of cuTENSOR including details on notation and accuracy.

• A Getting Started guide that steps through a simple tensor contraction example.

• An API Reference that provides a comprehensive overview of all library routines, constants, and data types.

Support

Operating System	CPU Architectures
RHEL 8, openSUSE 15, SLES 15, Ubuntu 22.04/20.04/18.04	x86_64, SBSA
RHEL 8, Ubuntu 22.04/20.04/18.04	OpenPOWER
Windows 10	x86_64

Supported CUDA Toolkits: 11.0, 11.8, 12.x Supported SM Architectures : SM 6.0, SM 7.0, SM7.5, SM 8.0, SM 8.9, SM 9.0 Deprecated OSs : Ubuntu 16.04, RHEL 7

Prerequisites

• Dependencies : cudart, cutensor.h headers

Contents

• Release Notes
  • cuTENSOR v2.0.2
  • cuTENSOR v2.0.0
  • cuTENSOR v1.7.0
  • cuTENSOR v1.6.2
  • cuTENSOR v1.6.1
  • cuTENSOR v1.6.0
  • cuTENSOR v1.5.0
  • cuTENSOR v1.4.0
  • cuTENSOR v1.3.3
  • cuTENSOR v1.3.2
  • cuTENSOR v1.3.1
  • cuTENSOR v1.3.0
  • cuTENSOR v1.2.2
  • cuTENSOR v1.2.1
  • cuTENSOR v1.2.0
  • cuTENSOR v1.1.0
  • cuTENSOR v1.0.1
  • cuTENSOR v1.0.0
• User Guide
  • Nomenclature
  • Einstein Notation
  • Performance Guidelines
  • Software-managed Plan Cache
  • Accuracy Guarantees
  • Scalar Types
  • Supported Unary Operators
  • Supported GPUs
  • CUDA Graph Support
  • Logging
  • Environment Variables
• Getting Started
  • Installation and Compilation
  • Headers and Data Types
  • Define Tensor Sizes
  • Initialize Tensor Data
  • Create Tensor Descriptors
  • Create Contraction Descriptor
  • Determine Algorithm and Workspace
  • Plan and reduce workspace
  • Execute
• Transition to cuTENSOR 2.x
  • Overview
  • Differences at a glance
  • Example 1: Migrating a contraction from 1.x to 2.x
  • Example 2: Migrating a reduction operation from 1.x to 2.x
  • Example 3: Migrating a permutation/elementwise operation from 1.x to 2.x
• Just In Time (JIT) Compilation
  • Introductory Example
  • Reading and writing the kernel cache to disk
• Plan Cache
  • Incremental Autotuning
  • Introductory Example
  • Advanced Example
• Multi-GPU support - cuTENSORMg
  • Performance Guidelines
  • Accuracy Guarantees
  • Scalar Types
  • CUDA Graph Support
  • cuTENSORMg Logging
• API Reference
  • cuTENSOR Data Types
  • cuTENSOR Functions
• API Reference - cuTENSORMg
  • General
  • Copy Operations
  • Contraction Operations
• Software License Agreement
• Third Party License Agreements
  • HPTT

Indices And Tables

• Index

• Search Page