MathDx Tile APIs#

The MathDx Tile APIs let Tile kernels call the existing SIMT implementations of cuFFTDx, cuBLASDx, cuSolverDx, and nvCOMPDx. This section describes how these calls work and the conventions that all MathDx Tile APIs use. For a description of the Tile programming model itself, refer to Writing Tile Kernels in the CUDA Programming Guide.

How it works#

The compiler resolves a MathDx call inside a __tile__ function or a @ct.kernel at compile time. tileiras selects a SIMT implementation for the tile’s shape, precision, and CTA size. It inlines that implementation into the surrounding Tile kernel. Compiling both together produces a single kernel. No launch boundary or round trip to global memory separates the tile code from the MathDx routine. The simt preview mode provides this method of linking SIMT code into a Tile kernel.

Threads, warps and CTA size#

The compiler assigns the thread count for a Tile kernel. The CTA size is the number of worker threads, expressed in warps by num_worker_warps. MathDx routines support multiple CTA sizes, and the compiler selects a compatible SIMT implementation for each call. The user can influence this choice with the worker-warp hint:

@ct.kernel(num_worker_warps=4)   # 128 threads
def kernel(source: ct.Array, destination: ct.Array):
    ...
[[cutile::hint(0, num_worker_warps_per_cta = 4)]]   // 128 threads
__tile_global__ void kernel(const float* input, float* output) { /* ... */ }

The worker-warp parameter is a hint. The compiler may ignore it. A different CTA size can affect performance and floating-point rounding while preserving the mathematical operation. Autotuning can vary this hint. See Autotuning.

The launch configuration is a separate matter. kernel<<<grid, 1>>> and ct.launch(stream, grid, ...) select the grid, and the block dimension does not determine the CTA size. For this reason the examples in this documentation are launched with a block dimension of 1.

Large FFTs and CTA sizes#

Large FFTs can require more than one worker warp for compilation. Compilation fails if the compiler selects a CTA size with no compatible implementation. The table lists worker-warp counts common to all measured shapes for each size in the current FFT databases. Each worker warp contains 32 threads. Real transforms use a complex transform of half their length, so their sizes are twice the complex sizes.

Measured CTA choices for large FFTs#

Precision

Complex size

Real size

Worker warps

Architectures

Single

2048

4096

2, 4, 8, 16, 32

SM 80, 90, 100, 120

Single

4096

8192

4, 8, 16, 32

SM 80, 90, 100, 120

Single

8192

16384

8, 16, 32

SM 80, 90, 100, 120

Single

16384

32768

16, 32

SM 80, 90, 100, 120

Single

32768

65536

32

SM 80, 90, 100

Double

1024, 2048

2048, 4096

2, 4, 8, 16, 32

SM 80, 90, 100, 120

Double

4096

8192

4, 8, 16, 32

SM 80, 90, 100, 120

Double

8192

16384

8, 16, 32

SM 80, 90, 100, 120

Double

16384

32768

16, 32

SM 80, 90, 100

These entries cover all axes and measured batch shapes for the listed sizes. Some shapes also have a one-warp entry for double-precision complex size 1024 and real size 2048. They list measured configurations rather than every implementation the tilelib can generate. SM 120 has no measured entries above complex size 16384 in single precision or 8192 in double precision. The full tile shape and shared-memory budget also affect compilation. A listed count is not a guarantee for an arbitrary batch shape.

Use a suggestion with found == true from suggested_tile_config, as Suggested FFT Configuration describes. If no entry is found, select a compatible CTA size explicitly and check compilation. The one-warp fallback may not support a large FFT. The worker-warp setting remains a compiler hint.

Tile shapes#

Tile extents must be compile-time powers of two. This restriction applies only to the tile. It does not apply to the source tensor. Zero rows can pad a tile only when they do not change the result, as in reg_least_squares. Padding changes an FFT and makes a square matrix singular. See Tiles and Scalars in the cuTile Python data model.

FFT conventions#

Complex layout#

The MathDx Tile APIs provide no complex element type. A real tile represents a complex tile with a trailing axis of extent 2 for the (real, imaginary) pair. Use tile operations to form complex products:

  • In Python, use ct.extract and ct.cat.

  • In C++, use ct::extract and ct::cat.

Do not form complex products by treating the pair’s components as independent real values.

rfft and irfft take a layout argument, which is currently always the full-spectrum layout, and return the complete \([0, N)\) bin order in the same interleaved representation.

Normalization#

Transforms are unnormalized. Neither direction applies a scale factor, so a forward transform followed by an inverse transform requires an explicit multiplication by \(1/N\).

Linear algebra conventions#

Batching#

The linear algebra routines are rank-polymorphic. An optional leading batch extent converts a single problem into a batch of independent problems, all solved by the same CTA.

Status codes#

Some routines can fail at runtime. For example, a matrix may not be positive definite or may have a singular minor. These routines return an integer info value with the result. For one problem, info is a scalar. For a batch, it is a tile of shape (B,). \(info = 0\) indicates success. \(info > 0\) typically indicates an error at factorization step info. The caller must check info. The API does not check it automatically.

Option spellings#

The two languages spell the same options differently.

Option

C++ (cusolverdx::tiles, cublasdx::tiles)

Python (nvmath.tile.linalg)

Triangle

lower, upper

FillMode.LOWER, FillMode.UPPER

Diagonal

unit, non_unit

DiagMode.UNIT, DiagMode.NON_UNIT

Side

left, right

SideMode.LEFT, SideMode.RIGHT

Transpose

no_trans, trans

TransposeMode

Pivoting

partial_pivot, no_pivot

PivotMode

Compression conventions#

Each call processes one one-dimensional chunk. The uncompressed data and the compressed payload can each be a tile or a global memory operand: a pointer in C++, or a contiguous ct.Array in Python. The chunk’s uncompressed element count is fixed at compile time; the compressed byte count is returned at runtime.

Supported compression element types#

Algorithm

C++ element types

Python element types

LZ4

std::uint8_t, std::uint16_t, std::uint32_t

ct.uint8, ct.uint16, ct.uint32

ANS

std::uint8_t, __half

ct.uint8, ct.float16

Chunk-size bounds depend on the algorithm and element type. Unsupported sizes are rejected at compile time in C++ or with ValueError in Python. Tile extents must also be powers of two.

Sizing helpers take the uncompressed size in bytes: N * sizeof(T) in C++, or the element count times the dtype’s item size in Python. In contrast, C++’s N and Python’s num_elements and uncompressed_size count elements. Decompression must use the original algorithm, element type, and element count.

Tile outputs have a fixed capacity returned by comp_tile_size in C++ or compressed_tile_size in Python; only the returned byte count is meaningful. To copy just those bytes between a tile and global memory, use masked gather and scatter as in Runtime-sized buffers.

Direct-memory outputs instead need the worst-case byte capacity returned by max_comp_chunk_size in C++ or max_compressed_size in Python. Round the per-chunk slot stride up to buffer_alignment() so every slot stays aligned. Direct-memory calls handle the runtime byte count without explicit gather or scatter. See nvCOMPDx in Tile and Compression cutile-python API for worked examples.

Every call also needs private global memory scratch sized for its direction. Concurrent calls require disjoint slices; sequential compression and decompression can share an allocation sized to the larger requirement.

Limitations#

  • __tile__ functions cannot be called from __global__ or __device__ functions, and __device__ functions cannot be called from __tile_global__ or __tile__ functions. The two programming models coexist within a translation unit and within an application, but not within a single call. This restriction originates in the Tile programming model rather than in MathDx, and may be lifted in a future CUDA release.

  • A mixed application is therefore divided at kernel boundaries. The parts that map onto tiles are written as Tile kernels and the remainder as ordinary SIMT kernels, in the same translation unit if desired. The filtered_backprojection_3d example follows this structure, combining a __tile_global__ filtering kernel, a __global__ SIMT backprojection kernel, and a fused Tile-only variant of both. It is a suitable reference for porting an existing SIMT application incrementally.

  • The MathDx Tile APIs are a preview. Coverage, performance, and the spelling of these APIs are subject to change.