MathDx Tile APIs#
The MathDx Tile APIs let Tile kernels call the existing SIMT implementations of cuFFTDx, cuBLASDx, cuSolverDx, and nvCOMPDx. This section describes how these calls work and the conventions that all MathDx Tile APIs use. For a description of the Tile programming model itself, refer to Writing Tile Kernels in the CUDA Programming Guide.
How it works#
The compiler resolves a MathDx call inside a __tile__ function or a @ct.kernel at compile time.
tileiras selects a SIMT implementation for the tile’s shape, precision, and CTA size.
It inlines that implementation into the surrounding Tile kernel.
Compiling both together produces a single kernel.
No launch boundary or round trip to global memory separates the tile code from the MathDx routine.
The simt preview mode provides this method of linking SIMT code into a Tile kernel.
Threads, warps and CTA size#
The compiler assigns the thread count for a Tile kernel. The CTA size is the number of worker threads, expressed in warps by
num_worker_warps. MathDx routines support multiple CTA sizes, and the
compiler selects a compatible SIMT implementation for each call. The user
can influence this choice with the worker-warp hint:
@ct.kernel(num_worker_warps=4) # 128 threads
def kernel(source: ct.Array, destination: ct.Array):
...
[[cutile::hint(0, num_worker_warps_per_cta = 4)]] // 128 threads
__tile_global__ void kernel(const float* input, float* output) { /* ... */ }
The worker-warp parameter is a hint. The compiler may ignore it. A different CTA size can affect performance and floating-point rounding while preserving the mathematical operation. Autotuning can vary this hint. See Autotuning.
The launch configuration is a separate matter. kernel<<<grid, 1>>> and
ct.launch(stream, grid, ...) select the grid, and the block dimension does
not determine the CTA size. For this reason the examples in this
documentation are launched with a block dimension of 1.
Large FFTs and CTA sizes#
Large FFTs can require more than one worker warp for compilation. Compilation fails if the compiler selects a CTA size with no compatible implementation. The table lists worker-warp counts common to all measured shapes for each size in the current FFT databases. Each worker warp contains 32 threads. Real transforms use a complex transform of half their length, so their sizes are twice the complex sizes.
Precision |
Complex size |
Real size |
Worker warps |
Architectures |
|---|---|---|---|---|
Single |
2048 |
4096 |
2, 4, 8, 16, 32 |
SM 80, 90, 100, 120 |
Single |
4096 |
8192 |
4, 8, 16, 32 |
SM 80, 90, 100, 120 |
Single |
8192 |
16384 |
8, 16, 32 |
SM 80, 90, 100, 120 |
Single |
16384 |
32768 |
16, 32 |
SM 80, 90, 100, 120 |
Single |
32768 |
65536 |
32 |
SM 80, 90, 100 |
Double |
1024, 2048 |
2048, 4096 |
2, 4, 8, 16, 32 |
SM 80, 90, 100, 120 |
Double |
4096 |
8192 |
4, 8, 16, 32 |
SM 80, 90, 100, 120 |
Double |
8192 |
16384 |
8, 16, 32 |
SM 80, 90, 100, 120 |
Double |
16384 |
32768 |
16, 32 |
SM 80, 90, 100 |
These entries cover all axes and measured batch shapes for the listed sizes. Some shapes also have a one-warp entry for double-precision complex size 1024 and real size 2048. They list measured configurations rather than every implementation the tilelib can generate. SM 120 has no measured entries above complex size 16384 in single precision or 8192 in double precision. The full tile shape and shared-memory budget also affect compilation. A listed count is not a guarantee for an arbitrary batch shape.
Use a suggestion with found == true from suggested_tile_config,
as Suggested FFT Configuration describes.
If no entry is found, select a compatible CTA size explicitly and check compilation.
The one-warp fallback may not support a large FFT.
The worker-warp setting remains a compiler hint.
Tile shapes#
Tile extents must be compile-time powers of two.
This restriction applies only to the tile. It does not apply to the source tensor.
Zero rows can pad a tile only when they do not change the result, as in reg_least_squares. Padding changes an FFT and makes a square matrix singular. See Tiles and Scalars in the cuTile Python data model.
FFT conventions#
Complex layout#
The MathDx Tile APIs provide no complex element type.
A real tile represents a complex tile with a trailing axis of extent 2 for the (real, imaginary) pair.
Use tile operations to form complex products:
In Python, use
ct.extractandct.cat.In C++, use
ct::extractandct::cat.
Do not form complex products by treating the pair’s components as independent real values.
rfft and irfft take a layout argument, which is currently always the
full-spectrum layout, and return the complete \([0, N)\) bin order in the
same interleaved representation.
Normalization#
Transforms are unnormalized. Neither direction applies a scale factor, so a forward transform followed by an inverse transform requires an explicit multiplication by \(1/N\).
Linear algebra conventions#
Batching#
The linear algebra routines are rank-polymorphic. An optional leading batch extent converts a single problem into a batch of independent problems, all solved by the same CTA.
Status codes#
Some routines can fail at runtime. For example, a matrix may not be positive definite or may have a singular minor.
These routines return an integer info value with the result.
For one problem, info is a scalar. For a batch, it is a tile of shape (B,).
\(info = 0\) indicates success.
\(info > 0\) typically indicates an error at factorization step info.
The caller must check info. The API does not check it automatically.
Option spellings#
The two languages spell the same options differently.
Option |
C++ ( |
Python ( |
|---|---|---|
Triangle |
|
|
Diagonal |
|
|
Side |
|
|
Transpose |
|
|
Pivoting |
|
|
Compression conventions#
Each call processes one one-dimensional chunk. The uncompressed data and the
compressed payload can each be a tile or a global memory operand: a pointer in
C++, or a contiguous ct.Array in Python. The chunk’s uncompressed element
count is fixed at compile time; the compressed byte count is returned at runtime.
Algorithm |
C++ element types |
Python element types |
|---|---|---|
LZ4 |
|
|
ANS |
|
|
Chunk-size bounds depend on the algorithm and element type. Unsupported sizes
are rejected at compile time in C++ or with ValueError in Python. Tile
extents must also be powers of two.
Sizing helpers take the uncompressed size in bytes: N * sizeof(T) in
C++, or the element count times the dtype’s item size in Python. In contrast,
C++’s N and Python’s num_elements and uncompressed_size count
elements. Decompression must use the original algorithm, element type, and
element count.
Tile outputs have a fixed capacity returned by comp_tile_size in C++ or
compressed_tile_size in Python; only the returned byte count is meaningful.
To copy just those bytes between a tile and global memory, use masked gather
and scatter as in Runtime-sized buffers.
Direct-memory outputs instead need the worst-case byte capacity returned by
max_comp_chunk_size in C++ or max_compressed_size in Python. Round the
per-chunk slot stride up to buffer_alignment() so every slot stays aligned.
Direct-memory calls handle the runtime byte count without explicit gather or
scatter. See nvCOMPDx in Tile and Compression cutile-python API for worked examples.
Every call also needs private global memory scratch sized for its direction. Concurrent calls require disjoint slices; sequential compression and decompression can share an allocation sized to the larger requirement.
Limitations#
__tile__functions cannot be called from__global__or__device__functions, and__device__functions cannot be called from__tile_global__or__tile__functions. The two programming models coexist within a translation unit and within an application, but not within a single call. This restriction originates in the Tile programming model rather than in MathDx, and may be lifted in a future CUDA release.A mixed application is therefore divided at kernel boundaries. The parts that map onto tiles are written as Tile kernels and the remainder as ordinary SIMT kernels, in the same translation unit if desired. The
filtered_backprojection_3dexample follows this structure, combining a__tile_global__filtering kernel, a__global__SIMT backprojection kernel, and a fused Tile-only variant of both. It is a suitable reference for porting an existing SIMT application incrementally.The MathDx Tile APIs are a preview. Coverage, performance, and the spelling of these APIs are subject to change.