Getting Started#

This section shows how to install MathDx Tile packages and build three kernels:

  • A 256-point FFT.

  • A Cholesky factorization.

  • A compression round trip.

Each kernel has cuTile Python and CUDA Tile C++ versions based on the quick-start sources included with this documentation. Refer to the Examples section for the complete example suites.

Requirements#

Component

Requirement

GPU

Compute capability 8.0 (NVIDIA Ampere) or newer

Driver

r580 or newer

Operating system

Linux or Windows

CPU architecture

x86_64 or aarch64

Toolchain (C++)

A CUDA 13.5 Tile toolchain, providing both nvcc and tileiras

C++ standard

C++20

Python

A Python version supported by cuTile Python

Warning

MathDx Tile is compatible with CUDA 13.5 only. To check the version, run tileiras --version for an installed toolkit, or python -m pip show nvidia-cuda-tileiras for the cuda-toolkit wheels. If both are present, cuTile uses the wheel’s tileiras.

Both the C++ and the Python paths require a Tile-capable toolchain. A standard CUDA compiler cannot build these kernels.

Installation#

MathDx Tile uses the libmathdx version number: the mathdx_tile archive, the nvidia-libmathdx-cu13 wheel, and the libmathdx inside them share one version. Use it for <version> in the C++ commands below. See Release notes.

On the Python side, install nvmath-python with its cu13-dx extra. The MathDx Tile APIs need all of these packages, and the extra brings them in:

  • nvidia-libmathdx-cu13: the Tile tilelibs and the nvmath.tile modules.

  • cuda-tile: cuTile Python.

  • cuda-tile-preview: the experimental preview APIs of cuTile Python, including the foreign call API that MathDx Tile uses. It is released together with cuda-tile.

MathDx Tile 0.5.0 pairs with nvmath-python 1.0.1 or newer. See the compatibility table in the release notes.

python -m venv .venv
source .venv/bin/activate
python -m pip install "nvmath-python[cu13-dx]>=1.0.1"

The nvidia-libmathdx-cu13 wheel includes the MathDx Tile libraries, and nvmath.tile finds them without any path settings. The cuda-toolkit wheels supply NVRTC and tileiras, and pip installs CUDA 13.5, which MathDx Tile requires.

Use a system CUDA toolkit

To use an installed CUDA 13.5 toolkit instead of the cuda-toolkit wheels, point the environment at it:

export CUDA_HOME=/usr/local/cuda-13.5
export PATH="${CUDA_HOME}/bin:${PATH}"
export LD_LIBRARY_PATH="${CUDA_HOME}/lib64:${LD_LIBRARY_PATH}"

Then install nvmath-python with its sysctk13-dx extra, which installs the libmathdx wheel and the cuTile packages but not cuda-toolkit:

python -m pip install "nvmath-python[sysctk13-dx]>=1.0.1"

Download the mathdx_tile archive from the MathDx Tile download page, then unpack it:

mkdir mathdx_tile
tar xzf mathdx_tile-linux-x86_64-<version>.tar.gz -C mathdx_tile --strip-components=1
export MATHDX_TILE="${PWD}/mathdx_tile"

${MATHDX_TILE}/include holds the Tile headers, ${MATHDX_TILE}/lib holds libmathdx and the .tilelib libraries, and ${MATHDX_TILE}/examples holds the C++ Examples. Windows ships the same tree as a .zip.

1. FFT#

The first kernel computes a 256-point forward FFT of an all-ones signal. Its DFT is \(N\) at bin 0 and zero at every other bin, so the result is easy to check. The Python source is a complete program. The C++ source contains the kernel and its launch wrapper. Call the wrapper from the application’s main.

import cuda.tile as ct
import cupy as cp
import nvmath.tile.fft as mfft


@ct.kernel(num_worker_warps=4)
def fft_kernel(source: ct.Array, destination: ct.Array):
    signal = ct.load(source, (0, 0), shape=(256, 2))
    ct.store(destination, (0, 0), mfft.fft(signal, axis=0))


def main():
    source = cp.zeros((256, 2), dtype=cp.float32)
    source[:, 0] = 1
    destination = cp.zeros_like(source)
    ct.launch(cp.cuda.get_current_stream(), (1, 1, 1), fft_kernel, (source, destination))
    cp.cuda.runtime.deviceSynchronize()
    assert cp.allclose(destination[0], cp.asarray([256, 0], dtype=cp.float32))
    assert cp.allclose(destination[1:], 0, atol=1e-3)


if __name__ == "__main__":
    main()
#include "cufftdx_tile.hpp"

namespace ct = cuda::tiles;

__tile_global__ void fft_kernel(float* input, float* output) {
    using signal_shape = ct::shape<256, 2>;
    auto source = ct::partition_view { ct::tensor_span { input, signal_shape {} }, signal_shape {} };
    auto destination = ct::partition_view { ct::tensor_span { output, signal_shape {} }, signal_shape {} };

    auto spectrum = cufftdx::tiles::fft(source.load(0, 0), cufftdx::tiles::axis<0>);
    destination.store(spectrum, 0, 0);
}

void launch_fft(float* input, float* output) {
    fft_kernel<<<1, 1>>>(input, output);
}

Four properties of this kernel merit further explanation.

  1. The tile shape is (256, 2) rather than (256,). The MathDx Tile APIs provide no complex element type, so a complex signal is represented as a real tile with a trailing interleaved (real, imaginary) axis of extent 2. A rank-one transform of \(N\) complex values is therefore a rank-two tile, transformed along axis=0. Refer to FFT conventions.

  2. The transform axis is specified rather than inferred. axis=0 in Python and cufftdx::tiles::axis<0> in C++ select the logical transform dimension, excluding the complex-pair axis. Negative values are counted from the last logical dimension.

  3. The launch block dimension is 1, because the CTA size is not determined by the launch configuration. The compiler selects it, optionally guided by the num_worker_warps hint in Python or [[cutile::hint(0, num_worker_warps_per_cta = ...)]] in C++. The grid dimension is unaffected. Refer to Threads, warps and CTA size.

  4. The result is unnormalized. fft and ifft apply no scaling in either direction, so a forward transform followed by an inverse transform requires an explicit multiplication by \(1/N\).

2. Factorization#

The linear algebra routines return a status value alongside the result and accept a batch dimension. The Python source below performs a triangular solve and a factorization within a single kernel, which is a common reason for using the Tile APIs.

import cuda.tile as ct
import cupy as cp
import nvmath.tile.linalg as mlinalg


@ct.kernel(num_worker_warps=4)
def linalg_kernel(
    triangular: ct.Array,
    rhs: ct.Array,
    positive_definite: ct.Array,
    solution: ct.Array,
    factor: ct.Array,
    status: ct.Array,
):
    a = ct.load(triangular, (0, 0), shape=(16, 16))
    b = ct.load(rhs, (0, 0), shape=(16, 16))
    x = mlinalg.triangular_solve(
        a, b, mlinalg.FillMode.LOWER, mlinalg.DiagMode.NON_UNIT, mlinalg.SideMode.LEFT
    )
    ct.store(solution, (0, 0), x)

    matrix = ct.load(positive_definite, (0, 0), shape=(16, 16))
    cholesky, info = mlinalg.cholesky_factor(matrix, mlinalg.FillMode.LOWER)
    ct.store(factor, (0, 0), cholesky)
    ct.store(status, (0,), info)


def main():
    triangular = cp.tril(cp.ones((16, 16), dtype=cp.float32)) + cp.eye(16, dtype=cp.float32)
    rhs = cp.eye(16, dtype=cp.float32)
    positive_definite = triangular @ triangular.T
    solution = cp.zeros_like(rhs)
    factor = cp.zeros_like(positive_definite)
    status = cp.full((1,), -1, dtype=cp.int32)
    ct.launch(
        cp.cuda.get_current_stream(),
        (1, 1, 1),
        linalg_kernel,
        (triangular, rhs, positive_definite, solution, factor, status),
    )
    cp.cuda.runtime.deviceSynchronize()
    assert int(status[0]) == 0
    assert cp.allclose(triangular @ solution, rhs, atol=1e-4)
    lower = cp.tril(factor)
    assert cp.allclose(lower @ lower.T, positive_definite, atol=1e-4)


if __name__ == "__main__":
    main()
#include "cusolverdx_tile.hpp"

namespace ct = cuda::tiles;

__tile_global__ void cholesky_kernel(float* matrix, float* factor, int* status) {
    using matrix_shape = ct::shape<16, 16>;
    auto source = ct::partition_view { ct::tensor_span { matrix, matrix_shape {} }, matrix_shape {} };
    auto destination = ct::partition_view { ct::tensor_span { factor, matrix_shape {} }, matrix_shape {} };

    auto [result, info] = cusolverdx::tiles::cholesky_factor(source.load(0, 0), cusolverdx::tiles::lower);
    destination.store(result, 0, 0);
    *status = static_cast<int>(info);
}

void launch_cholesky(float* matrix, float* factor, int* status) {
    cholesky_kernel<<<1, 1>>>(matrix, factor, status);
}

info is 0 on success, while a positive value identifies the step of the factorization at which the failure occurred. It is not checked automatically, and a kernel that ignores it will store an invalid factor. Refer to Status codes.

Every linear algebra routine also accepts a batch dimension. Loading a (B, M, M) tile in place of an (M, M) tile factors B independent matrices within the same CTA, and returns info with shape (B,). This arrangement allows small factorizations to utilize a GPU that would otherwise remain largely idle:

@ct.kernel(num_worker_warps=4)
def cholesky_batched_kernel(a_data: ct.Array, l_data: ct.Array, status: ct.Array):
    a = ct.load(a_data, (0, 0, 0), shape=(B, M, M))
    factors, info = mlinalg.cholesky_factor(a, mlinalg.FillMode.LOWER)
    ct.store(l_data, (0, 0, 0), factors)
    ct.store(status, (0,), info)
using batch_shape = ct::shape<B, M, M>;
auto source = ct::partition_view { ct::tensor_span { matrix, batch_shape {} }, batch_shape {} };

auto [result, info] = cusolverdx::tiles::cholesky_factor(source.load(0, 0, 0),
                                                         cusolverdx::tiles::lower);

The number of problems that fit within one CTA is limited by shared memory, so the batch extent must be balanced against the matrix size. Refer to Achieving High Performance.

3. Compression#

The number of bytes to which a chunk compresses depends on the data, so compress returns a count alongside the compressed tile, and decompress requires that count in order to reverse the operation.

import cuda.tile as ct
import cupy as cp
import nvmath.tile.compress as mcomp


@ct.kernel
def compression_round_trip_kernel(source: ct.Array, restored: ct.Array, scratch: ct.Array):
    data = ct.load(source, (0,), shape=(4096,))
    compressed, count = mcomp.compress(data, "ans", scratch=scratch)
    result = mcomp.decompress(compressed, count, "ans", 4096, ct.uint8, scratch)
    ct.store(restored, (0,), result)


def main():
    source = (cp.arange(4096, dtype=cp.uint32) % 23).astype(cp.uint8)
    restored = cp.zeros_like(source)
    # The two calls run one after the other, so they can share one scratch.
    scratch_bytes = max(
        mcomp.compress_scratch_size(4096, "ans", ct.uint8), mcomp.decompress_scratch_size(4096, "ans", ct.uint8)
    )
    scratch = cp.empty((scratch_bytes,), dtype=cp.uint8)
    ct.launch(
        cp.cuda.get_current_stream(),
        (1, 1, 1),
        compression_round_trip_kernel,
        (source, restored, scratch),
    )
    cp.cuda.runtime.deviceSynchronize()
    assert cp.array_equal(source, restored)


if __name__ == "__main__":
    main()
#include <algorithm>
#include <cstddef>
#include <cstdint>

#include "nvcompdx_tile.hpp"

namespace ct = cuda::tiles;
namespace cn = nvcompdx::tiles;

constexpr std::size_t chunk_bytes = 4096;

__tile_global__ void compression_round_trip_kernel(std::uint8_t* input, std::uint8_t* restored, void* scratch) {
    using chunk_shape = ct::shape<chunk_bytes>;
    auto source = ct::partition_view { ct::tensor_span { input, chunk_shape {} }, chunk_shape {} };
    auto destination = ct::partition_view { ct::tensor_span { restored, chunk_shape {} }, chunk_shape {} };

    auto [compressed, compressed_size] = cn::compress<cn::ans>(source.load(0), cn::scratch { scratch });
    destination.store(
        cn::decompress<std::uint8_t, chunk_bytes, cn::ans>(compressed, compressed_size, cn::scratch { scratch }), 0);
}

void launch_compression_round_trip(std::uint8_t* input, std::uint8_t* restored) {
    // The two calls run one after the other, so they can share one scratch.
    constexpr std::size_t scratch_bytes = std::max(cn::compress_scratch_size<cn::ans, std::uint8_t>(chunk_bytes),
                                                   cn::decompress_scratch_size<cn::ans, std::uint8_t>(chunk_bytes));
    void* scratch = nullptr;
    cudaMalloc(&scratch, scratch_bytes);
    compression_round_trip_kernel<<<1, 1>>>(input, restored, scratch);
    cudaFree(scratch);
}

Both routines take a global memory scratch buffer, sized by compress_scratch_size and decompress_scratch_size. The two calls run one after the other, so the quickstarts share one allocation of the larger size between them.

The compressed tile has a fixed capacity; only the returned count of bytes is meaningful. To transfer just those bytes to or from global memory, use masked gather and scatter. Both APIs also accept global memory inputs and outputs directly, without these transfers. Refer to Compression conventions and the direct-memory examples in nvCOMPDx in Tile and Compression cutile-python API.

Building and running#

Install CuPy in addition to the Tile packages to run the Python quickstarts. All three examples use CuPy to allocate device arrays and verify their results.

python fft.py

cuTile Python compiles the kernel at first call and resolves the MathDx calls against the installed libraries.

Compiling directly, -l= hands tileiras the MathDx Tile library that implements fft:

nvcc --enable-tile --tile-preview=simt -std=c++20 -arch=native \
     -I "${MATHDX_TILE}/include" \
     -Xtileiras -l="${MATHDX_TILE}/lib/cufftdx.tilelib" \
     fft.cu -o fft

In a CMake project the imported target carries the Tile compile and link flags, but the project still has to request CUDA C++20 and select an architecture:

find_package(mathdx_tile CONFIG REQUIRED COMPONENTS cufftdx)
target_link_libraries(fft PRIVATE mathdx_tile::cufftdx)
cmake -S . -B build \
      -DCMAKE_CUDA_STANDARD=20 \
      -DCMAKE_CUDA_STANDARD_REQUIRED=ON \
      -DCMAKE_CUDA_ARCHITECTURES=native \
      -DCMAKE_PREFIX_PATH="${MATHDX_TILE}"
cmake --build build

Use the mathdx_tile component corresponding to the library being called: cufftdx, cublasdx, cusolverdx, or nvcompdx. For kernels that use several libraries, see Using several MathDx libraries in one kernel below.

For complete C++ programs, with host allocation, error checking, and a reference result around the kernels above, see Examples. The runtime archives install them under examples/.

Using several MathDx libraries in one kernel#

Each MathDx library provides its own .tilelib file. A kernel that calls routines from more than one library must list all of them on the tileiras command line, or link all of the corresponding CMake targets:

nvcc --enable-tile --tile-preview=simt -std=c++20 -arch=native \
     -I "${MATHDX_TILE}/include" \
     -Xtileiras -l="${MATHDX_TILE}/lib/cusolverdx.tilelib" \
     -Xtileiras -l="${MATHDX_TILE}/lib/cublasdx.tilelib" \
     blocked_potrf.cu -o blocked_potrf

With CMake, request both components and link both imported targets:

find_package(mathdx_tile CONFIG REQUIRED COMPONENTS cusolverdx cublasdx)
target_link_libraries(blocked_potrf PRIVATE
                      mathdx_tile::cusolverdx mathdx_tile::cublasdx)

The blocked_potrf example links both, because it combines cusolverdx::tiles::cholesky_factor with cublasdx::tiles::triangular_solve.

No equivalent step is required in Python. Importing nvmath.tile.linalg binds the libraries that module requires, and each submodule binds only its own, so a program that performs only linear algebra does not require the FFT library to be present.

Troubleshooting#

Symptom

Cause

A library fails to load when nvmath.tile.fft (or linalg, compress) is imported

Verify that nvmath-python and nvidia-libmathdx-cu13 are installed in the active Python environment, for example with python -m pip show nvmath-python nvidia-libmathdx-cu13. The wheels handle library discovery automatically.

The Tile assembler rejects the generated bytecode

The toolchain is not CUDA 13.5. tileiras --version reports which one is on PATH.

A large FFT fails to compile with the CTA size left free

The size only has implementations at larger CTA sizes. Set the count explicitly to a suggestion with found == true from suggested_tile_config, or choose a supported value when no entry is found. See Threads, warps and CTA size.

A factorization result is wrong but nothing failed

info was not checked. See Status codes.

Where to go next#

  • MathDx Tile APIs describes the interop mechanism and the conventions common to every MathDx Tile API.

  • Achieving High Performance describes the parameters that affect performance and how to select them.

  • Examples lists the complete C++ and Python example suites.