Getting Started#
This section shows how to install MathDx Tile packages and build three kernels:
A 256-point FFT.
A Cholesky factorization.
A compression round trip.
Each kernel has cuTile Python and CUDA Tile C++ versions based on the quick-start sources included with this documentation. Refer to the Examples section for the complete example suites.
Requirements#
Component |
Requirement |
|---|---|
GPU |
Compute capability 8.0 (NVIDIA Ampere) or newer |
Driver |
r580 or newer |
Operating system |
Linux or Windows |
CPU architecture |
x86_64 or aarch64 |
Toolchain (C++) |
A CUDA 13.5 Tile toolchain, providing both |
C++ standard |
C++20 |
Python |
A Python version supported by cuTile Python |
Warning
MathDx Tile is compatible with CUDA 13.5 only. To check the version,
run tileiras --version for an installed toolkit, or
python -m pip show nvidia-cuda-tileiras for the cuda-toolkit
wheels. If both are present, cuTile uses the wheel’s tileiras.
Both the C++ and the Python paths require a Tile-capable toolchain. A standard CUDA compiler cannot build these kernels.
Installation#
MathDx Tile uses the libmathdx version number: the mathdx_tile archive,
the nvidia-libmathdx-cu13 wheel, and the libmathdx inside them share
one version. Use it for <version> in the C++ commands below. See
Release notes.
On the Python side, install nvmath-python with its cu13-dx extra. The
MathDx Tile APIs need all of these packages, and the extra brings them in:
nvidia-libmathdx-cu13: the Tile tilelibs and thenvmath.tilemodules.cuda-tile: cuTile Python.cuda-tile-preview: the experimental preview APIs of cuTile Python, including the foreign call API that MathDx Tile uses. It is released together withcuda-tile.
MathDx Tile 0.5.0 pairs with nvmath-python 1.0.1 or newer. See the
compatibility table in the release notes.
python -m venv .venv
source .venv/bin/activate
python -m pip install "nvmath-python[cu13-dx]>=1.0.1"
The nvidia-libmathdx-cu13 wheel includes the MathDx Tile libraries,
and nvmath.tile finds them without any path settings. The
cuda-toolkit wheels supply NVRTC and tileiras, and pip installs
CUDA 13.5, which MathDx Tile requires.
Use a system CUDA toolkit
To use an installed CUDA 13.5 toolkit instead of the cuda-toolkit
wheels, point the environment at it:
export CUDA_HOME=/usr/local/cuda-13.5
export PATH="${CUDA_HOME}/bin:${PATH}"
export LD_LIBRARY_PATH="${CUDA_HOME}/lib64:${LD_LIBRARY_PATH}"
Then install nvmath-python with its sysctk13-dx extra, which
installs the libmathdx wheel and the cuTile packages but not
cuda-toolkit:
python -m pip install "nvmath-python[sysctk13-dx]>=1.0.1"
Download the mathdx_tile archive from the
MathDx Tile download page,
then unpack it:
mkdir mathdx_tile
tar xzf mathdx_tile-linux-x86_64-<version>.tar.gz -C mathdx_tile --strip-components=1
export MATHDX_TILE="${PWD}/mathdx_tile"
${MATHDX_TILE}/include holds the Tile headers, ${MATHDX_TILE}/lib
holds libmathdx and the .tilelib libraries, and
${MATHDX_TILE}/examples holds the C++ Examples. Windows ships
the same tree as a .zip.
1. FFT#
The first kernel computes a 256-point forward FFT of an all-ones signal.
Its DFT is \(N\) at bin 0 and zero at every other bin, so the result is easy to check.
The Python source is a complete program.
The C++ source contains the kernel and its launch wrapper.
Call the wrapper from the application’s main.
import cuda.tile as ct
import cupy as cp
import nvmath.tile.fft as mfft
@ct.kernel(num_worker_warps=4)
def fft_kernel(source: ct.Array, destination: ct.Array):
signal = ct.load(source, (0, 0), shape=(256, 2))
ct.store(destination, (0, 0), mfft.fft(signal, axis=0))
def main():
source = cp.zeros((256, 2), dtype=cp.float32)
source[:, 0] = 1
destination = cp.zeros_like(source)
ct.launch(cp.cuda.get_current_stream(), (1, 1, 1), fft_kernel, (source, destination))
cp.cuda.runtime.deviceSynchronize()
assert cp.allclose(destination[0], cp.asarray([256, 0], dtype=cp.float32))
assert cp.allclose(destination[1:], 0, atol=1e-3)
if __name__ == "__main__":
main()
#include "cufftdx_tile.hpp"
namespace ct = cuda::tiles;
__tile_global__ void fft_kernel(float* input, float* output) {
using signal_shape = ct::shape<256, 2>;
auto source = ct::partition_view { ct::tensor_span { input, signal_shape {} }, signal_shape {} };
auto destination = ct::partition_view { ct::tensor_span { output, signal_shape {} }, signal_shape {} };
auto spectrum = cufftdx::tiles::fft(source.load(0, 0), cufftdx::tiles::axis<0>);
destination.store(spectrum, 0, 0);
}
void launch_fft(float* input, float* output) {
fft_kernel<<<1, 1>>>(input, output);
}
Four properties of this kernel merit further explanation.
The tile shape is
(256, 2)rather than(256,). The MathDx Tile APIs provide no complex element type, so a complex signal is represented as a real tile with a trailing interleaved(real, imaginary)axis of extent 2. A rank-one transform of \(N\) complex values is therefore a rank-two tile, transformed alongaxis=0. Refer to FFT conventions.The transform axis is specified rather than inferred.
axis=0in Python andcufftdx::tiles::axis<0>in C++ select the logical transform dimension, excluding the complex-pair axis. Negative values are counted from the last logical dimension.The launch block dimension is 1, because the CTA size is not determined by the launch configuration. The compiler selects it, optionally guided by the
num_worker_warpshint in Python or[[cutile::hint(0, num_worker_warps_per_cta = ...)]]in C++. The grid dimension is unaffected. Refer to Threads, warps and CTA size.The result is unnormalized.
fftandifftapply no scaling in either direction, so a forward transform followed by an inverse transform requires an explicit multiplication by \(1/N\).
2. Factorization#
The linear algebra routines return a status value alongside the result and accept a batch dimension. The Python source below performs a triangular solve and a factorization within a single kernel, which is a common reason for using the Tile APIs.
import cuda.tile as ct
import cupy as cp
import nvmath.tile.linalg as mlinalg
@ct.kernel(num_worker_warps=4)
def linalg_kernel(
triangular: ct.Array,
rhs: ct.Array,
positive_definite: ct.Array,
solution: ct.Array,
factor: ct.Array,
status: ct.Array,
):
a = ct.load(triangular, (0, 0), shape=(16, 16))
b = ct.load(rhs, (0, 0), shape=(16, 16))
x = mlinalg.triangular_solve(
a, b, mlinalg.FillMode.LOWER, mlinalg.DiagMode.NON_UNIT, mlinalg.SideMode.LEFT
)
ct.store(solution, (0, 0), x)
matrix = ct.load(positive_definite, (0, 0), shape=(16, 16))
cholesky, info = mlinalg.cholesky_factor(matrix, mlinalg.FillMode.LOWER)
ct.store(factor, (0, 0), cholesky)
ct.store(status, (0,), info)
def main():
triangular = cp.tril(cp.ones((16, 16), dtype=cp.float32)) + cp.eye(16, dtype=cp.float32)
rhs = cp.eye(16, dtype=cp.float32)
positive_definite = triangular @ triangular.T
solution = cp.zeros_like(rhs)
factor = cp.zeros_like(positive_definite)
status = cp.full((1,), -1, dtype=cp.int32)
ct.launch(
cp.cuda.get_current_stream(),
(1, 1, 1),
linalg_kernel,
(triangular, rhs, positive_definite, solution, factor, status),
)
cp.cuda.runtime.deviceSynchronize()
assert int(status[0]) == 0
assert cp.allclose(triangular @ solution, rhs, atol=1e-4)
lower = cp.tril(factor)
assert cp.allclose(lower @ lower.T, positive_definite, atol=1e-4)
if __name__ == "__main__":
main()
#include "cusolverdx_tile.hpp"
namespace ct = cuda::tiles;
__tile_global__ void cholesky_kernel(float* matrix, float* factor, int* status) {
using matrix_shape = ct::shape<16, 16>;
auto source = ct::partition_view { ct::tensor_span { matrix, matrix_shape {} }, matrix_shape {} };
auto destination = ct::partition_view { ct::tensor_span { factor, matrix_shape {} }, matrix_shape {} };
auto [result, info] = cusolverdx::tiles::cholesky_factor(source.load(0, 0), cusolverdx::tiles::lower);
destination.store(result, 0, 0);
*status = static_cast<int>(info);
}
void launch_cholesky(float* matrix, float* factor, int* status) {
cholesky_kernel<<<1, 1>>>(matrix, factor, status);
}
info is 0 on success, while a positive value identifies the step of
the factorization at which the failure occurred. It is not checked
automatically, and a kernel that ignores it will store an invalid factor.
Refer to Status codes.
Every linear algebra routine also accepts a batch dimension. Loading a
(B, M, M)
tile in place of an (M, M) tile factors B independent matrices within
the same CTA, and returns info with shape (B,). This arrangement
allows small factorizations to utilize a GPU that would otherwise remain
largely idle:
@ct.kernel(num_worker_warps=4)
def cholesky_batched_kernel(a_data: ct.Array, l_data: ct.Array, status: ct.Array):
a = ct.load(a_data, (0, 0, 0), shape=(B, M, M))
factors, info = mlinalg.cholesky_factor(a, mlinalg.FillMode.LOWER)
ct.store(l_data, (0, 0, 0), factors)
ct.store(status, (0,), info)
using batch_shape = ct::shape<B, M, M>;
auto source = ct::partition_view { ct::tensor_span { matrix, batch_shape {} }, batch_shape {} };
auto [result, info] = cusolverdx::tiles::cholesky_factor(source.load(0, 0, 0),
cusolverdx::tiles::lower);
The number of problems that fit within one CTA is limited by shared memory, so the batch extent must be balanced against the matrix size. Refer to Achieving High Performance.
3. Compression#
The number of bytes to which a chunk compresses depends on the data, so
compress returns a count alongside the compressed tile, and decompress
requires that count in order to reverse the operation.
import cuda.tile as ct
import cupy as cp
import nvmath.tile.compress as mcomp
@ct.kernel
def compression_round_trip_kernel(source: ct.Array, restored: ct.Array, scratch: ct.Array):
data = ct.load(source, (0,), shape=(4096,))
compressed, count = mcomp.compress(data, "ans", scratch=scratch)
result = mcomp.decompress(compressed, count, "ans", 4096, ct.uint8, scratch)
ct.store(restored, (0,), result)
def main():
source = (cp.arange(4096, dtype=cp.uint32) % 23).astype(cp.uint8)
restored = cp.zeros_like(source)
# The two calls run one after the other, so they can share one scratch.
scratch_bytes = max(
mcomp.compress_scratch_size(4096, "ans", ct.uint8), mcomp.decompress_scratch_size(4096, "ans", ct.uint8)
)
scratch = cp.empty((scratch_bytes,), dtype=cp.uint8)
ct.launch(
cp.cuda.get_current_stream(),
(1, 1, 1),
compression_round_trip_kernel,
(source, restored, scratch),
)
cp.cuda.runtime.deviceSynchronize()
assert cp.array_equal(source, restored)
if __name__ == "__main__":
main()
#include <algorithm>
#include <cstddef>
#include <cstdint>
#include "nvcompdx_tile.hpp"
namespace ct = cuda::tiles;
namespace cn = nvcompdx::tiles;
constexpr std::size_t chunk_bytes = 4096;
__tile_global__ void compression_round_trip_kernel(std::uint8_t* input, std::uint8_t* restored, void* scratch) {
using chunk_shape = ct::shape<chunk_bytes>;
auto source = ct::partition_view { ct::tensor_span { input, chunk_shape {} }, chunk_shape {} };
auto destination = ct::partition_view { ct::tensor_span { restored, chunk_shape {} }, chunk_shape {} };
auto [compressed, compressed_size] = cn::compress<cn::ans>(source.load(0), cn::scratch { scratch });
destination.store(
cn::decompress<std::uint8_t, chunk_bytes, cn::ans>(compressed, compressed_size, cn::scratch { scratch }), 0);
}
void launch_compression_round_trip(std::uint8_t* input, std::uint8_t* restored) {
// The two calls run one after the other, so they can share one scratch.
constexpr std::size_t scratch_bytes = std::max(cn::compress_scratch_size<cn::ans, std::uint8_t>(chunk_bytes),
cn::decompress_scratch_size<cn::ans, std::uint8_t>(chunk_bytes));
void* scratch = nullptr;
cudaMalloc(&scratch, scratch_bytes);
compression_round_trip_kernel<<<1, 1>>>(input, restored, scratch);
cudaFree(scratch);
}
Both routines take a global memory scratch buffer, sized by
compress_scratch_size and decompress_scratch_size. The two calls run
one after the other, so the quickstarts share one allocation of the larger
size between them.
The compressed tile has a fixed capacity; only the returned count of bytes is meaningful. To transfer just those bytes to or from global memory, use masked gather and scatter. Both APIs also accept global memory inputs and outputs directly, without these transfers. Refer to Compression conventions and the direct-memory examples in nvCOMPDx in Tile and Compression cutile-python API.
Building and running#
Install CuPy in addition to the Tile packages to run the Python quickstarts. All three examples use CuPy to allocate device arrays and verify their results.
python fft.py
cuTile Python compiles the kernel at first call and resolves the MathDx calls against the installed libraries.
Compiling directly, -l= hands tileiras the MathDx Tile library
that implements fft:
nvcc --enable-tile --tile-preview=simt -std=c++20 -arch=native \
-I "${MATHDX_TILE}/include" \
-Xtileiras -l="${MATHDX_TILE}/lib/cufftdx.tilelib" \
fft.cu -o fft
In a CMake project the imported target carries the Tile compile and link flags, but the project still has to request CUDA C++20 and select an architecture:
find_package(mathdx_tile CONFIG REQUIRED COMPONENTS cufftdx)
target_link_libraries(fft PRIVATE mathdx_tile::cufftdx)
cmake -S . -B build \
-DCMAKE_CUDA_STANDARD=20 \
-DCMAKE_CUDA_STANDARD_REQUIRED=ON \
-DCMAKE_CUDA_ARCHITECTURES=native \
-DCMAKE_PREFIX_PATH="${MATHDX_TILE}"
cmake --build build
Use the mathdx_tile component corresponding to the library being called:
cufftdx, cublasdx, cusolverdx, or nvcompdx. For kernels that
use several libraries, see Using several MathDx libraries in one kernel below.
For complete C++ programs, with host allocation, error checking, and a
reference result around the kernels above, see Examples. The runtime
archives install them under examples/.
Using several MathDx libraries in one kernel#
Each MathDx library provides its own .tilelib file. A kernel that calls
routines from more than one library must list all of them on the tileiras
command line, or link all of the corresponding CMake targets:
nvcc --enable-tile --tile-preview=simt -std=c++20 -arch=native \
-I "${MATHDX_TILE}/include" \
-Xtileiras -l="${MATHDX_TILE}/lib/cusolverdx.tilelib" \
-Xtileiras -l="${MATHDX_TILE}/lib/cublasdx.tilelib" \
blocked_potrf.cu -o blocked_potrf
With CMake, request both components and link both imported targets:
find_package(mathdx_tile CONFIG REQUIRED COMPONENTS cusolverdx cublasdx)
target_link_libraries(blocked_potrf PRIVATE
mathdx_tile::cusolverdx mathdx_tile::cublasdx)
The blocked_potrf example links both, because it combines
cusolverdx::tiles::cholesky_factor with
cublasdx::tiles::triangular_solve.
No equivalent step is required in Python. Importing nvmath.tile.linalg
binds the libraries that module requires, and each submodule binds only its
own, so a program that performs only linear algebra does not require the FFT
library to be present.
Troubleshooting#
Symptom |
Cause |
|---|---|
A library fails to load when |
Verify that |
The Tile assembler rejects the generated bytecode |
The toolchain is not CUDA 13.5. |
A large FFT fails to compile with the CTA size left free |
The size only has implementations at larger CTA sizes. Set
the count explicitly to a suggestion with |
A factorization result is wrong but nothing failed |
|
Where to go next#
MathDx Tile APIs describes the interop mechanism and the conventions common to every MathDx Tile API.
Achieving High Performance describes the parameters that affect performance and how to select them.
Examples lists the complete C++ and Python example suites.