Examples#

The cuBLASDx library provides block-level BLAS examples that cover GEMM and TRSM basics, precision and layout variants, pipelining, batching, fusion, runtime compilation, and performance measurement. If you are new to cuBLASDx, start with Learning Path Through Examples before using this page as a reference list.

Building And Running Examples#

Configure the examples with the CUDA architecture you plan to run on, then build the aggregate target:

cmake -S . -B build -DBUILD_EXAMPLE=ON -DCUBLASDX_CUDA_ARCHITECTURES=80-real
cmake --build build --target cublasdx_examples -j

Executables are written under build/example/cublasdx/<example-dir>/<example>. For example:

./build/example/cublasdx/01_gemm_introduction/introduction_example
./build/example/cublasdx/11_gemm_device_performance/device_gemm_performance

The CMake target name is a build-system identifier, not the executable name. For examples whose directory and file name would repeat the same words, the build target keeps the numeric prefix and drops the duplicate part. For example, build 11_gemm_device_performance to produce the executable shown above. Run cmake --build build --target help to list the exact targets generated for your configuration.

Architecture notation appears in three forms:

Context	Example	Meaning
cuBLASDx descriptor	`SM<890>()`	Ada SM 89.0 encoded as an integer template argument.
`nvcc` command line	`-arch=sm_89`	Compile device code for SM 89.
CMake	`CUBLASDX_CUDA_ARCHITECTURES=89-real`	Build real GPU code for SM 89 through CMake’s CUDA architecture property.

Some examples are conditional:

batched_gemm, batched_gemm_pipeline, dgemm_emulation, dgemm_emulation_pipeline, and sgemm_emulation_pipeline require CUDA Toolkit 13.1 or newer.
TRSM examples require cuBLASDx fatbin/LTO support and are disabled when fatbin linking is skipped.
gemm_fft, gemm_fft_fp16, and gemm_fft_performance require cuFFTDx to be enabled and found.
nvrtc_gemm is built as an executable, but its CTest registration is skipped for NVHPC/NVC++.
nvrtc_trsm requires cuBLASDx fatbin/LTO support. When built, its CTest registration is skipped for NVHPC/NVC++.

When adapting examples, use the following metadata as a first-pass guide:

Table 4 Example adaptation metadata#
Area	Typical sources	Build/runtime gate	Safe first knobs
Basic GEMM	`02_gemm_precisions` and `03_gemm_complex`	MMA support depends on precision and SM.	`Size`, `Precision`, `Arrangement`, `Alignment`, `BlockDim`.
Pipelined GEMM	`01_gemm_introduction/introduction_pipeline.cu` and `11_gemm_device_performance`	Shows asynchronous load and compute stage overlap; use `pipeline->get_block_dim()` unless `fixed_blocksize` is selected.	Tile sizes, pipeline depth, `EnableInputStreaming`, `StaticBlockDim`.
Batched GEMM	`19_gemm_batched`	Requires CUDA Toolkit 13.1 or newer.	Batch count, batch stride, grid-stride loop, `reset_tile`.
TRSM	`17_trsm` and `18_tensor_transform/trsm_conj_transpose.cu`	Requires fatbin/LTO; disabled when fatbin linking is skipped.	`Side`, `FillMode`, `Diag`, `BatchesPerBlock`, batch padding.
Emulation	`16_dgemm_emulation` and `20_dgemm_emulation_pipeline`	Requires CUDA Toolkit 13.1 or newer.	`RequiredMantissaBits`, tile sizes, `reusable_accumulator` validation.

Table 5 Example overview#
Group	Subgroup	Example	Description
Introduction	GEMM	`introduction_example`	Walks through descriptor creation and the shared-memory and register-result GEMM APIs.
Introduction	Pipeline	`introduction_pipeline`	Introduces the host pipeline object, device handle, tile pipeline, and epilogue flow.
Simple GEMM	Basic	`simple_gemm_fp32`	Performs a checked fp32 GEMM and is the smallest general-purpose starting point.
Simple GEMM	Precision	`simple_gemm_mixed_precision`	Uses different precisions and storage types for matrices `A`, `B`, and `C`.
Simple GEMM	Precision	`simple_gemm_int8_int8_int32`	Performs integral GEMM using int8 inputs and int32 accumulation.
Simple GEMM	Precision	`simple_gemm_fp8`	Performs fp8 GEMM on architectures that support the required fp8 operations.
Simple GEMM	Complex	`simple_gemm_cfp16`	Performs complex half-precision GEMM.
Simple GEMM	Complex	`simple_gemm_std_complex_fp32`	Uses `cuda::std::complex<float>` as the matrix element type.
Simple GEMM	Layout	`simple_gemm_leading_dimensions`	Demonstrates non-default leading dimensions and padded matrix storage.
Simple GEMM	Layout	`simple_gemm_custom_layout`	Uses custom CuTe layouts for shared-memory matrices.
Simple GEMM	Layout	`simple_gemm_aat`	Computes `C = A * A^T` while reusing one shared-memory allocation for both views of `A`.
Simple GEMM	Transform	`simple_gemm_transform`	Applies element-wise load and store transform operators around GEMM.
Simple GEMM	Transform	`gemm_conj_transpose`	Applies tensor views such as conjugate transpose to GEMM inputs without materializing a copy.
Simple GEMM	Register I/O	`simple_gemm_fp32_decoupled`	Uses lower-precision input/output storage with higher-precision computation and register fragments.
Runtime Compilation	NVRTC	`nvrtc_gemm`	Compiles a GEMM kernel at runtime with NVRTC and passes cuBLASDx headers to device code. Source: `example/cublasdx/15_nvrtc/nvrtc_gemm.cpp`.
Runtime Compilation	NVRTC	`nvrtc_trsm`	Compiles a block-level TRSM kernel at runtime with NVRTC. Source: `example/cublasdx/15_nvrtc/nvrtc_trsm.cpp`. Requires cuBLASDx fatbin/LTO support.
Performance	Block GEMM	`single_gemm_performance`	Benchmarks a single block-level GEMM tile configuration.
Performance	Device GEMM	`device_gemm_performance`	Builds a full-device GEMM from cuBLASDx tiles and compares with a reference path.
Performance	Fusion	`fused_gemm_performance`	Benchmarks two fused GEMMs against an unfused reference path.
Advanced GEMM	Batching	`batched_gemm`	Demonstrates rank-3 tensor batching for non-pipelined GEMM. Requires CUDA Toolkit 13.1 or newer.
Advanced GEMM	Batching	`batched_gemm_pipeline`	Demonstrates rank-3 tensor batching with the pipeline API. Requires CUDA Toolkit 13.1 or newer.
Advanced GEMM	Batching	`batched_gemm_fp64`	Shows manual batching inside one CUDA block with `BlockDim`.
Advanced GEMM	Block shape	`blockdim_gemm_fp16`	Shows how to launch dimensions interact with `BlockDim` and participating threads.
Advanced GEMM	Accuracy	`gemm_device_partial_sums`	Offloads partial accumulation to a higher-precision register array.
Advanced GEMM	Fusion	`gemm_fusion`	Performs two dependent GEMMs in one CUDA kernel.
Advanced GEMM	cuFFTDx fusion	`gemm_fft`	Fuses GEMM and FFT in one kernel. Requires cuFFTDx.
Advanced GEMM	cuFFTDx fusion	`gemm_fft_fp16`	Fuses half-precision complex GEMM and FFT. Requires cuFFTDx.
Advanced GEMM	cuFFTDx fusion	`gemm_fft_performance`	Benchmarks GEMM and FFT fusion. Requires cuFFTDx.
TRSM	Block	`trsm_block`	Solves triangular systems cooperatively in a CUDA block using shared memory.
TRSM	Thread	`trsm_thread`	Solves many small triangular systems independently, one per CUDA thread.
TRSM	Tensor views	`trsm_conj_transpose`	Uses `conj_transpose_view` for TRSM without copying or transposing matrix data.
Emulation	Ozaki	`dgemm_emulation`	Emulates double-precision GEMM using lower-precision GEMM operations. Requires CUDA Toolkit 13.1 or newer.
Emulation	Pipeline	`dgemm_emulation_pipeline`	Runs pipelined double-precision emulation controlled by required mantissa bits. Requires CUDA Toolkit 13.1 or newer.
Emulation	Pipeline	`sgemm_emulation_pipeline`	Runs pipelined single-precision emulation using the same mantissa-bit mechanism. Requires CUDA Toolkit 13.1 or newer.

Introduction Examples#

introduction_example
introduction_pipeline

Introduction examples are used throughout the documentation to explain the basics of the cuBLASDx API. introduction_example is the companion example for Using cuBLASDx GEMM and intentionally demonstrates multiple GEMM execution paths: a shared-memory C path, an explicit register accumulator path, and a return-value register accumulator path. Seeing more than one kernel launch in this example is expected; the launches contrast different API styles rather than representing hidden work required by one API.

introduction_pipeline is the companion example for Using Pipelined GEMM. It shows the host-side pipeline object, the device handle passed to the kernel, per-block tile pipelines, asynchronous load and compute stage overlap, and the single epilogue stage for the output tile.

Simple GEMM Examples#

simple_gemm_fp32
simple_gemm_mixed_precision
simple_gemm_int8_int8_int32
simple_gemm_fp8
simple_gemm_cfp16
simple_gemm_std_complex_fp32
simple_gemm_leading_dimensions
simple_gemm_custom_layout
simple_gemm_aat
simple_gemm_transform
gemm_conj_transpose
simple_gemm_fp32_decoupled

These examples perform a general matrix multiply (GEMM) operation within a CUDA block. They demonstrate how to create a BLAS description, allocate memory, choose block dimensions, configure shared memory, move data into shared memory or register fragments, execute GEMM, store results, and verify against cuBLAS.

simple_gemm_fp32 is the smallest checked GEMM example and is the easiest place to modify m, n, k, precision, or arrangement.

simple_gemm_mixed_precision shows how matrices A, B, and C can use different storage types. The scaling factors \(\alpha\) and \(\beta\) use the same precision and type as matrix C.

simple_gemm_int8_int8_int32 and simple_gemm_fp8 demonstrate lower precision input types with promoted accumulation. Check the supported SMs before changing the descriptor because not every architecture supports every precision combination.

simple_gemm_cfp16 and simple_gemm_std_complex_fp32 cover complex data. The latter uses cuda::std::complex<float> from the CUDA C++ Standard Library, but the same pattern applies to compatible custom element types such as CUDA float2.

simple_gemm_leading_dimensions shows static leading dimensions for matrices A, B, and C using the LeadingDimension operator. For optimal performance, start with suggested_leading_dimension_of.

simple_gemm_custom_layout uses custom CuTe layouts for shared-memory tensors. Study it before writing hand-tuned shared-memory slicing or non-standard layouts.

simple_gemm_aat computes C = A * A^T while both views of A share the same memory allocation. This is useful when aliasing can reduce shared-memory usage or increase occupancy.

simple_gemm_transform applies a_load_op, b_load_op, c_load_op, and c_store_op element-wise when loading or storing matrices. gemm_conj_transpose demonstrates lazy tensor views such as conj_transpose_view without materializing a transposed copy.

simple_gemm_fp32_decoupled demonstrates the register-output path with separate input/output storage type and compute precision.

NVRTC Examples#

nvrtc_gemm (example/cublasdx/15_nvrtc/nvrtc_gemm.cpp)
nvrtc_trsm (example/cublasdx/15_nvrtc/nvrtc_trsm.cpp)

The NVRTC examples demonstrate how to use cuBLASDx with runtime compilation. The BLAS descriptions created with cuBLASDx operators live in the device code passed to NVRTC, and cublasdx.hpp is included by that runtime-compiled source. nvrtc_trsm runtime-compiles and launches a block-level triangular solve.

Note

Since version 0.1, cuBLASDx has experimental support for compilation with NVRTC. See Requirements And Functionality.

GEMM Performance#

single_gemm_performance
device_gemm_performance
fused_gemm_performance

single_gemm_performance measures one block-level GEMM tile configuration. Use it when you want to change a descriptor and measure the impact without the extra scheduling logic of a full-device GEMM.

device_gemm_performance compares cuBLASDx with cuBLAS for a GEMM that spans the entire GPU. It does not provide universally optimal tile sizes; custom precisions and dimensions may require parameter search. Global GEMM dimensions are dynamic values and can be passed as command line arguments:

# Perform default size GEMM with cuBLASDx static tile specified in code
./device_gemm_performance

# Perform custom size GEMM with cuBLASDx static tile specified in code
./device_gemm_performance m n k

fused_gemm_performance measures two GEMM operations fused into one kernel and compares the result against an unfused reference path.

Advanced GEMM Examples#

batched_gemm
batched_gemm_pipeline
batched_gemm_fp64
blockdim_gemm_fp16
gemm_device_partial_sums
gemm_fusion
gemm_fft
gemm_fft_fp16
gemm_fft_performance

batched_gemm and batched_gemm_pipeline demonstrate rank-3 tensor batching for GEMM and pipelined GEMM. They require CUDA Toolkit 13.1 or newer.

batched_gemm_fp64 and blockdim_gemm_fp16 demonstrate the BlockDim operator. batched_gemm_fp64 uses a 1D BlockDim and launches 2D block dimensions to process multiple GEMMs in a single CUDA block. blockdim_gemm_fp16 shows a safe execution when the launch block dimensions differ from the participating BLAS block dimensions.

gemm_device_partial_sums uses an extra register array in higher precision to offload partial accumulation every N iterations and reduce precision loss. Treat it as an advanced accuracy/performance example.

gemm_fusion fuses two dependent GEMMs into one CUDA kernel. gemm_fft, gemm_fft_fp16, and gemm_fft_performance fuse GEMM and FFT using both cuBLASDx and cuFFTDx.

TRSM Examples#

trsm_block
trsm_thread
trsm_conj_transpose

trsm_block demonstrates block-level triangular solve: a CUDA block cooperates to solve one or more TRSM instances through shared memory. See Using cuBLASDx TRSM for a detailed walkthrough.

trsm_thread demonstrates thread-level triangular solve: each CUDA thread solves one independent TRSM instance directly from global memory without shared memory.

trsm_conj_transpose applies cublasdx::conj_transpose_view to the triangular matrix at execute time to solve A^H * X = B without copying or transposing data in memory.

Emulation Examples#

dgemm_emulation
dgemm_emulation_pipeline
sgemm_emulation_pipeline

The emulation examples demonstrate Ozaki-style precision emulation using lower precision GEMM operations. dgemm_emulation shows the non-pipelined double precision emulation flow:

Decompose higher precision matrices into multiple lower precision slices.
Perform GEMM on combinations of slices.
Reconstruct the final higher precision result.

dgemm_emulation_pipeline and sgemm_emulation_pipeline combine the emulation flow with the pipeline API and are controlled by the required mantissa bits. These examples require CUDA Toolkit 13.1 or newer.