Learning Path Through Examples#

This chapter gives a recommended path through the cuBLASDx examples. It is written for readers who want to learn the library by building and modifying working examples, not by reading the API reference front to back.

The examples live under example/cublasdx. When built from the repository, executables are written under build/example/cublasdx/<example-dir>/<example>. Most correctness examples print Success or Failure. Performance examples print timings, throughput, and error measurements.

Before You Start#

Configure the examples for the GPU architecture you plan to run on. The default configuration targets SM80; if your runtime GPU is different, pass the matching architecture explicitly.

cmake -S . -B build -DBUILD_EXAMPLE=ON -DCUBLASDX_CUDA_ARCHITECTURES=80-real
cmake --build build --target cublasdx_examples -j

Then run examples directly from their output directories. For example:

./build/example/cublasdx/01_gemm_introduction/introduction_example
./build/example/cublasdx/02_gemm_precisions/simple_gemm_fp32

To build one example instead of the aggregate target, use its CMake target name. For examples whose directory and file name would otherwise repeat the same words, the target keeps the numeric prefix and drops the duplicate part:

cmake --build build --target 11_gemm_device_performance

Some examples are conditional:

  • GEMM batched tensor examples and emulation examples require CUDA Toolkit 13.1 or newer.

  • TRSM examples require cuBLASDx fatbin/LTO support and may be disabled when fatbin linking is skipped.

  • cuBLASDx/cuFFTDx examples require cuFFTDx to be enabled and found.

  • NVRTC examples are built as executables, but their CTest registration is skipped for NVHPC/NVC++.

Choose Examples By Goal#

Basic GEMM Concepts#

Example

Use it when you want to learn

Useful follow-up

introduction_example

Descriptor composition and the three main GEMM execution styles: shared-memory output, register accumulation into existing C, and register return value.

simple_gemm_fp32

simple_gemm_fp32

A small, direct, correctness-checked GEMM that is easier to modify than performance examples.

simple_gemm_leading_dimensions

simple_gemm_transform

Element-wise load/store transforms around GEMM and how transform work relates to register accumulators.

gemm_conj_transpose

Precision And Data Types#

Example

Use it when you want to learn

Notes

simple_gemm_mixed_precision

How computation precision, storage type, complex type, and scalar type interact.

Good first stop before custom input/output types.

simple_gemm_int8_int8_int32

Integral GEMM and promoted accumulation.

Check supported SMs before modifying.

simple_gemm_fp8

FP8 A/B inputs and higher-precision accumulation/output.

Requires an FP8-capable architecture.

simple_gemm_cfp16

Complex half-precision GEMM.

Compare with simple_gemm_std_complex_fp32.

simple_gemm_std_complex_fp32

Using cuda::std::complex as the data type.

Useful when integrating with C++ code that already uses standard complex types.

Layouts, Tensors, And Shared Memory#

Example

Use it when you want to learn

Notes

simple_gemm_leading_dimensions

Static and dynamic leading dimensions for shared-memory tensor layouts.

Read with LeadingDimension Operator.

simple_gemm_custom_layout

How custom CuTe layouts drive shared-memory sizing, slicing, and tensor construction.

Good before writing hand-tuned data movement.

simple_gemm_aat

How one shared-memory allocation can be viewed as both A and A^T for C = A * A^T.

Useful for aliasing and occupancy tradeoffs.

gemm_conj_transpose

Lazy tensor views such as conj_transpose_view and transpose_view.

Good bridge from GEMM transforms to TRSM tensor views.

Pipelining And Batching#

Example

Use it when you want to learn

Notes

introduction_pipeline

Basic pipelined GEMM setup: host pipeline object, device handle, tile selection, pipeline block dimensions, asynchronous load and compute stages, and execution.

Read before modifying performance examples.

batched_gemm_pipeline

Rank-3 batched tensors, batch tile coordinates, and reset_tile in a batch loop.

Requires CUDA Toolkit 13.1 or newer.

batched_gemm

Non-pipelined batched GEMM using tensor interfaces.

Compare directly with batched_gemm_pipeline.

batched_gemm_fp64

Manual batching inside one CUDA block using BlockDim.

Older style; useful for understanding thread/block ownership.

Performance And Fusion#

Example

Use it when you want to learn

Notes

single_gemm_performance

Block-level GEMM timing for one GEMM tile configuration.

Build in Release mode for meaningful numbers.

device_gemm_performance

Full-device GEMM assembled from cuBLASDx tiles and compared with a library reference path.

Command-line m n k must satisfy tile constraints.

gemm_device_partial_sums

Splitting accumulation into partial sums to trade performance and accuracy.

Treat as an advanced accuracy/performance example.

gemm_fusion

Fusing two dependent GEMMs in one CUDA kernel.

Study shared-memory reuse and synchronization.

fused_gemm_performance

Timing fused GEMMs against an unfused reference path.

Build in Release mode.

blockdim_gemm_fp16

How launch dimensions and BLAS participating block dimensions interact.

Useful before writing non-standard block layouts.

TRSM#

Example

Use it when you want to learn

Notes

trsm_block

Block-level triangular solve through shared memory.

Start here for TRSM.

trsm_thread

One thread solving one TRSM instance.

Good for many small independent solves.

trsm_conj_transpose

Applying conj_transpose_view to solve against a viewed triangular matrix without materializing a transpose.

Read after gemm_conj_transpose.

Specialized And Optional Examples#

Example

Use it when you want to learn

Notes

nvrtc_gemm

Runtime compilation with NVRTC and passing cuBLASDx headers into device source compiled at runtime.

Useful for generated kernels. Source: example/cublasdx/15_nvrtc/nvrtc_gemm.cpp.

nvrtc_trsm

Runtime compilation with NVRTC for a block-level triangular solve.

example/cublasdx/15_nvrtc/nvrtc_trsm.cpp. Requires cuBLASDx fatbin/LTO support.

dgemm_emulation

Ozaki-style double precision emulation using lower precision GEMM operations.

Requires CUDA Toolkit 13.1 or newer.

dgemm_emulation_pipeline

Pipelined emulation controlled by required mantissa bits.

Requires CUDA Toolkit 13.1 or newer.

sgemm_emulation_pipeline

Single precision emulation pipeline using the same mechanism as the double precision pipeline example.

Requires CUDA Toolkit 13.1 or newer.

gemm_fft

GEMM followed by FFT in one kernel using cuBLASDx and cuFFTDx.

Optional; requires cuFFTDx.

gemm_fft_fp16

Half-precision complex FFT/GEMM fusion.

Optional; requires cuFFTDx.

gemm_fft_performance

Timing GEMM+FFT fusion.

Optional; requires cuFFTDx and Release mode for useful timings.

Suggested Modification Order#

When adapting examples to your own kernel, change one layer at a time:

  1. Change m, n, k and rebuild.

  2. Change Precision and Type.

  3. Change Arrangement or leading dimensions.

  4. Change shared-memory layouts.

  5. Move from shared-memory output to register-fragment output.

  6. Add pipelining only after the non-pipelined version is correct.

  7. Add fusion after each individual operation has a checked standalone example.

If a modified example fails, first check that the binary was built for the GPU SM you are running on, then check shared-memory size, launch block dimensions, alignment, and any tile divisibility requirements.

Adaptation Checklists#

Before changing a GEMM example, verify:

  • The descriptor has exactly one Size, Precision, Type, Function, and SM operator.

  • Arrangement or any explicit leading dimensions match the memory allocation and reference path.

  • Alignment matches pointer alignment and leading dimensions.

  • Dynamic shared memory uses get_shared_storage_size for shared-memory C and get_shared_storage_size_ab for register-accumulator C.

  • Every asynchronous global/shared copy is followed by copy_wait() before the copied data is read.

  • The modified example still compares against a reference result before returning success.

Before changing a pipelined GEMM example, verify:

  • global_m and global_n are divisible by tile_m and tile_n, or the kernel has explicit edge handling.

  • global_k / tile_k is at least the chosen pipeline depth, so staged asynchronous loads and staged asynchronous compute both have valid K tiles.

  • The launch uses pipeline->get_block_dim() unless the pipeline is created with fixed_blocksize.

  • The device handle is passed as __grid_constant__ when following the standard pipeline examples.

  • One tile_pipeline object is constructed per kernel execution, reused with reset_tile for persistent work, and destroyed once.

  • Reusable accumulators call finish_accumulation() with the same thread scope as tile_pipeline.execute(accumulator) before reading results unless tile_pipeline.epilogue is used; do not guard the call with if(accumulator.is_thread_active()) because internal warp-specialization may require broader participation.

Before changing a TRSM example, verify:

  • The build links the cuBLASDx fatbin/LTO target; cublasdx_no_lto is GEMM-only.

  • Size<M, N> describes B and X. A is M x M for left-side and N x N for right-side.

  • Arrangement describes storage, while tensor views can change what FillMode means at execute().

  • num_batches is padded or otherwise divisible by BatchesPerBlock for block-level TRSM.

  • B is treated as input/output: restore it before repeated timing runs.

  • The example fails when the reference error is not acceptable.

Before changing emulation examples, verify:

  • The build uses CUDA Toolkit 13.1 or newer.

  • RequiredMantissaBits is set deliberately and the measured error is inspected.

  • Pipelined emulation uses reusable_accumulator where required by the algorithm.