Achieving High Performance#
In High-Performance Computing, the ability to write customized code enables users to target better performance. In the case of cuFFTDx, the potential for performance improvement of existing FFT applications is high, but it greatly depends on how the library is used. Taking the regular cuFFT library as baseline, the performance may be up to one order of magnitude better or worse. For this reason porting existing sources to cuFFTDx should always be done in parallel with performance analysis. Below we list general advice that may help in this process.
General Advice#
Try library-provided default settings to start with best compute performance.
Best parameters for compute bound and memory bound kernels might not be identical.
Ensure FFT kernel runs enough blocks to fill a GPU for peak performance.
Merge adjacent memory bound kernels (pre- and post-processing) with an FFT kernel to save global memory trips.
Memory Management#
Avoid reading/writing data from global memory.
Ensure global memory reads/writes are coalesced (increase the value of FFTs Per Block Operator if needed).
Use
sharedmemory or extra registers to store the temporary data.
Kernel Fusion#
For complex kernels consider adjusting FFT operation to match user kernel. (ie. tweaking Elements Per Thread Operator will change required CUDA block size). Upcoming versions of cuFFTDx will offer more customization options.
For simple operations consider merging operations into FFT kernel optimized for FFT performance.
Tuning Block Execution#
The following operators change how a block FFT uses shared memory and threads. Start with the defaults
(transposition::any, thread_layout::regular), then try the alternatives below and measure. The best choice
depends on the FFT size, precision and GPU architecture, and it interacts with
ElementsPerThread and FFTsPerBlock, so tune those together.
In the cuFFTDx examples, tuning ElementsPerThread and FFTsPerBlock per size gave up to a further 1.4x.
Transposition
The Transposition operator selects the shared memory layout the FFT uses internally to exchange data between threads.
transposition::by_component: needs the least shared memory. Try it when shared memory limits how many FFTs or blocks fit on an SM.transposition::by_complex: needs more shared memory but executes fewer instructions. Try it when shared memory is not a constraint.transposition::by_component_padded: trades part of thetransposition::by_componentsaving for fewer shared memory bank conflicts. It is available only with the default PTX code (not with LTO) and for power-of-two sizes above 32. In the mixed-precision example it is worth about 1.5x compared to the default transposition on B200.transposition::none: uses no shared memory scratch, and requiresElementsPerThreadequal to the size, so it fits only small sizes.
Not every layout exists for every size, precision and architecture. Use transposition::any to let cuFFTDx choose, or list the available
layouts at runtime with the trp argument of get_all_implementations.
ThreadLayout
The ThreadLayout operator selects which threads of a block work on which data.
With thread_layout::regular, consecutive threads (threadIdx.x) work on elements of the same FFT. With thread_layout::transposed, consecutive threads
work on different FFTs. Choose the layout that gives consecutive threads consecutive global memory addresses, which
usually means thread_layout::transposed for the strided dimensions of multi-dimensional FFTs. In the 3D convolution example,
using thread_layout::transposed for the strided dimensions gives speedups of 1.3x to 2.6x over cuFFT with Thrust. thread_layout::transposed is
available only with PTX code. Compute data indices with FFT::working_group, not from threadIdx.
ThreadLinearization
The ThreadLinearization operator lets the FFT use any block shape, because it maps
all three block dimensions to one linear thread index. Use it, together with BlockDim, when
the block shape is dictated by other code in your kernel, for example fused pre- or post-processing. It does not make the FFT
itself faster. Launch the kernel with exactly the BlockDim you set, or with FFT::block_dim if you set none. Threads the FFT
does not need are inactive (is_thread_active() is false), but they must still call
execute: guard only the data accesses around it.
Advanced#
For FFT loads not filling the GPU entirely consider running parallel kernels in a separate stream.
Nsight Compute Occupancy Calculator [5] and/or cudaOccupancyMaxActiveBlocksPerMultiprocessor [7] function to determine the optimum launch parameters.
Use the Nsight Compute [6] to determine what extra resources are available without losing occupancy.