Overview

The CUDA Profiling Tools Interface (CUPTI) enables the creation of profiling and tracing tools that target CUDA applications. CUPTI provides the following APIs: the Activity API, the Callback API, the Host Profiling API, the Range Profiling API, the PC Sampling API, the SASS Metric API, the PM Sampling API, the Checkpoint API, the Profiling API, the Event API, and the Metric API. Using these APIs, you can develop profiling tools that offer insights into the CPU and GPU behavior of CUDA applications. CUPTI is delivered as a dynamic library on all platforms supported by CUDA.

In this CUPTI document, Tracing refers to the collection of timestamps and additional information for CUDA activities such as CUDA APIs, kernel launches and memory copies during the execution of a CUDA application. Tracing helps in identifying performance issues for the CUDA code by telling you which parts of a program require the most time. Tracing information can be collected using the Activity and Callback APIs.

In this CUPTI document, Profiling refers to the collection of GPU performance metrics for individual kernels or a set of kernels in isolation. This process may involve multiple replays of the kernels or the entire application to gather comprehensive GPU performance data. For Volta and earlier GPU architectures, these metrics can be collected using CUPTI Event and Metric APIs. For Volta and later GPU architectures, Range and Host Profiling APIs replace this functionality. Refer to the section Evolution of the profiling APIs for more details. The SASS Metric API collects kernel performance metrics at the source level. The PM Sampling API collects metrics by sampling the GPU’s performance monitors (PM) periodically at fixed intervals. And periodic sampling of the warp program counter and warp scheduler state is captured using the PC Sampling API.

Table 1. Description of CUPTI APIs

CUPTI API	Feature Description
Activity	Asynchronously record CUDA activities, e.g. CUDA API, Kernel, memory copy
Callback	CUDA event callback mechanism to notify subscriber that a specific CUDA event executed e.g. “Entering CUDA runtime memory copy”
Host Profiling	Host APIs for enumeration, configuration and evaluation of performance metrics
Range Profiling	Target APIs for collection of performance metrics for a range of execution
PC Sampling	Collect continuous mode PC Sampling data without serializing kernel execution
SASS Metrics	Collect kernel performance metrics at the source level using SASS patching
PM Sampling	Collect hardware metrics by sampling the GPU performance monitors (PM) periodically at fixed intervals
Profiling	Collect performance metrics for a range of execution
Event	Collect kernel performance counters for a kernel execution
Metric	Collect kernel performance metrics for a kernel execution
Checkpoint	Provides support for automatically saving and restoring the functional state of the CUDA device

CUPTI Profiling API vs. NVIDIA Nsight Perf SDK

CUPTI Profiling API and CUPTI Range Profiling API support profiling of CUDA kernels and these allow collection of GPU performance metrics for a particular kernel or range of kernels at the CUDA context level. NVIDIA Nsight Perf SDK supports graphics APIs (i.e. DirectX, Vulkan, OpenGL) allowing collection of GPU performance metrics at graphics device, context and queue levels. Both NVIDIA Nsight PerfSDK and CUPTI Profiling API share the host APIs (i.e. metrics enumeration, configuration and evaluation) but differ in which GPU APIs they target on the device.