Class Fragment::GPUResidentAccessor
Defined in File fragment.hpp
This class is a nested type of Class Fragment.
-
class GPUResidentAccessor
Accessor class for GPU-resident specific functions of a Fragment.
This class provides a convenient interface for accessing GPU-resident specific functionality of a Fragment. It acts as a mediator to expose GPU-resident operations through a cleaner API pattern:
fragment->gpu_resident().function().This is a lightweight accessor class that maintains a reference to the parent Fragment.
Public Functions
-
GPUResidentAccessor() = delete
-
inline explicit GPUResidentAccessor(Fragment *fragment)
Construct a new GPUResidentAccessor object.
- Parameters
fragment – Pointer to the parent Fragment
-
void timeout_ms(unsigned long long timeout_ms)
Set the timeout for GPU-resident execution.
GPU-resident execution occurs asynchronously. This sets the timeout so that execution is stopped after it exceeds the specified duration.
- Parameters
timeout_ms – The timeout in milliseconds.
-
void tear_down()
Send a tear down signal to the GPU-resident CUDA graph.
The timeout has to be set to zero for this to work for now.
-
bool result_ready()
Check if the result of a single iteration of the GPU-resident CUDA graph is ready.
- Returns
true if the result is ready, false otherwise.
-
void data_ready()
Inform the GPU-resident CUDA graph that the data is ready for the main workload.
-
bool is_launched()
Check if the GPU-resident CUDA graph has been launched.
- Returns
true if the CUDA graph has been launched, false otherwise.
-
cudaGraph_t workload_graph()
Get the CUDA graph of the main workload in this fragment. This returns a clone of the main workload graph, and certain CUDA graph nodes (e.g. memory allocation, memory free, conditional nodes) are not supported for cloning.
- Returns
A clone of the CUDA graph of the main workload in this fragment.
-
void *data_ready_device_address()
Get the CUDA device pointer for the data_ready signal.
This returns the actual device memory address that the GPU-resident CUDA graph uses to check if data is ready for processing. Can be used for advanced GPU-resident applications that need direct access to these control signals.
- Returns
Pointer to the device memory location for data_ready signal.
-
void *result_ready_device_address()
Get the CUDA device pointer for the result_ready signal.
Similar to data_ready_device_address(), but for the result_ready signal.
- Returns
Pointer to the device memory location for result_ready signal.
-
void *tear_down_device_address()
Get the CUDA device pointer for the tear_down signal.
Similar to data_ready_device_address(), but for the tear_down signal.
- Returns
Pointer to the device memory location for tear_down signal.
Register a data ready handler to this fragment. The data ready handler will be executed at the beginning of every iteration of the GPU-resident CUDA Graph. The data ready handler will usually indicate whether input data is ready for processing. If the data ready handler marks data to be ready, then main workload CUDA graph will be executed on this iteration, otherwise main workload processing will be skipped for this iteration.
- Parameters
data_ready_handler_fragment – Shared pointer to a fragment that will be added as the data ready handler to this fragmemt.
-
std::shared_ptr<Fragment> data_ready_handler_fragment()
Get the registered data ready handler fragment.
- Returns
The data ready handler fragment, or nullptr if none is registered.
-
GPUResidentAccessor() = delete