Class GPUResidentDeck

GPU-resident deck is the CPU-side software component that is responsible for communication with the asynchronously running GPU-resident CUDA workload.

Public Functions

GPUResidentDeck()

~GPUResidentDeck()

std::future<void> launch_cuda_graph(cudaGraphExec_t graph)

This function launches an executable CUDA graph asynchronously.

Parameters

graph – The executable CUDA graph to launch asynchronously.

Returns

std::future<void> A future that will be set when the CUDA graph has finished executing.

inline void *data_ready_device_address()

inline void *result_ready_device_address()

inline void *tear_down_device_address()

inline void timeout_ms(unsigned long long timeout_ms)

Sets the timeout for the GPU-resident CUDA graph execution. If timeout is zero, then the asynchronous execution will wait until an external tear down is triggered.

Parameters

timeout_ms – The timeout in milliseconds.

bool result_ready()

Indicates whether the result of a single iteration of the GPU-resident CUDA graph is ready or not.

Returns

true if the result is ready, false otherwise.

void tear_down()

Sends a tear down signal to the GPU-resident CUDA graph. The timeout has to be set to zero for this to work for now. In the future, we will support ignoring the timeout when tear down is triggered.

void set_data_ready()

This function informs GPU-resident CUDA graph that the data is ready for the main workload.

bool is_launched() const

Indicates whether the GPU-resident CUDA graph has been launched.

Returns

true if the CUDA graph has been launched, false otherwise.