CUDA Driver API :: CUDA Toolkit Documentation

6.21. Graph Management

This section describes the graph management functions of the low-level CUDA driver application programming interface.

Functions

CUresult cuDeviceGetGraphMemAttribute ( CUdevice device, CUgraphMem_attribute attr, void* value ): Query asynchronous allocation attributes related to graphs.
CUresult cuDeviceGraphMemTrim ( CUdevice device ): Free unused memory that was cached on the specified device for use with graphs back to the OS.
CUresult cuDeviceSetGraphMemAttribute ( CUdevice device, CUgraphMem_attribute attr, void* value ): Set asynchronous allocation attributes related to graphs.
CUresult cuGraphAddChildGraphNode ( CUgraphNode* phGraphNode, CUgraph hGraph, const CUgraphNode* dependencies, size_t numDependencies, CUgraph childGraph ): Creates a child graph node and adds it to a graph.
CUresult cuGraphAddDependencies ( CUgraph hGraph, const CUgraphNode* from, const CUgraphNode* to, size_t numDependencies ): Adds dependency edges to a graph.
CUresult cuGraphAddEmptyNode ( CUgraphNode* phGraphNode, CUgraph hGraph, const CUgraphNode* dependencies, size_t numDependencies ): Creates an empty node and adds it to a graph.
CUresult cuGraphAddEventRecordNode ( CUgraphNode* phGraphNode, CUgraph hGraph, const CUgraphNode* dependencies, size_t numDependencies, CUevent event ): Creates an event record node and adds it to a graph.
CUresult cuGraphAddEventWaitNode ( CUgraphNode* phGraphNode, CUgraph hGraph, const CUgraphNode* dependencies, size_t numDependencies, CUevent event ): Creates an event wait node and adds it to a graph.
CUresult cuGraphAddExternalSemaphoresSignalNode ( CUgraphNode* phGraphNode, CUgraph hGraph, const CUgraphNode* dependencies, size_t numDependencies, const CUDA_EXT_SEM_SIGNAL_NODE_PARAMS* nodeParams ): Creates an external semaphore signal node and adds it to a graph.
CUresult cuGraphAddExternalSemaphoresWaitNode ( CUgraphNode* phGraphNode, CUgraph hGraph, const CUgraphNode* dependencies, size_t numDependencies, const CUDA_EXT_SEM_WAIT_NODE_PARAMS* nodeParams ): Creates an external semaphore wait node and adds it to a graph.
CUresult cuGraphAddHostNode ( CUgraphNode* phGraphNode, CUgraph hGraph, const CUgraphNode* dependencies, size_t numDependencies, const CUDA_HOST_NODE_PARAMS* nodeParams ): Creates a host execution node and adds it to a graph.
CUresult cuGraphAddKernelNode ( CUgraphNode* phGraphNode, CUgraph hGraph, const CUgraphNode* dependencies, size_t numDependencies, const CUDA_KERNEL_NODE_PARAMS* nodeParams ): Creates a kernel execution node and adds it to a graph.
CUresult cuGraphAddMemAllocNode ( CUgraphNode* phGraphNode, CUgraph hGraph, const CUgraphNode* dependencies, size_t numDependencies, CUDA_MEM_ALLOC_NODE_PARAMS* nodeParams ): Creates an allocation node and adds it to a graph.
CUresult cuGraphAddMemFreeNode ( CUgraphNode* phGraphNode, CUgraph hGraph, const CUgraphNode* dependencies, size_t numDependencies, CUdeviceptr dptr ): Creates a memory free node and adds it to a graph.
CUresult cuGraphAddMemcpyNode ( CUgraphNode* phGraphNode, CUgraph hGraph, const CUgraphNode* dependencies, size_t numDependencies, const CUDA_MEMCPY3D* copyParams, CUcontext ctx ): Creates a memcpy node and adds it to a graph.
CUresult cuGraphAddMemsetNode ( CUgraphNode* phGraphNode, CUgraph hGraph, const CUgraphNode* dependencies, size_t numDependencies, const CUDA_MEMSET_NODE_PARAMS* memsetParams, CUcontext ctx ): Creates a memset node and adds it to a graph.
CUresult cuGraphChildGraphNodeGetGraph ( CUgraphNode hNode, CUgraph* phGraph ): Gets a handle to the embedded graph of a child graph node.
CUresult cuGraphClone ( CUgraph* phGraphClone, CUgraph originalGraph ): Clones a graph.
CUresult cuGraphCreate ( CUgraph* phGraph, unsigned int flags ): Creates a graph.
CUresult cuGraphDebugDotPrint ( CUgraph hGraph, const char* path, unsigned int flags ): Write a DOT file describing graph structure.
CUresult cuGraphDestroy ( CUgraph hGraph ): Destroys a graph.
CUresult cuGraphDestroyNode ( CUgraphNode hNode ): Remove a node from the graph.
CUresult cuGraphEventRecordNodeGetEvent ( CUgraphNode hNode, CUevent* event_out ): Returns the event associated with an event record node.
CUresult cuGraphEventRecordNodeSetEvent ( CUgraphNode hNode, CUevent event ): Sets an event record node's event.
CUresult cuGraphEventWaitNodeGetEvent ( CUgraphNode hNode, CUevent* event_out ): Returns the event associated with an event wait node.
CUresult cuGraphEventWaitNodeSetEvent ( CUgraphNode hNode, CUevent event ): Sets an event wait node's event.
CUresult cuGraphExecChildGraphNodeSetParams ( CUgraphExec hGraphExec, CUgraphNode hNode, CUgraph childGraph ): Updates node parameters in the child graph node in the given graphExec.
CUresult cuGraphExecDestroy ( CUgraphExec hGraphExec ): Destroys an executable graph.
CUresult cuGraphExecEventRecordNodeSetEvent ( CUgraphExec hGraphExec, CUgraphNode hNode, CUevent event ): Sets the event for an event record node in the given graphExec.
CUresult cuGraphExecEventWaitNodeSetEvent ( CUgraphExec hGraphExec, CUgraphNode hNode, CUevent event ): Sets the event for an event wait node in the given graphExec.
CUresult cuGraphExecExternalSemaphoresSignalNodeSetParams ( CUgraphExec hGraphExec, CUgraphNode hNode, const CUDA_EXT_SEM_SIGNAL_NODE_PARAMS* nodeParams ): Sets the parameters for an external semaphore signal node in the given graphExec.
CUresult cuGraphExecExternalSemaphoresWaitNodeSetParams ( CUgraphExec hGraphExec, CUgraphNode hNode, const CUDA_EXT_SEM_WAIT_NODE_PARAMS* nodeParams ): Sets the parameters for an external semaphore wait node in the given graphExec.
CUresult cuGraphExecHostNodeSetParams ( CUgraphExec hGraphExec, CUgraphNode hNode, const CUDA_HOST_NODE_PARAMS* nodeParams ): Sets the parameters for a host node in the given graphExec.
CUresult cuGraphExecKernelNodeSetParams ( CUgraphExec hGraphExec, CUgraphNode hNode, const CUDA_KERNEL_NODE_PARAMS* nodeParams ): Sets the parameters for a kernel node in the given graphExec.
CUresult cuGraphExecMemcpyNodeSetParams ( CUgraphExec hGraphExec, CUgraphNode hNode, const CUDA_MEMCPY3D* copyParams, CUcontext ctx ): Sets the parameters for a memcpy node in the given graphExec.
CUresult cuGraphExecMemsetNodeSetParams ( CUgraphExec hGraphExec, CUgraphNode hNode, const CUDA_MEMSET_NODE_PARAMS* memsetParams, CUcontext ctx ): Sets the parameters for a memset node in the given graphExec.
CUresult cuGraphExecUpdate ( CUgraphExec hGraphExec, CUgraph hGraph, CUgraphNode* hErrorNode_out, CUgraphExecUpdateResult* updateResult_out ): Check whether an executable graph can be updated with a graph and perform the update if possible.
CUresult cuGraphExternalSemaphoresSignalNodeGetParams ( CUgraphNode hNode, CUDA_EXT_SEM_SIGNAL_NODE_PARAMS* params_out ): Returns an external semaphore signal node's parameters.
CUresult cuGraphExternalSemaphoresSignalNodeSetParams ( CUgraphNode hNode, const CUDA_EXT_SEM_SIGNAL_NODE_PARAMS* nodeParams ): Sets an external semaphore signal node's parameters.
CUresult cuGraphExternalSemaphoresWaitNodeGetParams ( CUgraphNode hNode, CUDA_EXT_SEM_WAIT_NODE_PARAMS* params_out ): Returns an external semaphore wait node's parameters.
CUresult cuGraphExternalSemaphoresWaitNodeSetParams ( CUgraphNode hNode, const CUDA_EXT_SEM_WAIT_NODE_PARAMS* nodeParams ): Sets an external semaphore wait node's parameters.
CUresult cuGraphGetEdges ( CUgraph hGraph, CUgraphNode* from, CUgraphNode* to, size_t* numEdges ): Returns a graph's dependency edges.
CUresult cuGraphGetNodes ( CUgraph hGraph, CUgraphNode* nodes, size_t* numNodes ): Returns a graph's nodes.
CUresult cuGraphGetRootNodes ( CUgraph hGraph, CUgraphNode* rootNodes, size_t* numRootNodes ): Returns a graph's root nodes.
CUresult cuGraphHostNodeGetParams ( CUgraphNode hNode, CUDA_HOST_NODE_PARAMS* nodeParams ): Returns a host node's parameters.
CUresult cuGraphHostNodeSetParams ( CUgraphNode hNode, const CUDA_HOST_NODE_PARAMS* nodeParams ): Sets a host node's parameters.
CUresult cuGraphInstantiate ( CUgraphExec* phGraphExec, CUgraph hGraph, CUgraphNode* phErrorNode, char* logBuffer, size_t bufferSize ): Creates an executable graph from a graph.
CUresult cuGraphInstantiateWithFlags ( CUgraphExec* phGraphExec, CUgraph hGraph, unsigned long long flags ): Creates an executable graph from a graph.
CUresult cuGraphKernelNodeCopyAttributes ( CUgraphNode dst, CUgraphNode src ): Copies attributes from source node to destination node.
CUresult cuGraphKernelNodeGetAttribute ( CUgraphNode hNode, CUkernelNodeAttrID attr, CUkernelNodeAttrValue* value_out ): Queries node attribute.
CUresult cuGraphKernelNodeGetParams ( CUgraphNode hNode, CUDA_KERNEL_NODE_PARAMS* nodeParams ): Returns a kernel node's parameters.
CUresult cuGraphKernelNodeSetAttribute ( CUgraphNode hNode, CUkernelNodeAttrID attr, const CUkernelNodeAttrValue* value ): Sets node attribute.
CUresult cuGraphKernelNodeSetParams ( CUgraphNode hNode, const CUDA_KERNEL_NODE_PARAMS* nodeParams ): Sets a kernel node's parameters.
CUresult cuGraphLaunch ( CUgraphExec hGraphExec, CUstream hStream ): Launches an executable graph in a stream.
CUresult cuGraphMemAllocNodeGetParams ( CUgraphNode hNode, CUDA_MEM_ALLOC_NODE_PARAMS* params_out ): Returns a memory alloc node's parameters.
CUresult cuGraphMemFreeNodeGetParams ( CUgraphNode hNode, CUdeviceptr* dptr_out ): Returns a memory free node's parameters.
CUresult cuGraphMemcpyNodeGetParams ( CUgraphNode hNode, CUDA_MEMCPY3D* nodeParams ): Returns a memcpy node's parameters.
CUresult cuGraphMemcpyNodeSetParams ( CUgraphNode hNode, const CUDA_MEMCPY3D* nodeParams ): Sets a memcpy node's parameters.
CUresult cuGraphMemsetNodeGetParams ( CUgraphNode hNode, CUDA_MEMSET_NODE_PARAMS* nodeParams ): Returns a memset node's parameters.
CUresult cuGraphMemsetNodeSetParams ( CUgraphNode hNode, const CUDA_MEMSET_NODE_PARAMS* nodeParams ): Sets a memset node's parameters.
CUresult cuGraphNodeFindInClone ( CUgraphNode* phNode, CUgraphNode hOriginalNode, CUgraph hClonedGraph ): Finds a cloned version of a node.
CUresult cuGraphNodeGetDependencies ( CUgraphNode hNode, CUgraphNode* dependencies, size_t* numDependencies ): Returns a node's dependencies.
CUresult cuGraphNodeGetDependentNodes ( CUgraphNode hNode, CUgraphNode* dependentNodes, size_t* numDependentNodes ): Returns a node's dependent nodes.
CUresult cuGraphNodeGetType ( CUgraphNode hNode, CUgraphNodeType* type ): Returns a node's type.
CUresult cuGraphReleaseUserObject ( CUgraph graph, CUuserObject object, unsigned int count ): Release a user object reference from a graph.
CUresult cuGraphRemoveDependencies ( CUgraph hGraph, const CUgraphNode* from, const CUgraphNode* to, size_t numDependencies ): Removes dependency edges from a graph.
CUresult cuGraphRetainUserObject ( CUgraph graph, CUuserObject object, unsigned int count, unsigned int flags ): Retain a reference to a user object from a graph.
CUresult cuGraphUpload ( CUgraphExec hGraphExec, CUstream hStream ): Uploads an executable graph in a stream.
CUresult cuUserObjectCreate ( CUuserObject* object_out, void* ptr, CUhostFn destroy, unsigned int initialRefcount, unsigned int flags ): Create a user object.
CUresult cuUserObjectRelease ( CUuserObject object, unsigned int count ): Release a reference to a user object.
CUresult cuUserObjectRetain ( CUuserObject object, unsigned int count ): Retain a reference to a user object.

Functions

CUresult cuDeviceGetGraphMemAttribute ( CUdevice device, CUgraphMem_attribute attr, void* value )

Query asynchronous allocation attributes related to graphs.

Parameters

device: - Specifies the scope of the query
attr: - attribute to get
value: - retrieved value

Returns

CUDA_SUCCESS, CUDA_ERROR_INVALID_DEVICE

Description

Valid attributes are:

CU_GRAPH_MEM_ATTR_USED_MEM_CURRENT: Amount of memory, in bytes, currently associated with graphs
CU_GRAPH_MEM_ATTR_USED_MEM_HIGH: High watermark of memory, in bytes, associated with graphs since the last time it was reset. High watermark can only be reset to zero.
CU_GRAPH_MEM_ATTR_RESERVED_MEM_CURRENT: Amount of memory, in bytes, currently allocated for use by the CUDA graphs asynchronous allocator.
CU_GRAPH_MEM_ATTR_RESERVED_MEM_HIGH: High watermark of memory, in bytes, currently allocated for use by the CUDA graphs asynchronous allocator.

CUresult cuDeviceGraphMemTrim ( CUdevice device )

Free unused memory that was cached on the specified device for use with graphs back to the OS.

Parameters

device: - The device for which cached memory should be freed.

Returns

CUDA_SUCCESS, CUDA_ERROR_INVALID_DEVICE

Description

Blocks which are not in use by a graph that is either currently executing or scheduled to execute are freed back to the operating system.

CUresult cuDeviceSetGraphMemAttribute ( CUdevice device, CUgraphMem_attribute attr, void* value )

Set asynchronous allocation attributes related to graphs.

Parameters

device: - Specifies the scope of the query
attr: - attribute to get
value: - pointer to value to set

Returns

CUDA_SUCCESS, CUDA_ERROR_INVALID_DEVICE

Description

Valid attributes are:

CU_GRAPH_MEM_ATTR_USED_MEM_HIGH: High watermark of memory, in bytes, associated with graphs since the last time it was reset. High watermark can only be reset to zero.
CU_GRAPH_MEM_ATTR_RESERVED_MEM_HIGH: High watermark of memory, in bytes, currently allocated for use by the CUDA graphs asynchronous allocator.

CUresult cuGraphAddChildGraphNode ( CUgraphNode* phGraphNode, CUgraph hGraph, const CUgraphNode* dependencies, size_t numDependencies, CUgraph childGraph )

Creates a child graph node and adds it to a graph.

Parameters

phGraphNode: - Returns newly created node
hGraph: - Graph to which to add the node
dependencies: - Dependencies of the node
numDependencies: - Number of dependencies
childGraph: - The graph to clone into this node

Returns

CUDA_SUCCESS, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_INITIALIZED, CUDA_ERROR_INVALID_VALUE,

Description

Creates a new node which executes an embedded graph, and adds it to hGraph with numDependencies dependencies specified via dependencies. It is possible for numDependencies to be 0, in which case the node will be placed at the root of the graph. dependencies may not have any duplicate entries. A handle to the new node will be returned in phGraphNode.

If hGraph contains allocation or free nodes, this call will return an error.

The node executes an embedded child graph. The child graph is cloned in this call.

Note:

Graph objects are not threadsafe. More here.
Note that this function may also return error codes from previous, asynchronous launches.

CUresult cuGraphAddDependencies ( CUgraph hGraph, const CUgraphNode* from, const CUgraphNode* to, size_t numDependencies )

Adds dependency edges to a graph.

Parameters

hGraph: - Graph to which dependencies are added
from: - Array of nodes that provide the dependencies
to: - Array of dependent nodes
numDependencies: - Number of dependencies to be added

Returns

CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE

Description

The number of dependencies to be added is defined by numDependencies Elements in from and to at corresponding indices define a dependency. Each node in from and to must belong to hGraph.

If numDependencies is 0, elements in from and to will be ignored. Specifying an existing dependency will return an error.

Note:

Graph objects are not threadsafe. More here.
Note that this function may also return error codes from previous, asynchronous launches.

CUresult cuGraphAddEmptyNode ( CUgraphNode* phGraphNode, CUgraph hGraph, const CUgraphNode* dependencies, size_t numDependencies )

Creates an empty node and adds it to a graph.

Parameters

phGraphNode: - Returns newly created node
hGraph: - Graph to which to add the node
dependencies: - Dependencies of the node
numDependencies: - Number of dependencies

Returns

CUDA_SUCCESS, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_INITIALIZED, CUDA_ERROR_INVALID_VALUE,

Description

Creates a new node which performs no operation, and adds it to hGraph with numDependencies dependencies specified via dependencies. It is possible for numDependencies to be 0, in which case the node will be placed at the root of the graph. dependencies may not have any duplicate entries. A handle to the new node will be returned in phGraphNode.

An empty node performs no operation during execution, but can be used for transitive ordering. For example, a phased execution graph with 2 groups of n nodes with a barrier between them can be represented using an empty node and 2*n dependency edges, rather than no empty node and n^2 dependency edges.

Note:

Graph objects are not threadsafe. More here.
Note that this function may also return error codes from previous, asynchronous launches.

CUresult cuGraphAddEventRecordNode ( CUgraphNode* phGraphNode, CUgraph hGraph, const CUgraphNode* dependencies, size_t numDependencies, CUevent event )

Creates an event record node and adds it to a graph.

Parameters

phGraphNode: - Returns newly created node
hGraph: - Graph to which to add the node
dependencies: - Dependencies of the node
numDependencies: - Number of dependencies
event: - Event for the node

Returns

CUDA_SUCCESS, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_INITIALIZED, CUDA_ERROR_NOT_SUPPORTED, CUDA_ERROR_INVALID_VALUE

Description

Creates a new event record node and adds it to hGraph with numDependencies dependencies specified via dependencies and event specified in event. It is possible for numDependencies to be 0, in which case the node will be placed at the root of the graph. dependencies may not have any duplicate entries. A handle to the new node will be returned in phGraphNode.

Each launch of the graph will record event to capture execution of the node's dependencies.

Note:

Graph objects are not threadsafe. More here.
Note that this function may also return error codes from previous, asynchronous launches.

CUresult cuGraphAddEventWaitNode ( CUgraphNode* phGraphNode, CUgraph hGraph, const CUgraphNode* dependencies, size_t numDependencies, CUevent event )

Creates an event wait node and adds it to a graph.

Parameters

phGraphNode: - Returns newly created node
hGraph: - Graph to which to add the node
dependencies: - Dependencies of the node
numDependencies: - Number of dependencies
event: - Event for the node

Returns

CUDA_SUCCESS, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_INITIALIZED, CUDA_ERROR_NOT_SUPPORTED, CUDA_ERROR_INVALID_VALUE

Description

Creates a new event wait node and adds it to hGraph with numDependencies dependencies specified via dependencies and event specified in event. It is possible for numDependencies to be 0, in which case the node will be placed at the root of the graph. dependencies may not have any duplicate entries. A handle to the new node will be returned in phGraphNode.

The graph node will wait for all work captured in event. See cuEventRecord() for details on what is captured by an event. event may be from a different context or device than the launch stream.

Note:

Graph objects are not threadsafe. More here.
Note that this function may also return error codes from previous, asynchronous launches.

CUresult cuGraphAddExternalSemaphoresSignalNode ( CUgraphNode* phGraphNode, CUgraph hGraph, const CUgraphNode* dependencies, size_t numDependencies, const CUDA_EXT_SEM_SIGNAL_NODE_PARAMS* nodeParams )

Creates an external semaphore signal node and adds it to a graph.

Parameters

phGraphNode: - Returns newly created node
hGraph: - Graph to which to add the node
dependencies: - Dependencies of the node
numDependencies: - Number of dependencies
nodeParams: - Parameters for the node

Returns

CUDA_SUCCESS, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_INITIALIZED, CUDA_ERROR_NOT_SUPPORTED, CUDA_ERROR_INVALID_VALUE

Description

Creates a new external semaphore signal node and adds it to hGraph with numDependencies dependencies specified via dependencies and arguments specified in nodeParams. It is possible for numDependencies to be 0, in which case the node will be placed at the root of the graph. dependencies may not have any duplicate entries. A handle to the new node will be returned in phGraphNode.

Performs a signal operation on a set of externally allocated semaphore objects when the node is launched. The operation(s) will occur after all of the node's dependencies have completed.

Note:

Graph objects are not threadsafe. More here.
Note that this function may also return error codes from previous, asynchronous launches.

CUresult cuGraphAddExternalSemaphoresWaitNode ( CUgraphNode* phGraphNode, CUgraph hGraph, const CUgraphNode* dependencies, size_t numDependencies, const CUDA_EXT_SEM_WAIT_NODE_PARAMS* nodeParams )

Creates an external semaphore wait node and adds it to a graph.

Parameters

phGraphNode: - Returns newly created node
hGraph: - Graph to which to add the node
dependencies: - Dependencies of the node
numDependencies: - Number of dependencies
nodeParams: - Parameters for the node

Returns

CUDA_SUCCESS, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_INITIALIZED, CUDA_ERROR_NOT_SUPPORTED, CUDA_ERROR_INVALID_VALUE

Description

Creates a new external semaphore wait node and adds it to hGraph with numDependencies dependencies specified via dependencies and arguments specified in nodeParams. It is possible for numDependencies to be 0, in which case the node will be placed at the root of the graph. dependencies may not have any duplicate entries. A handle to the new node will be returned in phGraphNode.

Performs a wait operation on a set of externally allocated semaphore objects when the node is launched. The node's dependencies will not be launched until the wait operation has completed.

Note:

Graph objects are not threadsafe. More here.
Note that this function may also return error codes from previous, asynchronous launches.

CUresult cuGraphAddHostNode ( CUgraphNode* phGraphNode, CUgraph hGraph, const CUgraphNode* dependencies, size_t numDependencies, const CUDA_HOST_NODE_PARAMS* nodeParams )

Creates a host execution node and adds it to a graph.

Parameters

phGraphNode: - Returns newly created node
hGraph: - Graph to which to add the node
dependencies: - Dependencies of the node
numDependencies: - Number of dependencies
nodeParams: - Parameters for the host node

Returns

CUDA_SUCCESS, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_INITIALIZED, CUDA_ERROR_NOT_SUPPORTED, CUDA_ERROR_INVALID_VALUE

Description

Creates a new CPU execution node and adds it to hGraph with numDependencies dependencies specified via dependencies and arguments specified in nodeParams. It is possible for numDependencies to be 0, in which case the node will be placed at the root of the graph. dependencies may not have any duplicate entries. A handle to the new node will be returned in phGraphNode.

When the graph is launched, the node will invoke the specified CPU function. Host nodes are not supported under MPS with pre-Volta GPUs.

Note:

Graph objects are not threadsafe. More here.
Note that this function may also return error codes from previous, asynchronous launches.

CUresult cuGraphAddKernelNode ( CUgraphNode* phGraphNode, CUgraph hGraph, const CUgraphNode* dependencies, size_t numDependencies, const CUDA_KERNEL_NODE_PARAMS* nodeParams )

Creates a kernel execution node and adds it to a graph.

Parameters

phGraphNode: - Returns newly created node
hGraph: - Graph to which to add the node
dependencies: - Dependencies of the node
numDependencies: - Number of dependencies
nodeParams: - Parameters for the GPU execution node

Returns

CUDA_SUCCESS, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_INITIALIZED, CUDA_ERROR_INVALID_VALUE

Description

Creates a new kernel execution node and adds it to hGraph with numDependencies dependencies specified via dependencies and arguments specified in nodeParams. It is possible for numDependencies to be 0, in which case the node will be placed at the root of the graph. dependencies may not have any duplicate entries. A handle to the new node will be returned in phGraphNode.

The CUDA_KERNEL_NODE_PARAMS structure is defined as:

‎  typedef struct CUDA_KERNEL_NODE_PARAMS_st {
            CUfunction func;
            unsigned int gridDimX;
            unsigned int gridDimY;
            unsigned int gridDimZ;
            unsigned int blockDimX;
            unsigned int blockDimY;
            unsigned int blockDimZ;
            unsigned int sharedMemBytes;
            void **kernelParams;
            void **extra;
        } CUDA_KERNEL_NODE_PARAMS;

When the graph is launched, the node will invoke kernel func on a (gridDimX x gridDimY x gridDimZ) grid of blocks. Each block contains (blockDimX x blockDimY x blockDimZ) threads.

sharedMemBytes sets the amount of dynamic shared memory that will be available to each thread block.

Kernel parameters to func can be specified in one of two ways:

1) Kernel parameters can be specified via kernelParams. If the kernel has N parameters, then kernelParams needs to be an array of N pointers. Each pointer, from kernelParams[0] to kernelParams[N-1], points to the region of memory from which the actual parameter will be copied. The number of kernel parameters and their offsets and sizes do not need to be specified as that information is retrieved directly from the kernel's image.

2) Kernel parameters for non-cooperative kernels can also be packaged by the application into a single buffer that is passed in via extra. This places the burden on the application of knowing each kernel parameter's size and alignment/padding within the buffer. The extra parameter exists to allow this function to take additional less commonly used arguments. extra specifies a list of names of extra settings and their corresponding values. Each extra setting name is immediately followed by the corresponding value. The list must be terminated with either NULL or CU_LAUNCH_PARAM_END.

CU_LAUNCH_PARAM_END, which indicates the end of the extra array;
CU_LAUNCH_PARAM_BUFFER_POINTER, which specifies that the next value in extra will be a pointer to a buffer containing all the kernel parameters for launching kernel func;
CU_LAUNCH_PARAM_BUFFER_SIZE, which specifies that the next value in extra will be a pointer to a size_t containing the size of the buffer specified with CU_LAUNCH_PARAM_BUFFER_POINTER;

The error CUDA_ERROR_INVALID_VALUE will be returned if kernel parameters are specified with both kernelParams and extra (i.e. both kernelParams and extra are non-NULL). CUDA_ERROR_INVALID_VALUE will be returned if extra is used for a cooperative kernel.

The kernelParams or extra array, as well as the argument values it points to, are copied during this call.

Note:

Kernels launched using graphs must not use texture and surface references. Reading or writing through any texture or surface reference is undefined behavior. This restriction does not apply to texture and surface objects.

Note:

Graph objects are not threadsafe. More here.
Note that this function may also return error codes from previous, asynchronous launches.

CUresult cuGraphAddMemAllocNode ( CUgraphNode* phGraphNode, CUgraph hGraph, const CUgraphNode* dependencies, size_t numDependencies, CUDA_MEM_ALLOC_NODE_PARAMS* nodeParams )

Creates an allocation node and adds it to a graph.

Parameters

phGraphNode: - Returns newly created node
hGraph: - Graph to which to add the node
dependencies: - Dependencies of the node
numDependencies: - Number of dependencies
nodeParams: - Parameters for the node

Returns

CUDA_SUCCESS, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_INITIALIZED, CUDA_ERROR_NOT_SUPPORTED, CUDA_ERROR_INVALID_VALUE

Description

Creates a new allocation node and adds it to hGraph with numDependencies dependencies specified via dependencies and arguments specified in nodeParams. It is possible for numDependencies to be 0, in which case the node will be placed at the root of the graph. dependencies may not have any duplicate entries. A handle to the new node will be returned in phGraphNode.

When cuGraphAddMemAllocNode creates an allocation node, it returns the address of the allocation in If the allocation is freed in the same graph, by creating a free node using cuGraphAddMemFreeNode, the allocation can be accessed by nodes ordered after the allocation node but before the free node. These allocations cannot be freed outside the owning graph, and they can only be freed once in the owning graph.

If the allocation is not freed in the same graph, then it can be accessed not only by nodes in the graph which are ordered after the allocation node, but also by stream operations ordered after the graph's execution but before the allocation is freed.

Allocations which are not freed in the same graph can be freed by:

passing the allocation to cuMemFreeAsync or cuMemFree;
launching a graph with a free node for that allocation; or
specifying CUDA_GRAPH_INSTANTIATE_FLAG_AUTO_FREE_ON_LAUNCH during instantiation, which makes each launch behave as though it called cuMemFreeAsync for every unfreed allocation.

It is not possible to free an allocation in both the owning graph and another graph. If the allocation is freed in the same graph, a free node cannot be added to another graph. If the allocation is freed in another graph, a free node can no longer be added to the owning graph.

The following restrictions apply to graphs which contain allocation and/or memory free nodes:

Nodes and edges of the graph cannot be deleted.
The graph cannot be used in a child node.
Only one instantiation of the graph may exist at any point in time.
The graph cannot be cloned.

Note:

Graph objects are not threadsafe. More here.
Note that this function may also return error codes from previous, asynchronous launches.

CUresult cuGraphAddMemFreeNode ( CUgraphNode* phGraphNode, CUgraph hGraph, const CUgraphNode* dependencies, size_t numDependencies, CUdeviceptr dptr )

Creates a memory free node and adds it to a graph.

Parameters

phGraphNode: - Returns newly created node
hGraph: - Graph to which to add the node
dependencies: - Dependencies of the node
numDependencies: - Number of dependencies
dptr: - Address of memory to free

Returns

CUDA_SUCCESS, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_INITIALIZED, CUDA_ERROR_NOT_SUPPORTED, CUDA_ERROR_INVALID_VALUE

Description

Creates a new memory free node and adds it to hGraph with numDependencies dependencies specified via dependencies and arguments specified in nodeParams. It is possible for numDependencies to be 0, in which case the node will be placed at the root of the graph. dependencies may not have any duplicate entries. A handle to the new node will be returned in phGraphNode.

cuGraphAddMemFreeNode will return CUDA_ERROR_INVALID_VALUE if the user attempts to free:

an allocation twice in the same graph.
an address that was not returned by an allocation node.
an invalid address.

The following restrictions apply to graphs which contain allocation and/or memory free nodes:

Nodes and edges of the graph cannot be deleted.
The graph cannot be used in a child node.
Only one instantiation of the graph may exist at any point in time.
The graph cannot be cloned.

Note:

Graph objects are not threadsafe. More here.
Note that this function may also return error codes from previous, asynchronous launches.

CUresult cuGraphAddMemcpyNode ( CUgraphNode* phGraphNode, CUgraph hGraph, const CUgraphNode* dependencies, size_t numDependencies, const CUDA_MEMCPY3D* copyParams, CUcontext ctx )

Creates a memcpy node and adds it to a graph.

Parameters

phGraphNode: - Returns newly created node
hGraph: - Graph to which to add the node
dependencies: - Dependencies of the node
numDependencies: - Number of dependencies
copyParams: - Parameters for the memory copy
ctx: - Context on which to run the node

Returns

CUDA_SUCCESS, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_INITIALIZED, CUDA_ERROR_INVALID_VALUE

Description

Creates a new memcpy node and adds it to hGraph with numDependencies dependencies specified via dependencies. It is possible for numDependencies to be 0, in which case the node will be placed at the root of the graph. dependencies may not have any duplicate entries. A handle to the new node will be returned in phGraphNode.

When the graph is launched, the node will perform the memcpy described by copyParams. See cuMemcpy3D() for a description of the structure and its restrictions.

Memcpy nodes have some additional restrictions with regards to managed memory, if the system contains at least one device which has a zero value for the device attribute CU_DEVICE_ATTRIBUTE_CONCURRENT_MANAGED_ACCESS. If one or more of the operands refer to managed memory, then using the memory type CU_MEMORYTYPE_UNIFIED is disallowed for those operand(s). The managed memory will be treated as residing on either the host or the device, depending on which memory type is specified.

Note:

Graph objects are not threadsafe. More here.
Note that this function may also return error codes from previous, asynchronous launches.

CUresult cuGraphAddMemsetNode ( CUgraphNode* phGraphNode, CUgraph hGraph, const CUgraphNode* dependencies, size_t numDependencies, const CUDA_MEMSET_NODE_PARAMS* memsetParams, CUcontext ctx )

Creates a memset node and adds it to a graph.

Parameters

phGraphNode: - Returns newly created node
hGraph: - Graph to which to add the node
dependencies: - Dependencies of the node
numDependencies: - Number of dependencies
memsetParams: - Parameters for the memory set
ctx: - Context on which to run the node

Returns

CUDA_SUCCESS, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_INITIALIZED, CUDA_ERROR_INVALID_VALUE, CUDA_ERROR_INVALID_CONTEXT

Description

Creates a new memset node and adds it to hGraph with numDependencies dependencies specified via dependencies. It is possible for numDependencies to be 0, in which case the node will be placed at the root of the graph. dependencies may not have any duplicate entries. A handle to the new node will be returned in phGraphNode.

The element size must be 1, 2, or 4 bytes. When the graph is launched, the node will perform the memset described by memsetParams.

Note:

Graph objects are not threadsafe. More here.
Note that this function may also return error codes from previous, asynchronous launches.

CUresult cuGraphChildGraphNodeGetGraph ( CUgraphNode hNode, CUgraph* phGraph )

Gets a handle to the embedded graph of a child graph node.

Parameters

hNode: - Node to get the embedded graph for
phGraph: - Location to store a handle to the graph

Returns

CUDA_SUCCESS, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_INITIALIZED, CUDA_ERROR_INVALID_VALUE,

Description

Gets a handle to the embedded graph in a child graph node. This call does not clone the graph. Changes to the graph will be reflected in the node, and the node retains ownership of the graph.

Allocation and free nodes cannot be added to the returned graph. Attempting to do so will return an error.

Note:

Graph objects are not threadsafe. More here.
Note that this function may also return error codes from previous, asynchronous launches.

CUresult cuGraphClone ( CUgraph* phGraphClone, CUgraph originalGraph )

Clones a graph.

Parameters

phGraphClone: - Returns newly created cloned graph
originalGraph: - Graph to clone

Returns

CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE, CUDA_ERROR_OUT_OF_MEMORY

Description

This function creates a copy of originalGraph and returns it in phGraphClone. All parameters are copied into the cloned graph. The original graph may be modified after this call without affecting the clone.

Child graph nodes in the original graph are recursively copied into the clone.

Note:

Graph objects are not threadsafe. More here.
Note that this function may also return error codes from previous, asynchronous launches.

See also:

cuGraphCreate, cuGraphNodeFindInClone

CUresult cuGraphCreate ( CUgraph* phGraph, unsigned int flags )

Creates a graph.

Parameters

phGraph: - Returns newly created graph
flags: - Graph creation flags, must be 0

Returns

CUDA_SUCCESS, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_INITIALIZED, CUDA_ERROR_INVALID_VALUE, CUDA_ERROR_OUT_OF_MEMORY

Description

Creates an empty graph, which is returned via phGraph.

Note:

Graph objects are not threadsafe. More here.
Note that this function may also return error codes from previous, asynchronous launches.

CUresult cuGraphDebugDotPrint ( CUgraph hGraph, const char* path, unsigned int flags )

Write a DOT file describing graph structure.

Parameters

hGraph: - The graph to create a DOT file from
path: - The path to write the DOT file to
flags: - Flags from CUgraphDebugDot_flags for specifying which additional node information to write

Returns

CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE, CUDA_ERROR_OPERATING_SYSTEM

Description

Using the provided hGraph, write to path a DOT formatted description of the graph. By default this includes the graph topology, node types, node id, kernel names and memcpy direction. flags can be specified to write more detailed information about each node type such as parameter values, kernel attributes, node and function handles.

CUresult cuGraphDestroy ( CUgraph hGraph )

Destroys a graph.

Parameters

hGraph: - Graph to destroy

Returns

CUDA_SUCCESS, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_INITIALIZED, CUDA_ERROR_INVALID_VALUE

Description

Destroys the graph specified by hGraph, as well as all of its nodes.

Note:

Graph objects are not threadsafe. More here.
Note that this function may also return error codes from previous, asynchronous launches.

See also:

cuGraphCreate

CUresult cuGraphDestroyNode ( CUgraphNode hNode )

Remove a node from the graph.

Parameters

hNode: - Node to remove

Returns

CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE

Description

Removes hNode from its graph. This operation also severs any dependencies of other nodes on hNode and vice versa.

Nodes which belong to a graph which contains allocation or free nodes cannot be destroyed. Any attempt to do so will return an error.

Note:

Graph objects are not threadsafe. More here.
Note that this function may also return error codes from previous, asynchronous launches.

CUresult cuGraphEventRecordNodeGetEvent ( CUgraphNode hNode, CUevent* event_out )

Returns the event associated with an event record node.

Parameters

hNode: - Node to get the event for
event_out: - Pointer to return the event

Returns

CUDA_SUCCESS, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_INITIALIZED, CUDA_ERROR_INVALID_VALUE

Description

Returns the event of event record node hNode in event_out.

Note:

Graph objects are not threadsafe. More here.
Note that this function may also return error codes from previous, asynchronous launches.

CUresult cuGraphEventRecordNodeSetEvent ( CUgraphNode hNode, CUevent event )

Sets an event record node's event.

Parameters

hNode: - Node to set the event for
event: - Event to use

Returns

CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE, CUDA_ERROR_INVALID_HANDLE, CUDA_ERROR_OUT_OF_MEMORY

Description

Sets the event of event record node hNode to event.

Note:

Graph objects are not threadsafe. More here.
Note that this function may also return error codes from previous, asynchronous launches.

CUresult cuGraphEventWaitNodeGetEvent ( CUgraphNode hNode, CUevent* event_out )

Returns the event associated with an event wait node.

Parameters

hNode: - Node to get the event for
event_out: - Pointer to return the event

Returns

CUDA_SUCCESS, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_INITIALIZED, CUDA_ERROR_INVALID_VALUE

Description

Returns the event of event wait node hNode in event_out.

Note:

Graph objects are not threadsafe. More here.
Note that this function may also return error codes from previous, asynchronous launches.

CUresult cuGraphEventWaitNodeSetEvent ( CUgraphNode hNode, CUevent event )

Sets an event wait node's event.

Parameters

hNode: - Node to set the event for
event: - Event to use

Returns

CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE, CUDA_ERROR_INVALID_HANDLE, CUDA_ERROR_OUT_OF_MEMORY

Description

Sets the event of event wait node hNode to event.

Note:

Graph objects are not threadsafe. More here.
Note that this function may also return error codes from previous, asynchronous launches.

CUresult cuGraphExecChildGraphNodeSetParams ( CUgraphExec hGraphExec, CUgraphNode hNode, CUgraph childGraph )

Updates node parameters in the child graph node in the given graphExec.

Parameters

hGraphExec: - The executable graph in which to set the specified node
hNode: - Host node from the graph which was used to instantiate graphExec
childGraph: - The graph supplying the updated parameters

Returns

CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE,

Description

Updates the work represented by hNode in hGraphExec as though the nodes contained in hNode's graph had the parameters contained in childGraph's nodes at instantiation. hNode must remain in the graph which was used to instantiate hGraphExec. Changed edges to and from hNode are ignored.

The modifications only affect future launches of hGraphExec. Already enqueued or running launches of hGraphExec are not affected by this call. hNode is also not modified by this call.

The topology of childGraph, as well as the node insertion order, must match that of the graph contained in hNode. See cuGraphExecUpdate() for a list of restrictions on what can be updated in an instantiated graph. The update is recursive, so child graph nodes contained within the top level child graph will also be updated.

Note:

Graph objects are not threadsafe. More here.
Note that this function may also return error codes from previous, asynchronous launches.

CUresult cuGraphExecDestroy ( CUgraphExec hGraphExec )

Destroys an executable graph.

Parameters

hGraphExec: - Executable graph to destroy

Returns

CUDA_SUCCESS, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_INITIALIZED, CUDA_ERROR_INVALID_VALUE

Description

Destroys the executable graph specified by hGraphExec, as well as all of its executable nodes. If the executable graph is in-flight, it will not be terminated, but rather freed asynchronously on completion.

Note:

Graph objects are not threadsafe. More here.
Note that this function may also return error codes from previous, asynchronous launches.

CUresult cuGraphExecEventRecordNodeSetEvent ( CUgraphExec hGraphExec, CUgraphNode hNode, CUevent event )

Sets the event for an event record node in the given graphExec.

Parameters

hGraphExec: - The executable graph in which to set the specified node
hNode: - event record node from the graph from which graphExec was instantiated
event: - Updated event to use

Returns

CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE,

Description

Sets the event of an event record node in an executable graph hGraphExec. The node is identified by the corresponding node hNode in the non-executable graph, from which the executable graph was instantiated.

The modifications only affect future launches of hGraphExec. Already enqueued or running launches of hGraphExec are not affected by this call. hNode is also not modified by this call.

Note:

Graph objects are not threadsafe. More here.
Note that this function may also return error codes from previous, asynchronous launches.

CUresult cuGraphExecEventWaitNodeSetEvent ( CUgraphExec hGraphExec, CUgraphNode hNode, CUevent event )

Sets the event for an event wait node in the given graphExec.

Parameters

hGraphExec: - The executable graph in which to set the specified node
hNode: - event wait node from the graph from which graphExec was instantiated
event: - Updated event to use

Returns

CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE,

Description

Sets the event of an event wait node in an executable graph hGraphExec. The node is identified by the corresponding node hNode in the non-executable graph, from which the executable graph was instantiated.

The modifications only affect future launches of hGraphExec. Already enqueued or running launches of hGraphExec are not affected by this call. hNode is also not modified by this call.

Note:

Graph objects are not threadsafe. More here.
Note that this function may also return error codes from previous, asynchronous launches.

CUresult cuGraphExecExternalSemaphoresSignalNodeSetParams ( CUgraphExec hGraphExec, CUgraphNode hNode, const CUDA_EXT_SEM_SIGNAL_NODE_PARAMS* nodeParams )

Sets the parameters for an external semaphore signal node in the given graphExec.

Parameters

hGraphExec: - The executable graph in which to set the specified node
hNode: - semaphore signal node from the graph from which graphExec was instantiated
nodeParams: - Updated Parameters to set

Returns

CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE,

Description

Sets the parameters of an external semaphore signal node in an executable graph hGraphExec. The node is identified by the corresponding node hNode in the non-executable graph, from which the executable graph was instantiated.

hNode must not have been removed from the original graph.

The modifications only affect future launches of hGraphExec. Already enqueued or running launches of hGraphExec are not affected by this call. hNode is also not modified by this call.

Changing nodeParams->numExtSems is not supported.

Note:

Graph objects are not threadsafe. More here.
Note that this function may also return error codes from previous, asynchronous launches.

CUresult cuGraphExecExternalSemaphoresWaitNodeSetParams ( CUgraphExec hGraphExec, CUgraphNode hNode, const CUDA_EXT_SEM_WAIT_NODE_PARAMS* nodeParams )

Sets the parameters for an external semaphore wait node in the given graphExec.

Parameters

hGraphExec: - The executable graph in which to set the specified node
hNode: - semaphore wait node from the graph from which graphExec was instantiated
nodeParams: - Updated Parameters to set

Returns

CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE,

Description

Sets the parameters of an external semaphore wait node in an executable graph hGraphExec. The node is identified by the corresponding node hNode in the non-executable graph, from which the executable graph was instantiated.

hNode must not have been removed from the original graph.

The modifications only affect future launches of hGraphExec. Already enqueued or running launches of hGraphExec are not affected by this call. hNode is also not modified by this call.

Changing nodeParams->numExtSems is not supported.

Note:

Graph objects are not threadsafe. More here.
Note that this function may also return error codes from previous, asynchronous launches.

CUresult cuGraphExecHostNodeSetParams ( CUgraphExec hGraphExec, CUgraphNode hNode, const CUDA_HOST_NODE_PARAMS* nodeParams )

Sets the parameters for a host node in the given graphExec.

Parameters

hGraphExec: - The executable graph in which to set the specified node
hNode: - Host node from the graph which was used to instantiate graphExec
nodeParams: - The updated parameters to set

Returns

CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE,

Description

Updates the work represented by hNode in hGraphExec as though hNode had contained nodeParams at instantiation. hNode must remain in the graph which was used to instantiate hGraphExec. Changed edges to and from hNode are ignored.

The modifications only affect future launches of hGraphExec. Already enqueued or running launches of hGraphExec are not affected by this call. hNode is also not modified by this call.

Note:

Graph objects are not threadsafe. More here.
Note that this function may also return error codes from previous, asynchronous launches.

CUresult cuGraphExecKernelNodeSetParams ( CUgraphExec hGraphExec, CUgraphNode hNode, const CUDA_KERNEL_NODE_PARAMS* nodeParams )

Sets the parameters for a kernel node in the given graphExec.

Parameters

hGraphExec: - The executable graph in which to set the specified node
hNode: - kernel node from the graph from which graphExec was instantiated
nodeParams: - Updated Parameters to set

Returns

CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE,

Description

Sets the parameters of a kernel node in an executable graph hGraphExec. The node is identified by the corresponding node hNode in the non-executable graph, from which the executable graph was instantiated.

hNode must not have been removed from the original graph. The func field of nodeParams cannot be modified and must match the original value. All other values can be modified.

The modifications only affect future launches of hGraphExec. Already enqueued or running launches of hGraphExec are not affected by this call. hNode is also not modified by this call.

Note:

Graph objects are not threadsafe. More here.
Note that this function may also return error codes from previous, asynchronous launches.

CUresult cuGraphExecMemcpyNodeSetParams ( CUgraphExec hGraphExec, CUgraphNode hNode, const CUDA_MEMCPY3D* copyParams, CUcontext ctx )

Sets the parameters for a memcpy node in the given graphExec.

Parameters

hGraphExec: - The executable graph in which to set the specified node
hNode: - Memcpy node from the graph which was used to instantiate graphExec
copyParams: - The updated parameters to set
ctx: - Context on which to run the node

Returns

CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE,

Description

Updates the work represented by hNode in hGraphExec as though hNode had contained copyParams at instantiation. hNode must remain in the graph which was used to instantiate hGraphExec. Changed edges to and from hNode are ignored.

The source and destination memory in copyParams must be allocated from the same contexts as the original source and destination memory. Both the instantiation-time memory operands and the memory operands in copyParams must be 1-dimensional. Zero-length operations are not supported.

The modifications only affect future launches of hGraphExec. Already enqueued or running launches of hGraphExec are not affected by this call. hNode is also not modified by this call.

Returns CUDA_ERROR_INVALID_VALUE if the memory operands' mappings changed or either the original or new memory operands are multidimensional.

Note:

Graph objects are not threadsafe. More here.
Note that this function may also return error codes from previous, asynchronous launches.

CUresult cuGraphExecMemsetNodeSetParams ( CUgraphExec hGraphExec, CUgraphNode hNode, const CUDA_MEMSET_NODE_PARAMS* memsetParams, CUcontext ctx )

Sets the parameters for a memset node in the given graphExec.

Parameters

hGraphExec: - The executable graph in which to set the specified node
hNode: - Memset node from the graph which was used to instantiate graphExec
memsetParams: - The updated parameters to set
ctx: - Context on which to run the node

Returns

CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE,

Description

Updates the work represented by hNode in hGraphExec as though hNode had contained memsetParams at instantiation. hNode must remain in the graph which was used to instantiate hGraphExec. Changed edges to and from hNode are ignored.

The destination memory in memsetParams must be allocated from the same contexts as the original destination memory. Both the instantiation-time memory operand and the memory operand in memsetParams must be 1-dimensional. Zero-length operations are not supported.

The modifications only affect future launches of hGraphExec. Already enqueued or running launches of hGraphExec are not affected by this call. hNode is also not modified by this call.

Returns CUDA_ERROR_INVALID_VALUE if the memory operand's mappings changed or either the original or new memory operand are multidimensional.

Note:

Graph objects are not threadsafe. More here.
Note that this function may also return error codes from previous, asynchronous launches.

CUresult cuGraphExecUpdate ( CUgraphExec hGraphExec, CUgraph hGraph, CUgraphNode* hErrorNode_out, CUgraphExecUpdateResult* updateResult_out )

Check whether an executable graph can be updated with a graph and perform the update if possible.

Parameters

hGraphExec: The instantiated graph to be updated
hGraph: The graph containing the updated parameters
hErrorNode_out: The node which caused the permissibility check to forbid the update, if any
updateResult_out: Whether the graph update was permitted. If was forbidden, the reason why

Returns

CUDA_SUCCESS, CUDA_ERROR_GRAPH_EXEC_UPDATE_FAILURE,

Description

Updates the node parameters in the instantiated graph specified by hGraphExec with the node parameters in a topologically identical graph specified by hGraph.

Limitations:

Kernel nodes:
- The owning context of the function cannot change.
- A node whose function originally did not use CUDA dynamic parallelism cannot be updated to a function which uses CDP
Memset and memcpy nodes:
- The CUDA device(s) to which the operand(s) was allocated/mapped cannot change.
- The source/destination memory must be allocated from the same contexts as the original source/destination memory.
- Only 1D memsets can be changed.
Additional memcpy node restrictions:
- Changing either the source or destination memory type(i.e. CU_MEMORYTYPE_DEVICE, CU_MEMORYTYPE_ARRAY, etc.) is not supported.
External semaphore wait nodes and record nodes:
- Changing the number of semaphores is not supported.

Note: The API may add further restrictions in future releases. The return code should always be checked.

cuGraphExecUpdate sets updateResult_out to CU_GRAPH_EXEC_UPDATE_ERROR_TOPOLOGY_CHANGED under the following conditions:

The count of nodes directly in hGraphExec and hGraph differ, in which case hErrorNode_out is NULL.
A node is deleted in hGraph but not not its pair from hGraphExec, in which case hErrorNode_out is NULL.
A node is deleted in hGraphExec but not its pair from hGraph, in which case hErrorNode_out is the pairless node from hGraph.
The dependent nodes of a pair differ, in which case hErrorNode_out is the node from hGraph.

cuGraphExecUpdate sets updateResult_out to:

CU_GRAPH_EXEC_UPDATE_ERROR if passed an invalid value.
CU_GRAPH_EXEC_UPDATE_ERROR_TOPOLOGY_CHANGED if the graph topology changed
CU_GRAPH_EXEC_UPDATE_ERROR_NODE_TYPE_CHANGED if the type of a node changed, in which case hErrorNode_out is set to the node from hGraph.
CU_GRAPH_EXEC_UPDATE_ERROR_UNSUPPORTED_FUNCTION_CHANGE if the function changed in an unsupported way(see note above), in which case hErrorNode_out is set to the node from hGraph
CU_GRAPH_EXEC_UPDATE_ERROR_PARAMETERS_CHANGED if any parameters to a node changed in a way that is not supported, in which case hErrorNode_out is set to the node from hGraph.
CU_GRAPH_EXEC_UPDATE_ERROR_NOT_SUPPORTED if something about a node is unsupported, like the node's type or configuration, in which case hErrorNode_out is set to the node from hGraph

If updateResult_out isn't set in one of the situations described above, the update check passes and cuGraphExecUpdate updates hGraphExec to match the contents of hGraph. If an error happens during the update, updateResult_out will be set to CU_GRAPH_EXEC_UPDATE_ERROR; otherwise, updateResult_out is set to CU_GRAPH_EXEC_UPDATE_SUCCESS.

cuGraphExecUpdate returns CUDA_SUCCESS when the updated was performed successfully. It returns CUDA_ERROR_GRAPH_EXEC_UPDATE_FAILURE if the graph update was not performed because it included changes which violated constraints specific to instantiated graph update.

Note:

Graph objects are not threadsafe. More here.
Note that this function may also return error codes from previous, asynchronous launches.

See also:

cuGraphInstantiate,

CUresult cuGraphExternalSemaphoresSignalNodeGetParams ( CUgraphNode hNode, CUDA_EXT_SEM_SIGNAL_NODE_PARAMS* params_out )

Returns an external semaphore signal node's parameters.

Parameters

hNode: - Node to get the parameters for
params_out: - Pointer to return the parameters

Returns

CUDA_SUCCESS, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_INITIALIZED, CUDA_ERROR_INVALID_VALUE

Description

Returns the parameters of an external semaphore signal node hNode in params_out. The extSemArray and paramsArray returned in params_out, are owned by the node. This memory remains valid until the node is destroyed or its parameters are modified, and should not be modified directly. Use cuGraphExternalSemaphoresSignalNodeSetParams to update the parameters of this node.

Note:

Graph objects are not threadsafe. More here.
Note that this function may also return error codes from previous, asynchronous launches.

CUresult cuGraphExternalSemaphoresSignalNodeSetParams ( CUgraphNode hNode, const CUDA_EXT_SEM_SIGNAL_NODE_PARAMS* nodeParams )

Sets an external semaphore signal node's parameters.

Parameters

hNode: - Node to set the parameters for
nodeParams: - Parameters to copy

Returns

CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE, CUDA_ERROR_INVALID_HANDLE, CUDA_ERROR_OUT_OF_MEMORY

Description

Sets the parameters of an external semaphore signal node hNode to nodeParams.

Note:

Graph objects are not threadsafe. More here.
Note that this function may also return error codes from previous, asynchronous launches.

CUresult cuGraphExternalSemaphoresWaitNodeGetParams ( CUgraphNode hNode, CUDA_EXT_SEM_WAIT_NODE_PARAMS* params_out )

Returns an external semaphore wait node's parameters.

Parameters

hNode: - Node to get the parameters for
params_out: - Pointer to return the parameters

Returns

CUDA_SUCCESS, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_INITIALIZED, CUDA_ERROR_INVALID_VALUE

Description

Returns the parameters of an external semaphore wait node hNode in params_out. The extSemArray and paramsArray returned in params_out, are owned by the node. This memory remains valid until the node is destroyed or its parameters are modified, and should not be modified directly. Use cuGraphExternalSemaphoresSignalNodeSetParams to update the parameters of this node.

Note:

Graph objects are not threadsafe. More here.
Note that this function may also return error codes from previous, asynchronous launches.

CUresult cuGraphExternalSemaphoresWaitNodeSetParams ( CUgraphNode hNode, const CUDA_EXT_SEM_WAIT_NODE_PARAMS* nodeParams )

Sets an external semaphore wait node's parameters.

Parameters

hNode: - Node to set the parameters for
nodeParams: - Parameters to copy

Returns

CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE, CUDA_ERROR_INVALID_HANDLE, CUDA_ERROR_OUT_OF_MEMORY

Description

Sets the parameters of an external semaphore wait node hNode to nodeParams.

Note:

Graph objects are not threadsafe. More here.
Note that this function may also return error codes from previous, asynchronous launches.

CUresult cuGraphGetEdges ( CUgraph hGraph, CUgraphNode* from, CUgraphNode* to, size_t* numEdges )

Returns a graph's dependency edges.

Parameters

hGraph: - Graph to get the edges from
from: - Location to return edge endpoints
to: - Location to return edge endpoints
numEdges: - See description

Returns

CUDA_SUCCESS, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_INITIALIZED, CUDA_ERROR_INVALID_VALUE

Description

Returns a list of hGraph's dependency edges. Edges are returned via corresponding indices in from and to; that is, the node in to[i] has a dependency on the node in from[i]. from and to may both be NULL, in which case this function only returns the number of edges in numEdges. Otherwise, numEdges entries will be filled in. If numEdges is higher than the actual number of edges, the remaining entries in from and to will be set to NULL, and the number of edges actually returned will be written to numEdges.

Note:

Graph objects are not threadsafe. More here.
Note that this function may also return error codes from previous, asynchronous launches.

CUresult cuGraphGetNodes ( CUgraph hGraph, CUgraphNode* nodes, size_t* numNodes )

Returns a graph's nodes.

Parameters

hGraph: - Graph to query
nodes: - Pointer to return the nodes
numNodes: - See description

Returns

CUDA_SUCCESS, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_INITIALIZED, CUDA_ERROR_INVALID_VALUE

Description

Returns a list of hGraph's nodes. nodes may be NULL, in which case this function will return the number of nodes in numNodes. Otherwise, numNodes entries will be filled in. If numNodes is higher than the actual number of nodes, the remaining entries in nodes will be set to NULL, and the number of nodes actually obtained will be returned in numNodes.

Note:

Graph objects are not threadsafe. More here.
Note that this function may also return error codes from previous, asynchronous launches.

CUresult cuGraphGetRootNodes ( CUgraph hGraph, CUgraphNode* rootNodes, size_t* numRootNodes )

Returns a graph's root nodes.

Parameters

hGraph: - Graph to query
rootNodes: - Pointer to return the root nodes
numRootNodes: - See description

Returns

CUDA_SUCCESS, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_INITIALIZED, CUDA_ERROR_INVALID_VALUE

Description

Returns a list of hGraph's root nodes. rootNodes may be NULL, in which case this function will return the number of root nodes in numRootNodes. Otherwise, numRootNodes entries will be filled in. If numRootNodes is higher than the actual number of root nodes, the remaining entries in rootNodes will be set to NULL, and the number of nodes actually obtained will be returned in numRootNodes.

Note:

Graph objects are not threadsafe. More here.
Note that this function may also return error codes from previous, asynchronous launches.

CUresult cuGraphHostNodeGetParams ( CUgraphNode hNode, CUDA_HOST_NODE_PARAMS* nodeParams )

Returns a host node's parameters.

Parameters

hNode: - Node to get the parameters for
nodeParams: - Pointer to return the parameters

Returns

CUDA_SUCCESS, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_INITIALIZED, CUDA_ERROR_INVALID_VALUE

Description

Returns the parameters of host node hNode in nodeParams.

Note:

Graph objects are not threadsafe. More here.
Note that this function may also return error codes from previous, asynchronous launches.

CUresult cuGraphHostNodeSetParams ( CUgraphNode hNode, const CUDA_HOST_NODE_PARAMS* nodeParams )

Sets a host node's parameters.

Parameters

hNode: - Node to set the parameters for
nodeParams: - Parameters to copy

Returns

CUDA_SUCCESS, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_INITIALIZED, CUDA_ERROR_INVALID_VALUE

Description

Sets the parameters of host node hNode to nodeParams.

Note:

Graph objects are not threadsafe. More here.
Note that this function may also return error codes from previous, asynchronous launches.

CUresult cuGraphInstantiate ( CUgraphExec* phGraphExec, CUgraph hGraph, CUgraphNode* phErrorNode, char* logBuffer, size_t bufferSize )

Creates an executable graph from a graph.

Parameters

phGraphExec: - Returns instantiated graph
hGraph: - Graph to instantiate
phErrorNode: - In case of an instantiation error, this may be modified to indicate a node contributing to the error
logBuffer: - A character buffer to store diagnostic messages
bufferSize: - Size of the log buffer in bytes

Returns

CUDA_SUCCESS, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_INITIALIZED, CUDA_ERROR_INVALID_VALUE

Description

Instantiates hGraph as an executable graph. The graph is validated for any structural constraints or intra-node constraints which were not previously validated. If instantiation is successful, a handle to the instantiated graph is returned in phGraphExec.

If there are any errors, diagnostic information may be returned in errorNode and logBuffer. This is the primary way to inspect instantiation errors. The output will be null terminated unless the diagnostics overflow the buffer. In this case, they will be truncated, and the last byte can be inspected to determine if truncation occurred.

Note:

Graph objects are not threadsafe. More here.
Note that this function may also return error codes from previous, asynchronous launches.

CUresult cuGraphInstantiateWithFlags ( CUgraphExec* phGraphExec, CUgraph hGraph, unsigned long long flags )

Creates an executable graph from a graph.

Parameters

phGraphExec: - Returns instantiated graph
hGraph: - Graph to instantiate
flags: - Flags to control instantiation. See CUgraphInstantiate_flags.

Returns

CUDA_SUCCESS, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_INITIALIZED, CUDA_ERROR_INVALID_VALUE

Description

The flags parameter controls the behavior of instantiation and subsequent graph launches. Valid flags are:

CUDA_GRAPH_INSTANTIATE_FLAG_AUTO_FREE_ON_LAUNCH, which configures a graph containing memory allocation nodes to automatically free any unfreed memory allocations before the graph is relaunched.

If hGraph contains any allocation or free nodes, there can be at most one executable graph in existence for that graph at a time.

An attempt to instantiate a second executable graph before destroying the first with cuGraphExecDestroy will result in an error.

Note:

Graph objects are not threadsafe. More here.
Note that this function may also return error codes from previous, asynchronous launches.

CUresult cuGraphKernelNodeCopyAttributes ( CUgraphNode dst, CUgraphNode src )

Copies attributes from source node to destination node.

Parameters

dst: Destination node
src: Source node For list of attributes see CUkernelNodeAttrID

Returns

CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE

Description

Copies attributes from source node src to destination node dst. Both node must have the same context.

Note:

Note that this function may also return error codes from previous, asynchronous launches.

See also:

CUaccessPolicyWindow

CUresult cuGraphKernelNodeGetAttribute ( CUgraphNode hNode, CUkernelNodeAttrID attr, CUkernelNodeAttrValue* value_out )

Queries node attribute.

Parameters

hNode
attr
value_out

Returns

CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE, CUDA_ERROR_INVALID_HANDLE

Description

Queries attribute attr from node hNode and stores it in corresponding member of value_out.

Note:

Note that this function may also return error codes from previous, asynchronous launches.

See also:

CUaccessPolicyWindow

CUresult cuGraphKernelNodeGetParams ( CUgraphNode hNode, CUDA_KERNEL_NODE_PARAMS* nodeParams )

Returns a kernel node's parameters.

Parameters

hNode: - Node to get the parameters for
nodeParams: - Pointer to return the parameters

Returns

CUDA_SUCCESS, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_INITIALIZED, CUDA_ERROR_INVALID_VALUE

Description

Returns the parameters of kernel node hNode in nodeParams. The kernelParams or extra array returned in nodeParams, as well as the argument values it points to, are owned by the node. This memory remains valid until the node is destroyed or its parameters are modified, and should not be modified directly. Use cuGraphKernelNodeSetParams to update the parameters of this node.

The params will contain either kernelParams or extra, according to which of these was most recently set on the node.

Note:

Graph objects are not threadsafe. More here.
Note that this function may also return error codes from previous, asynchronous launches.

CUresult cuGraphKernelNodeSetAttribute ( CUgraphNode hNode, CUkernelNodeAttrID attr, const CUkernelNodeAttrValue* value )

Sets node attribute.

Parameters

hNode
attr
value

Returns

CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE, CUDA_ERROR_INVALID_HANDLE

Description

Sets attribute attr on node hNode from corresponding attribute of value.

Note:

Note that this function may also return error codes from previous, asynchronous launches.

See also:

CUaccessPolicyWindow

CUresult cuGraphKernelNodeSetParams ( CUgraphNode hNode, const CUDA_KERNEL_NODE_PARAMS* nodeParams )

Sets a kernel node's parameters.

Parameters

hNode: - Node to set the parameters for
nodeParams: - Parameters to copy

Returns

CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE, CUDA_ERROR_INVALID_HANDLE, CUDA_ERROR_OUT_OF_MEMORY

Description

Sets the parameters of kernel node hNode to nodeParams.

Note:

Graph objects are not threadsafe. More here.
Note that this function may also return error codes from previous, asynchronous launches.

CUresult cuGraphLaunch ( CUgraphExec hGraphExec, CUstream hStream )

Launches an executable graph in a stream.

Parameters

hGraphExec: - Executable graph to launch
hStream: - Stream in which to launch the graph

Returns

CUDA_SUCCESS, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_INITIALIZED, CUDA_ERROR_INVALID_VALUE

Description

Executes hGraphExec in hStream. Only one instance of hGraphExec may be executing at a time. Each launch is ordered behind both any previous work in hStream and any previous launches of hGraphExec. To execute a graph concurrently, it must be instantiated multiple times into multiple executable graphs.

If any allocations created by hGraphExec remain unfreed (from a previous launch) and hGraphExec was not instantiated with CUDA_GRAPH_INSTANTIATE_FLAG_AUTO_FREE_ON_LAUNCH, the launch will fail with CUDA_ERROR_INVALID_VALUE.

Note:

Graph objects are not threadsafe. More here.
Note that this function may also return error codes from previous, asynchronous launches.

CUresult cuGraphMemAllocNodeGetParams ( CUgraphNode hNode, CUDA_MEM_ALLOC_NODE_PARAMS* params_out )

Returns a memory alloc node's parameters.

Parameters

hNode: - Node to get the parameters for
params_out: - Pointer to return the parameters

Returns

CUDA_SUCCESS, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_INITIALIZED, CUDA_ERROR_INVALID_VALUE

Description

Returns the parameters of a memory alloc node hNode in params_out. The poolProps and accessDescs returned in params_out, are owned by the node. This memory remains valid until the node is destroyed. The returned parameters must not be modified.

Note:

Graph objects are not threadsafe. More here.
Note that this function may also return error codes from previous, asynchronous launches.

CUresult cuGraphMemFreeNodeGetParams ( CUgraphNode hNode, CUdeviceptr* dptr_out )

Returns a memory free node's parameters.

Parameters

hNode: - Node to get the parameters for
dptr_out: - Pointer to return the device address

Returns

CUDA_SUCCESS, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_INITIALIZED, CUDA_ERROR_INVALID_VALUE

Description

Returns the address of a memory free node hNode in dptr_out.

Note:

Graph objects are not threadsafe. More here.
Note that this function may also return error codes from previous, asynchronous launches.

CUresult cuGraphMemcpyNodeGetParams ( CUgraphNode hNode, CUDA_MEMCPY3D* nodeParams )

Returns a memcpy node's parameters.

Parameters

hNode: - Node to get the parameters for
nodeParams: - Pointer to return the parameters

Returns

CUDA_SUCCESS, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_INITIALIZED, CUDA_ERROR_INVALID_VALUE

Description

Returns the parameters of memcpy node hNode in nodeParams.

Note:

Graph objects are not threadsafe. More here.
Note that this function may also return error codes from previous, asynchronous launches.

CUresult cuGraphMemcpyNodeSetParams ( CUgraphNode hNode, const CUDA_MEMCPY3D* nodeParams )

Sets a memcpy node's parameters.

Parameters

hNode: - Node to set the parameters for
nodeParams: - Parameters to copy

Returns

CUDA_SUCCESS, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_INITIALIZED, CUDA_ERROR_INVALID_VALUE,

Description

Sets the parameters of memcpy node hNode to nodeParams.

Note:

Graph objects are not threadsafe. More here.
Note that this function may also return error codes from previous, asynchronous launches.

CUresult cuGraphMemsetNodeGetParams ( CUgraphNode hNode, CUDA_MEMSET_NODE_PARAMS* nodeParams )

Returns a memset node's parameters.

Parameters

hNode: - Node to get the parameters for
nodeParams: - Pointer to return the parameters

Returns

CUDA_SUCCESS, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_INITIALIZED, CUDA_ERROR_INVALID_VALUE

Description

Returns the parameters of memset node hNode in nodeParams.

Note:

Graph objects are not threadsafe. More here.
Note that this function may also return error codes from previous, asynchronous launches.

CUresult cuGraphMemsetNodeSetParams ( CUgraphNode hNode, const CUDA_MEMSET_NODE_PARAMS* nodeParams )

Sets a memset node's parameters.

Parameters

hNode: - Node to set the parameters for
nodeParams: - Parameters to copy

Returns

CUDA_SUCCESS, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_INITIALIZED, CUDA_ERROR_INVALID_VALUE

Description

Sets the parameters of memset node hNode to nodeParams.

Note:

Graph objects are not threadsafe. More here.
Note that this function may also return error codes from previous, asynchronous launches.

CUresult cuGraphNodeFindInClone ( CUgraphNode* phNode, CUgraphNode hOriginalNode, CUgraph hClonedGraph )

Finds a cloned version of a node.

Parameters

phNode: - Returns handle to the cloned node
hOriginalNode: - Handle to the original node
hClonedGraph: - Cloned graph to query

Returns

CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE,

Description

This function returns the node in hClonedGraph corresponding to hOriginalNode in the original graph.

hClonedGraph must have been cloned from hOriginalGraph via cuGraphClone. hOriginalNode must have been in hOriginalGraph at the time of the call to cuGraphClone, and the corresponding cloned node in hClonedGraph must not have been removed. The cloned node is then returned via phClonedNode.

Note:

Graph objects are not threadsafe. More here.
Note that this function may also return error codes from previous, asynchronous launches.

See also:

cuGraphClone

CUresult cuGraphNodeGetDependencies ( CUgraphNode hNode, CUgraphNode* dependencies, size_t* numDependencies )

Returns a node's dependencies.

Parameters

hNode: - Node to query
dependencies: - Pointer to return the dependencies
numDependencies: - See description

Returns

CUDA_SUCCESS, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_INITIALIZED, CUDA_ERROR_INVALID_VALUE

Description

Returns a list of node's dependencies. dependencies may be NULL, in which case this function will return the number of dependencies in numDependencies. Otherwise, numDependencies entries will be filled in. If numDependencies is higher than the actual number of dependencies, the remaining entries in dependencies will be set to NULL, and the number of nodes actually obtained will be returned in numDependencies.

Note:

Graph objects are not threadsafe. More here.
Note that this function may also return error codes from previous, asynchronous launches.

CUresult cuGraphNodeGetDependentNodes ( CUgraphNode hNode, CUgraphNode* dependentNodes, size_t* numDependentNodes )

Returns a node's dependent nodes.

Parameters

hNode: - Node to query
dependentNodes: - Pointer to return the dependent nodes
numDependentNodes: - See description

Returns

CUDA_SUCCESS, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_INITIALIZED, CUDA_ERROR_INVALID_VALUE

Description

Returns a list of node's dependent nodes. dependentNodes may be NULL, in which case this function will return the number of dependent nodes in numDependentNodes. Otherwise, numDependentNodes entries will be filled in. If numDependentNodes is higher than the actual number of dependent nodes, the remaining entries in dependentNodes will be set to NULL, and the number of nodes actually obtained will be returned in numDependentNodes.

Note:

Graph objects are not threadsafe. More here.
Note that this function may also return error codes from previous, asynchronous launches.

CUresult cuGraphNodeGetType ( CUgraphNode hNode, CUgraphNodeType* type )

Returns a node's type.

Parameters

hNode: - Node to query
type: - Pointer to return the node type

Returns

CUDA_SUCCESS, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_INITIALIZED, CUDA_ERROR_INVALID_VALUE

Description

Returns the node type of hNode in type.

Note:

Graph objects are not threadsafe. More here.
Note that this function may also return error codes from previous, asynchronous launches.

CUresult cuGraphReleaseUserObject ( CUgraph graph, CUuserObject object, unsigned int count )

Release a user object reference from a graph.

Parameters

graph: - The graph that will release the reference
object: - The user object to release a reference for
count: - The number of references to release, typically 1. Must be nonzero and not larger than INT_MAX.

Returns

CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE

Description

Releases user object references owned by a graph.

See CUDA User Objects in the CUDA C++ Programming Guide for more information on user objects.

CUresult cuGraphRemoveDependencies ( CUgraph hGraph, const CUgraphNode* from, const CUgraphNode* to, size_t numDependencies )

Removes dependency edges from a graph.

Parameters

hGraph: - Graph from which to remove dependencies
from: - Array of nodes that provide the dependencies
to: - Array of dependent nodes
numDependencies: - Number of dependencies to be removed

Returns

CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE

Description

The number of dependencies to be removed is defined by numDependencies. Elements in from and to at corresponding indices define a dependency. Each node in from and to must belong to hGraph.

If numDependencies is 0, elements in from and to will be ignored. Specifying a non-existing dependency will return an error.

Dependencies cannot be removed from graphs which contain allocation or free nodes. Any attempt to do so will return an error.

Note:

Graph objects are not threadsafe. More here.
Note that this function may also return error codes from previous, asynchronous launches.

CUresult cuGraphRetainUserObject ( CUgraph graph, CUuserObject object, unsigned int count, unsigned int flags )

Retain a reference to a user object from a graph.

Parameters

graph: - The graph to associate the reference with
object: - The user object to retain a reference for
count: - The number of references to add to the graph, typically 1. Must be nonzero and not larger than INT_MAX.
flags: - The optional flag CU_GRAPH_USER_OBJECT_MOVE transfers references from the calling thread, rather than create new references. Pass 0 to create new references.

Returns

CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE

Description

Creates or moves user object references that will be owned by a CUDA graph.

See CUDA User Objects in the CUDA C++ Programming Guide for more information on user objects.

CUresult cuGraphUpload ( CUgraphExec hGraphExec, CUstream hStream )

Uploads an executable graph in a stream.

Parameters

hGraphExec: - Executable graph to upload
hStream: - Stream in which to upload the graph

Returns

CUDA_SUCCESS, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_INITIALIZED, CUDA_ERROR_INVALID_VALUE

Description

Uploads hGraphExec to the device in hStream without executing it. Uploads of the same hGraphExec will be serialized. Each upload is ordered behind both any previous work in hStream and any previous launches of hGraphExec. Uses memory cached by stream to back the allocations owned by hGraphExec.

Note:

Graph objects are not threadsafe. More here.
Note that this function may also return error codes from previous, asynchronous launches.

CUresult cuUserObjectCreate ( CUuserObject* object_out, void* ptr, CUhostFn destroy, unsigned int initialRefcount, unsigned int flags )

Create a user object.

Parameters

object_out: - Location to return the user object handle
ptr: - The pointer to pass to the destroy function
destroy: - Callback to free the user object when it is no longer in use
initialRefcount: - The initial refcount to create the object with, typically 1. The initial references are owned by the calling thread.
flags: - Currently it is required to pass CU_USER_OBJECT_NO_DESTRUCTOR_SYNC, which is the only defined flag. This indicates that the destroy callback cannot be waited on by any CUDA API. Users requiring synchronization of the callback should signal its completion manually.

Returns

CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE

Description

Create a user object with the specified destructor callback and initial reference count. The initial references are owned by the caller.

Destructor callbacks cannot make CUDA API calls and should avoid blocking behavior, as they are executed by a shared internal thread. Another thread may be signaled to perform such actions, if it does not block forward progress of tasks scheduled through CUDA.

See CUDA User Objects in the CUDA C++ Programming Guide for more information on user objects.

CUresult cuUserObjectRelease ( CUuserObject object, unsigned int count )

Release a reference to a user object.

Parameters

object: - The object to release
count: - The number of references to release, typically 1. Must be nonzero and not larger than INT_MAX.

Returns

CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE

Description

Releases user object references owned by the caller. The object's destructor is invoked if the reference count reaches zero.

It is undefined behavior to release references not owned by the caller, or to use a user object handle after all references are released.

See CUDA User Objects in the CUDA C++ Programming Guide for more information on user objects.

CUresult cuUserObjectRetain ( CUuserObject object, unsigned int count )

Retain a reference to a user object.

Parameters

object: - The object to retain
count: - The number of references to retain, typically 1. Must be nonzero and not larger than INT_MAX.

Returns

CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE

Description

Retains new references to a user object. The new references are owned by the caller.

See CUDA User Objects in the CUDA C++ Programming Guide for more information on user objects.