5.1. Device Management

This section describes the device management functions of the CUDA runtime application programming interface.

Functions

__host__cudaError_t cudaChooseDevice ( int* device, const cudaDeviceProp* prop )
Select compute-device which best matches criteria.
__host____device__cudaError_t cudaDeviceGetAttribute ( int* value, cudaDeviceAttr attr, int  device )
Returns information about the device.
__host__cudaError_t cudaDeviceGetByPCIBusId ( int* device, const char* pciBusId )
Returns a handle to a compute device.
__host____device__cudaError_t cudaDeviceGetCacheConfig ( cudaFuncCache ** pCacheConfig )
Returns the preferred cache configuration for the current device.
__host____device__cudaError_t cudaDeviceGetLimit ( size_t* pValue, cudaLimit limit )
Returns resource limits.
__host__cudaError_t cudaDeviceGetNvSciSyncAttributes ( void* nvSciSyncAttrList, int  device, int  flags )
Return NvSciSync attributes that this device can support.
__host__cudaError_t cudaDeviceGetP2PAttribute ( int* value, cudaDeviceP2PAttr attr, int  srcDevice, int  dstDevice )
Queries attributes of the link between two devices.
__host__cudaError_t cudaDeviceGetPCIBusId ( char* pciBusId, int  len, int  device )
Returns a PCI Bus Id string for the device.
__host____device__cudaError_t cudaDeviceGetSharedMemConfig ( cudaSharedMemConfig ** pConfig )
Returns the shared memory configuration for the current device.
__host__cudaError_t cudaDeviceGetStreamPriorityRange ( int* leastPriority, int* greatestPriority )
Returns numerical values that correspond to the least and greatest stream priorities.
__host__cudaError_t cudaDeviceReset ( void )
Destroy all allocations and reset all state on the current device in the current process.
__host__cudaError_t cudaDeviceSetCacheConfig ( cudaFuncCache cacheConfig )
Sets the preferred cache configuration for the current device.
__host__cudaError_t cudaDeviceSetLimit ( cudaLimit limit, size_t value )
Set resource limits.
__host__cudaError_t cudaDeviceSetSharedMemConfig ( cudaSharedMemConfig config )
Sets the shared memory configuration for the current device.
__host____device__cudaError_t cudaDeviceSynchronize ( void )
Wait for compute device to finish.
__host____device__cudaError_t cudaGetDevice ( int* device )
Returns which device is currently being used.
__host____device__cudaError_t cudaGetDeviceCount ( int* count )
Returns the number of compute-capable devices.
__host__cudaError_t cudaGetDeviceFlags ( unsigned int* flags )
Gets the flags for the current device.
__host__cudaError_t cudaGetDeviceProperties ( cudaDeviceProp* prop, int  device )
Returns information about the compute-device.
__host__cudaError_t cudaIpcCloseMemHandle ( void* devPtr )
Close memory mapped with cudaIpcOpenMemHandle.
__host__cudaError_t cudaIpcGetEventHandle ( cudaIpcEventHandle_t* handle, cudaEvent_t event )
Gets an interprocess handle for a previously allocated event.
__host__cudaError_t cudaIpcGetMemHandle ( cudaIpcMemHandle_t* handle, void* devPtr )
Gets an interprocess memory handle for an existing device memory allocation.
__host__cudaError_t cudaIpcOpenEventHandle ( cudaEvent_t* event, cudaIpcEventHandle_t handle )
Opens an interprocess event handle for use in the current process.
__host__cudaError_t cudaIpcOpenMemHandle ( void** devPtr, cudaIpcMemHandle_t handle, unsigned int  flags )
Opens an interprocess memory handle exported from another process and returns a device pointer usable in the local process.
__host__cudaError_t cudaSetDevice ( int  device )
Set device to be used for GPU executions.
__host__cudaError_t cudaSetDeviceFlags ( unsigned int  flags )
Sets flags to be used for device executions.
__host__cudaError_t cudaSetValidDevices ( int* device_arr, int  len )
Set a list of devices that can be used for CUDA.

Functions

__host__cudaError_t cudaChooseDevice ( int* device, const cudaDeviceProp* prop )
Select compute-device which best matches criteria.
Parameters
device
- Device with best match
prop
- Desired device properties
Description

Returns in *device the device which has properties that best match *prop.

Note:

See also:

cudaGetDeviceCount, cudaGetDevice, cudaSetDevice, cudaGetDeviceProperties

__host____device__cudaError_t cudaDeviceGetAttribute ( int* value, cudaDeviceAttr attr, int  device )
Returns information about the device.
Parameters
value
- Returned device attribute value
attr
- Device attribute to query
device
- Device number to query
Description

Returns in *value the integer value of the attribute attr on device device. The supported attributes are:

Note:

See also:

cudaGetDeviceCount, cudaGetDevice, cudaSetDevice, cudaChooseDevice, cudaGetDeviceProperties, cuDeviceGetAttribute

__host__cudaError_t cudaDeviceGetByPCIBusId ( int* device, const char* pciBusId )
Returns a handle to a compute device.
Parameters
device
- Returned device ordinal
pciBusId
- String in one of the following forms: [domain]:[bus]:[device].[function] [domain]:[bus]:[device] [bus]:[device].[function] where domain, bus, device, and function are all hexadecimal values
Description

Returns in *device a device ordinal given a PCI bus ID string.

Note:

See also:

cudaDeviceGetPCIBusId, cuDeviceGetByPCIBusId

__host____device__cudaError_t cudaDeviceGetCacheConfig ( cudaFuncCache ** pCacheConfig )
Returns the preferred cache configuration for the current device.
Parameters
pCacheConfig
- Returned cache configuration
Returns

cudaSuccess

Description

On devices where the L1 cache and shared memory use the same hardware resources, this returns through pCacheConfig the preferred cache configuration for the current device. This is only a preference. The runtime will use the requested configuration if possible, but it is free to choose a different configuration if required to execute functions.

This will return a pCacheConfig of cudaFuncCachePreferNone on devices where the size of the L1 cache and shared memory are fixed.

The supported cache configurations are:

Note:

See also:

cudaDeviceSetCacheConfig, cudaFuncSetCacheConfig ( C API), cudaFuncSetCacheConfig ( C++ API), cuCtxGetCacheConfig

__host____device__cudaError_t cudaDeviceGetLimit ( size_t* pValue, cudaLimit limit )
Returns resource limits.
Parameters
pValue
- Returned size of the limit
limit
- Limit to query
Description

Returns in *pValue the current size of limit. The supported cudaLimit values are:

Note:

See also:

cudaDeviceSetLimit, cuCtxGetLimit

__host__cudaError_t cudaDeviceGetNvSciSyncAttributes ( void* nvSciSyncAttrList, int  device, int  flags )
Return NvSciSync attributes that this device can support.
Parameters
nvSciSyncAttrList
- Return NvSciSync attributes supported.
device
- Valid Cuda Device to get NvSciSync attributes for.
flags
- flags describing NvSciSync usage.
Description

Returns in nvSciSyncAttrList, the properties of NvSciSync that this CUDA device, dev can support. The returned nvSciSyncAttrList can be used to create an NvSciSync that matches this device's capabilities.

If NvSciSyncAttrKey_RequiredPerm field in nvSciSyncAttrList is already set this API will return cudaErrorNotSupported.

The applications should set nvSciSyncAttrList to a valid NvSciSyncAttrList failing which this API will return cudaErrorInvalidHandle.

The flags controls how applications intends to use the NvSciSync created from the nvSciSyncAttrList. The valid flags are:

At least one of these flags must be set, failing which the API returns cudaErrorInvalidValue. Both the flags are orthogonal to one another: a developer may set both these flags that allows to set both wait and signal specific attributes in the same nvSciSyncAttrList.

cudaSuccess, cudaErrorDeviceUninitialized, cudaErrorInvalidValue, cudaErrorInvalidHandle, cudaErrorInvalidDevice, cudaErrorNotSupported, cudaErrorMemoryAllocation

See also:

cudaImportExternalSemaphore, cudaDestroyExternalSemaphore, cudaSignalExternalSemaphoresAsync, cudaWaitExternalSemaphoresAsync

__host__cudaError_t cudaDeviceGetP2PAttribute ( int* value, cudaDeviceP2PAttr attr, int  srcDevice, int  dstDevice )
Queries attributes of the link between two devices.
Parameters
value
- Returned value of the requested attribute
attr
srcDevice
- The source device of the target link.
dstDevice
- The destination device of the target link.
Description

Returns in *value the value of the requested attribute attrib of the link between srcDevice and dstDevice. The supported attributes are:

Returns cudaErrorInvalidDevice if srcDevice or dstDevice are not valid or if they represent the same device.

Returns cudaErrorInvalidValue if attrib is not valid or if value is a null pointer.

Note:

See also:

cudaCtxEnablePeerAccess, cudaCtxDisablePeerAccess, cudaCtxCanAccessPeer, cuDeviceGetP2PAttribute

__host__cudaError_t cudaDeviceGetPCIBusId ( char* pciBusId, int  len, int  device )
Returns a PCI Bus Id string for the device.
Parameters
pciBusId
- Returned identifier string for the device in the following format [domain]:[bus]:[device].[function] where domain, bus, device, and function are all hexadecimal values. pciBusId should be large enough to store 13 characters including the NULL-terminator.
len
- Maximum length of string to store in name
device
- Device to get identifier string for
Description

Returns an ASCII string identifying the device dev in the NULL-terminated string pointed to by pciBusId. len specifies the maximum length of the string that may be returned.

Note:

See also:

cudaDeviceGetByPCIBusId, cuDeviceGetPCIBusId

__host____device__cudaError_t cudaDeviceGetSharedMemConfig ( cudaSharedMemConfig ** pConfig )
Returns the shared memory configuration for the current device.
Parameters
pConfig
- Returned cache configuration
Description

This function will return in pConfig the current size of shared memory banks on the current device. On devices with configurable shared memory banks, cudaDeviceSetSharedMemConfig can be used to change this setting, so that all subsequent kernel launches will by default use the new bank size. When cudaDeviceGetSharedMemConfig is called on devices without configurable shared memory, it will return the fixed bank size of the hardware.

The returned bank configurations can be either:

  • cudaSharedMemBankSizeFourByte - shared memory bank width is four bytes.

  • cudaSharedMemBankSizeEightByte - shared memory bank width is eight bytes.

Note:

See also:

cudaDeviceSetCacheConfig, cudaDeviceGetCacheConfig, cudaDeviceSetSharedMemConfig, cudaFuncSetCacheConfig, cuCtxGetSharedMemConfig

__host__cudaError_t cudaDeviceGetStreamPriorityRange ( int* leastPriority, int* greatestPriority )
Returns numerical values that correspond to the least and greatest stream priorities.
Parameters
leastPriority
- Pointer to an int in which the numerical value for least stream priority is returned
greatestPriority
- Pointer to an int in which the numerical value for greatest stream priority is returned
Returns

cudaSuccess

Description

Returns in *leastPriority and *greatestPriority the numerical values that correspond to the least and greatest stream priorities respectively. Stream priorities follow a convention where lower numbers imply greater priorities. The range of meaningful stream priorities is given by [*greatestPriority, *leastPriority]. If the user attempts to create a stream with a priority value that is outside the the meaningful range as specified by this API, the priority is automatically clamped down or up to either *leastPriority or *greatestPriority respectively. See cudaStreamCreateWithPriority for details on creating a priority stream. A NULL may be passed in for *leastPriority or *greatestPriority if the value is not desired.

This function will return '0' in both *leastPriority and *greatestPriority if the current context's device does not support stream priorities (see cudaDeviceGetAttribute).

Note:

See also:

cudaStreamCreateWithPriority, cudaStreamGetPriority, cuCtxGetStreamPriorityRange

__host__cudaError_t cudaDeviceReset ( void )
Destroy all allocations and reset all state on the current device in the current process.
Returns

cudaSuccess

Description

Explicitly destroys and cleans up all resources associated with the current device in the current process. Any subsequent API call to this device will reinitialize the device.

Note that this function will reset the device immediately. It is the caller's responsibility to ensure that the device is not being accessed by any other host threads from the process when this function is called.

Note:

See also:

cudaDeviceSynchronize

__host__cudaError_t cudaDeviceSetCacheConfig ( cudaFuncCache cacheConfig )
Sets the preferred cache configuration for the current device.
Parameters
cacheConfig
- Requested cache configuration
Returns

cudaSuccess

Description

On devices where the L1 cache and shared memory use the same hardware resources, this sets through cacheConfig the preferred cache configuration for the current device. This is only a preference. The runtime will use the requested configuration if possible, but it is free to choose a different configuration if required to execute the function. Any function preference set via cudaFuncSetCacheConfig ( C API) or cudaFuncSetCacheConfig ( C++ API) will be preferred over this device-wide setting. Setting the device-wide cache configuration to cudaFuncCachePreferNone will cause subsequent kernel launches to prefer to not change the cache configuration unless required to launch the kernel.

This setting does nothing on devices where the size of the L1 cache and shared memory are fixed.

Launching a kernel with a different preference than the most recent preference setting may insert a device-side synchronization point.

The supported cache configurations are:

Note:

See also:

cudaDeviceGetCacheConfig, cudaFuncSetCacheConfig ( C API), cudaFuncSetCacheConfig ( C++ API), cuCtxSetCacheConfig

__host__cudaError_t cudaDeviceSetLimit ( cudaLimit limit, size_t value )
Set resource limits.
Parameters
limit
- Limit to set
value
- Size of limit
Description

Setting limit to value is a request by the application to update the current limit maintained by the device. The driver is free to modify the requested value to meet h/w requirements (this could be clamping to minimum or maximum values, rounding up to nearest element size, etc). The application can use cudaDeviceGetLimit() to find out exactly what the limit has been set to.

Setting each cudaLimit has its own specific restrictions, so each is discussed here.

  • cudaLimitDevRuntimeSyncDepth controls the maximum nesting depth of a grid at which a thread can safely call cudaDeviceSynchronize(). Setting this limit must be performed before any launch of a kernel that uses the device runtime and calls cudaDeviceSynchronize() above the default sync depth, two levels of grids. Calls to cudaDeviceSynchronize() will fail with error code cudaErrorSyncDepthExceeded if the limitation is violated. This limit can be set smaller than the default or up the maximum launch depth of 24. When setting this limit, keep in mind that additional levels of sync depth require the runtime to reserve large amounts of device memory which can no longer be used for user allocations. If these reservations of device memory fail, cudaDeviceSetLimit will return cudaErrorMemoryAllocation, and the limit can be reset to a lower value. This limit is only applicable to devices of compute capability 3.5 and higher. Attempting to set this limit on devices of compute capability less than 3.5 will result in the error cudaErrorUnsupportedLimit being returned.

  • cudaLimitDevRuntimePendingLaunchCount controls the maximum number of outstanding device runtime launches that can be made from the current device. A grid is outstanding from the point of launch up until the grid is known to have been completed. Device runtime launches which violate this limitation fail and return cudaErrorLaunchPendingCountExceeded when cudaGetLastError() is called after launch. If more pending launches than the default (2048 launches) are needed for a module using the device runtime, this limit can be increased. Keep in mind that being able to sustain additional pending launches will require the runtime to reserve larger amounts of device memory upfront which can no longer be used for allocations. If these reservations fail, cudaDeviceSetLimit will return cudaErrorMemoryAllocation, and the limit can be reset to a lower value. This limit is only applicable to devices of compute capability 3.5 and higher. Attempting to set this limit on devices of compute capability less than 3.5 will result in the error cudaErrorUnsupportedLimit being returned.

  • cudaLimitMaxL2FetchGranularity controls the L2 cache fetch granularity. Values can range from 0B to 128B. This is purely a performance hint and it can be ignored or clamped depending on the platform.

  • cudaLimitPersistingL2CacheSize controls size of window in bytes available for cudaAccessPolicyWindow. This is purely a performance hint and it can be ignored or clamped depending on the platform.

Note:

See also:

cudaDeviceGetLimit, cuCtxSetLimit

__host__cudaError_t cudaDeviceSetSharedMemConfig ( cudaSharedMemConfig config )
Sets the shared memory configuration for the current device.
Parameters
config
- Requested cache configuration
Description

On devices with configurable shared memory banks, this function will set the shared memory bank size which is used for all subsequent kernel launches. Any per-function setting of shared memory set via cudaFuncSetSharedMemConfig will override the device wide setting.

Changing the shared memory configuration between launches may introduce a device side synchronization point.

Changing the shared memory bank size will not increase shared memory usage or affect occupancy of kernels, but may have major effects on performance. Larger bank sizes will allow for greater potential bandwidth to shared memory, but will change what kinds of accesses to shared memory will result in bank conflicts.

This function will do nothing on devices with fixed shared memory bank size.

The supported bank configurations are:

  • cudaSharedMemBankSizeDefault: set bank width the device default (currently, four bytes)

  • cudaSharedMemBankSizeFourByte: set shared memory bank width to be four bytes natively.

  • cudaSharedMemBankSizeEightByte: set shared memory bank width to be eight bytes natively.

Note:

See also:

cudaDeviceSetCacheConfig, cudaDeviceGetCacheConfig, cudaDeviceGetSharedMemConfig, cudaFuncSetCacheConfig, cuCtxSetSharedMemConfig

__host____device__cudaError_t cudaDeviceSynchronize ( void )
Wait for compute device to finish.
Returns

cudaSuccess

Description

Blocks until the device has completed all preceding requested tasks. cudaDeviceSynchronize() returns an error if one of the preceding tasks has failed. If the cudaDeviceScheduleBlockingSync flag was set for this device, the host thread will block until the device has finished its work.

Note:

See also:

cudaDeviceReset, cuCtxSynchronize

__host____device__cudaError_t cudaGetDevice ( int* device )
Returns which device is currently being used.
Parameters
device
- Returns the device on which the active host thread executes the device code.
Description

Returns in *device the current device for the calling host thread.

Note:

See also:

cudaGetDeviceCount, cudaSetDevice, cudaGetDeviceProperties, cudaChooseDevice, cuCtxGetCurrent

__host____device__cudaError_t cudaGetDeviceCount ( int* count )
Returns the number of compute-capable devices.
Parameters
count
- Returns the number of devices with compute capability greater or equal to 2.0
Returns

cudaSuccess

Description

Returns in *count the number of devices with compute capability greater or equal to 2.0 that are available for execution.

Note:

See also:

cudaGetDevice, cudaSetDevice, cudaGetDeviceProperties, cudaChooseDevice, cuDeviceGetCount

__host__cudaError_t cudaGetDeviceFlags ( unsigned int* flags )
Gets the flags for the current device.
Parameters
flags
- Pointer to store the device flags
Description

Returns in flags the flags for the current device. If there is a current device for the calling thread, the flags for the device are returned. If there is no current device, the flags for the first device are returned, which may be the default flags. Compare to the behavior of cudaSetDeviceFlags.

Typically, the flags returned should match the behavior that will be seen if the calling thread uses a device after this call, without any change to the flags or current device inbetween by this or another thread. Note that if the device is not initialized, it is possible for another thread to change the flags for the current device before it is initialized. Additionally, when using exclusive mode, if this thread has not requested a specific device, it may use a device other than the first device, contrary to the assumption made by this function.

If a context has been created via the driver API and is current to the calling thread, the flags for that context are always returned.

Flags returned by this function may specifically include cudaDeviceMapHost even though it is not accepted by cudaSetDeviceFlags because it is implicit in runtime API flags. The reason for this is that the current context may have been created via the driver API in which case the flag is not implicit and may be unset.

Note:

See also:

cudaGetDevice, cudaGetDeviceProperties, cudaSetDevice, cudaSetDeviceFlags, cuCtxGetFlags, cuDevicePrimaryCtxGetState

__host__cudaError_t cudaGetDeviceProperties ( cudaDeviceProp* prop, int  device )
Returns information about the compute-device.
Parameters
prop
- Properties for the specified device
device
- Device number to get properties for
Description

Returns in *prop the properties of device dev. The cudaDeviceProp structure is defined as:

‎    struct cudaDeviceProp {
              char name[256];
              cudaUUID_t uuid;
              size_t totalGlobalMem;
              size_t sharedMemPerBlock;
              int regsPerBlock;
              int warpSize;
              size_t memPitch;
              int maxThreadsPerBlock;
              int maxThreadsDim[3];
              int maxGridSize[3];
              int clockRate;
              size_t totalConstMem;
              int major;
              int minor;
              size_t textureAlignment;
              size_t texturePitchAlignment;
              int deviceOverlap;
              int multiProcessorCount;
              int kernelExecTimeoutEnabled;
              int integrated;
              int canMapHostMemory;
              int computeMode;
              int maxTexture1D;
              int maxTexture1DMipmap;
              int maxTexture1DLinear;
              int maxTexture2D[2];
              int maxTexture2DMipmap[2];
              int maxTexture2DLinear[3];
              int maxTexture2DGather[2];
              int maxTexture3D[3];
              int maxTexture3DAlt[3];
              int maxTextureCubemap;
              int maxTexture1DLayered[2];
              int maxTexture2DLayered[3];
              int maxTextureCubemapLayered[2];
              int maxSurface1D;
              int maxSurface2D[2];
              int maxSurface3D[3];
              int maxSurface1DLayered[2];
              int maxSurface2DLayered[3];
              int maxSurfaceCubemap;
              int maxSurfaceCubemapLayered[2];
              size_t surfaceAlignment;
              int concurrentKernels;
              int ECCEnabled;
              int pciBusID;
              int pciDeviceID;
              int pciDomainID;
              int tccDriver;
              int asyncEngineCount;
              int unifiedAddressing;
              int memoryClockRate;
              int memoryBusWidth;
              int l2CacheSize;
              int persistingL2CacheMaxSize;
              int maxThreadsPerMultiProcessor;
              int streamPrioritiesSupported;
              int globalL1CacheSupported;
              int localL1CacheSupported;
              size_t sharedMemPerMultiprocessor;
              int regsPerMultiprocessor;
              int managedMemory;
              int isMultiGpuBoard;
              int multiGpuBoardGroupID;
              int singleToDoublePrecisionPerfRatio;
              int pageableMemoryAccess;
              int concurrentManagedAccess;
              int computePreemptionSupported;
              int canUseHostPointerForRegisteredMem;
              int cooperativeLaunch;
              int cooperativeMultiDeviceLaunch;
              int pageableMemoryAccessUsesHostPageTables;
              int directManagedMemAccessFromHost;
              int accessPolicyMaxWindowSize;
          }
where:

Note:

See also:

cudaGetDeviceCount, cudaGetDevice, cudaSetDevice, cudaChooseDevice, cudaDeviceGetAttribute, cuDeviceGetAttribute, cuDeviceGetName

__host__cudaError_t cudaIpcCloseMemHandle ( void* devPtr )
Close memory mapped with cudaIpcOpenMemHandle.
Parameters
devPtr
- Device pointer returned by cudaIpcOpenMemHandle
Description

Unmaps memory returnd by cudaIpcOpenMemHandle. The original allocation in the exporting process as well as imported mappings in other processes will be unaffected.

Any resources used to enable peer access will be freed if this is the last mapping using them.

IPC functionality is restricted to devices with support for unified addressing on Linux operating systems. IPC functionality is not supported on Tegra platforms.

Note:

See also:

cudaMalloc, cudaFree, cudaIpcGetEventHandle, cudaIpcOpenEventHandle, cudaIpcGetMemHandle, cudaIpcOpenMemHandle, cuIpcCloseMemHandle

__host__cudaError_t cudaIpcGetEventHandle ( cudaIpcEventHandle_t* handle, cudaEvent_t event )
Gets an interprocess handle for a previously allocated event.
Parameters
handle
- Pointer to a user allocated cudaIpcEventHandle in which to return the opaque event handle
event
- Event allocated with cudaEventInterprocess and cudaEventDisableTiming flags.
Description

Takes as input a previously allocated event. This event must have been created with the cudaEventInterprocess and cudaEventDisableTiming flags set. This opaque handle may be copied into other processes and opened with cudaIpcOpenEventHandle to allow efficient hardware synchronization between GPU work in different processes.

After the event has been been opened in the importing process, cudaEventRecord, cudaEventSynchronize, cudaStreamWaitEvent and cudaEventQuery may be used in either process. Performing operations on the imported event after the exported event has been freed with cudaEventDestroy will result in undefined behavior.

IPC functionality is restricted to devices with support for unified addressing on Linux operating systems. IPC functionality is not supported on Tegra platforms.

Note:

See also:

cudaEventCreate, cudaEventDestroy, cudaEventSynchronize, cudaEventQuery, cudaStreamWaitEvent, cudaIpcOpenEventHandle, cudaIpcGetMemHandle, cudaIpcOpenMemHandle, cudaIpcCloseMemHandle, cuIpcGetEventHandle

__host__cudaError_t cudaIpcGetMemHandle ( cudaIpcMemHandle_t* handle, void* devPtr )
Gets an interprocess memory handle for an existing device memory allocation.
Parameters
handle
- Pointer to user allocated cudaIpcMemHandle to return the handle in.
devPtr
- Base pointer to previously allocated device memory
Description

Takes a pointer to the base of an existing device memory allocation created with cudaMalloc and exports it for use in another process. This is a lightweight operation and may be called multiple times on an allocation without adverse effects.

If a region of memory is freed with cudaFree and a subsequent call to cudaMalloc returns memory with the same device address, cudaIpcGetMemHandle will return a unique handle for the new memory.

IPC functionality is restricted to devices with support for unified addressing on Linux operating systems. IPC functionality is not supported on Tegra platforms.

Note:

See also:

cudaMalloc, cudaFree, cudaIpcGetEventHandle, cudaIpcOpenEventHandle, cudaIpcOpenMemHandle, cudaIpcCloseMemHandle, cuIpcGetMemHandle

__host__cudaError_t cudaIpcOpenEventHandle ( cudaEvent_t* event, cudaIpcEventHandle_t handle )
Opens an interprocess event handle for use in the current process.
Parameters
event
- Returns the imported event
handle
- Interprocess handle to open
Description

Opens an interprocess event handle exported from another process with cudaIpcGetEventHandle. This function returns a cudaEvent_t that behaves like a locally created event with the cudaEventDisableTiming flag specified. This event must be freed with cudaEventDestroy.

Performing operations on the imported event after the exported event has been freed with cudaEventDestroy will result in undefined behavior.

IPC functionality is restricted to devices with support for unified addressing on Linux operating systems. IPC functionality is not supported on Tegra platforms.

Note:

See also:

cudaEventCreate, cudaEventDestroy, cudaEventSynchronize, cudaEventQuery, cudaStreamWaitEvent, cudaIpcGetEventHandle, cudaIpcGetMemHandle, cudaIpcOpenMemHandle, cudaIpcCloseMemHandle, cuIpcOpenEventHandle

__host__cudaError_t cudaIpcOpenMemHandle ( void** devPtr, cudaIpcMemHandle_t handle, unsigned int  flags )
Opens an interprocess memory handle exported from another process and returns a device pointer usable in the local process.
Parameters
devPtr
- Returned device pointer
handle
- cudaIpcMemHandle to open
flags
- Flags for this operation. Must be specified as cudaIpcMemLazyEnablePeerAccess
Description

Maps memory exported from another process with cudaIpcGetMemHandle into the current device address space. For contexts on different devices cudaIpcOpenMemHandle can attempt to enable peer access between the devices as if the user called cudaDeviceEnablePeerAccess. This behavior is controlled by the cudaIpcMemLazyEnablePeerAccess flag. cudaDeviceCanAccessPeer can determine if a mapping is possible.

cudaIpcOpenMemHandle can open handles to devices that may not be visible in the process calling the API.

Contexts that may open cudaIpcMemHandles are restricted in the following way. cudaIpcMemHandles from each device in a given process may only be opened by one context per device per other process.

Memory returned from cudaIpcOpenMemHandle must be freed with cudaIpcCloseMemHandle.

Calling cudaFree on an exported memory region before calling cudaIpcCloseMemHandle in the importing context will result in undefined behavior.

IPC functionality is restricted to devices with support for unified addressing on Linux operating systems. IPC functionality is not supported on Tegra platforms.

Note:

See also:

cudaMalloc, cudaFree, cudaIpcGetEventHandle, cudaIpcOpenEventHandle, cudaIpcGetMemHandle, cudaIpcCloseMemHandle, cudaDeviceEnablePeerAccess, cudaDeviceCanAccessPeer, cuIpcOpenMemHandle

__host__cudaError_t cudaSetDevice ( int  device )
Set device to be used for GPU executions.
Parameters
device
- Device on which the active host thread should execute the device code.
Description

Sets device as the current device for the calling host thread. Valid device id's are 0 to (cudaGetDeviceCount() - 1).

Any device memory subsequently allocated from this host thread using cudaMalloc(), cudaMallocPitch() or cudaMallocArray() will be physically resident on device. Any host memory allocated from this host thread using cudaMallocHost() or cudaHostAlloc() or cudaHostRegister() will have its lifetime associated with device. Any streams or events created from this host thread will be associated with device. Any kernels launched from this host thread using the <<<>>> operator or cudaLaunchKernel() will be executed on device.

This call may be made from any host thread, to any device, and at any time. This function will do no synchronization with the previous or new device, and should be considered a very low overhead call.

Note:

See also:

cudaGetDeviceCount, cudaGetDevice, cudaGetDeviceProperties, cudaChooseDevice, cuCtxSetCurrent

__host__cudaError_t cudaSetDeviceFlags ( unsigned int  flags )
Sets flags to be used for device executions.
Parameters
flags
- Parameters for device operation
Description

Records flags as the flags for the current device. If the current device has been set and that device has already been initialized, the previous flags are overwritten. If the current device has not been initialized, it is initialized with the provided flags. If no device has been made current to the calling thread, a default device is selected and initialized with the provided flags.

The two LSBs of the flags parameter can be used to control how the CPU thread interacts with the OS scheduler when waiting for results from the device.

  • cudaDeviceScheduleAuto: The default value if the flags parameter is zero, uses a heuristic based on the number of active CUDA contexts in the process C and the number of logical processors in the system P. If C > P, then CUDA will yield to other OS threads when waiting for the device, otherwise CUDA will not yield while waiting for results and actively spin on the processor. Additionally, on Tegra devices, cudaDeviceScheduleAuto uses a heuristic based on the power profile of the platform and may choose cudaDeviceScheduleBlockingSync for low-powered devices.

  • cudaDeviceScheduleSpin: Instruct CUDA to actively spin when waiting for results from the device. This can decrease latency when waiting for the device, but may lower the performance of CPU threads if they are performing work in parallel with the CUDA thread.

  • cudaDeviceScheduleYield: Instruct CUDA to yield its thread when waiting for results from the device. This can increase latency when waiting for the device, but can increase the performance of CPU threads performing work in parallel with the device.

  • cudaDeviceScheduleBlockingSync: Instruct CUDA to block the CPU thread on a synchronization primitive when waiting for the device to finish work.

  • cudaDeviceBlockingSync: Instruct CUDA to block the CPU thread on a synchronization primitive when waiting for the device to finish work.

    Deprecated: This flag was deprecated as of CUDA 4.0 and replaced with cudaDeviceScheduleBlockingSync.

  • cudaDeviceMapHost: This flag enables allocating pinned host memory that is accessible to the device. It is implicit for the runtime but may be absent if a context is created using the driver API. If this flag is not set, cudaHostGetDevicePointer() will always return a failure code.

  • cudaDeviceLmemResizeToMax: Instruct CUDA to not reduce local memory after resizing local memory for a kernel. This can prevent thrashing by local memory allocations when launching many kernels with high local memory usage at the cost of potentially increased memory usage.

    Deprecated: This flag is deprecated and the behavior enabled by this flag is now the default and cannot be disabled.

Note:

See also:

cudaGetDeviceFlags, cudaGetDeviceCount, cudaGetDevice, cudaGetDeviceProperties, cudaSetDevice, cudaSetValidDevices, cudaChooseDevice, cuDevicePrimaryCtxSetFlags

__host__cudaError_t cudaSetValidDevices ( int* device_arr, int  len )
Set a list of devices that can be used for CUDA.
Parameters
device_arr
- List of devices to try
len
- Number of devices in specified list
Description

Sets a list of devices for CUDA execution in priority order using device_arr. The parameter len specifies the number of elements in the list. CUDA will try devices from the list sequentially until it finds one that works. If this function is not called, or if it is called with a len of 0, then CUDA will go back to its default behavior of trying devices sequentially from a default list containing all of the available CUDA devices in the system. If a specified device ID in the list does not exist, this function will return cudaErrorInvalidDevice. If len is not 0 and device_arr is NULL or if len exceeds the number of devices in the system, then cudaErrorInvalidValue is returned.

Note:

See also:

cudaGetDeviceCount, cudaSetDevice, cudaGetDeviceProperties, cudaSetDeviceFlags, cudaChooseDevice