Cuda#

public class Cuda#

Java bindings for a subset of the CUDA runtime API.

Includes device management, memory copies and sets, and the Cuda.Stream and Cuda.Event wrappers.

Field Summary

DEFAULT_STREAM

Method Summary

`asyncMemset(long dst, byte value, long count)`	Sets count bytes starting at the memory area pointed to by dst, with value.
`autoSetDevice()`	Set the device for this thread to the appropriate one.
`deviceSynchronize()`	Synchronizes the whole device using cudaDeviceSynchronize.
`freeZero()`	Calls cudaFree(0).
`getComputeCapabilityMajor()`	Gets the major CUDA compute capability of the current device.
`getComputeCapabilityMinor()`	Gets the minor CUDA compute capability of the current device.
`getComputeMode()`	Gets the CUDA compute mode of the current device.
`getDevice()`	Get the id of the current device.
`getDeviceCount()`	Get the device count.
`getDriverVersion()`	Get the CUDA Driver version, which is the latest version of CUDA supported by the driver.
`getGpuUuid()`	Gets the GPU UUID of the current device.
`getRuntimeVersion()`	Get the CUDA Runtime version of the current CUDA Runtime instance.
`isEnvCompatibleForTesting()`	This should only be used for tests, to enable or disable tests if the current environment is not compatible with this version of the library.
`isPtdsEnabled()`	Whether per-thread default stream is enabled.
`memGetInfo()`	Mapping: cudaMemGetInfo(size_t free, size_t total).
`memset(long dst, byte value, long count)`	Sets count bytes starting at the memory area pointed to by dst, with value.
`multiBufferCopyAsync(long[] destAddrs, long[] srcAddrs, long[] copySizes, Stream stream)`	Copy data from multiple device buffer sources to multiple device buffer destinations.
`profilerStart()`	Begins an Nsight profiling session, if a profiler is currently attached.
`profilerStop()`	Stops an active Nsight profiling session.
`setDevice(int device)`	Set the id of the current device.

Static Fields

public static final Stream DEFAULT_STREAM#

Static Methods

public static CudaComputeMode getComputeMode()#

Gets the CUDA compute mode of the current device.

Returns:: the enum value of CudaComputeMode

public static byte[] getGpuUuid()#

Gets the GPU UUID of the current device.

Returns:: UUID of the current device as a byte array.

public static CudaMemInfo memGetInfo() throws CudaException #: Mapping: cudaMemGetInfo(size_t *free, size_t *total).

public static void memset( long dst, byte value, long count ) throws CudaException #

Sets count bytes starting at the memory area pointed to by dst, with value.

The operation has completed when this returns, but it could overlap with operations occurring on other streams.

Parameters:

dst – - Destination memory address
value – - Byte value to set dst with
count – - Size in bytes to set

public static void asyncMemset( long dst, byte value, long count ) throws CudaException #

Sets count bytes starting at the memory area pointed to by dst, with value.

The operation has not necessarily completed when this returns, but it could overlap with operations occurring on other streams.

Parameters:

dst – - Destination memory address
value – - Byte value to set dst with
count – - Size in bytes to set

public static int getDevice() throws CudaException #

Get the id of the current device.

Throws:: CudaException – on any error
Returns:: the id of the current device

public static int getDeviceCount() throws CudaException #

Get the device count.

Throws:: CudaException – on any error
Returns:: returns the number of compute-capable devices

public static void setDevice( int device ) throws CudaException, CudfException #

Set the id of the current device.

Note this is relative to CUDA_SET_VISIBLE_DEVICES, e.g. if CUDA_SET_VISIBLE_DEVICES=1,0, and you call setDevice(0), you will get device 1.

Note if RMM has been initialized and the requested device ID does not match the device used to initialize RMM then this will throw an error.

Throws:: CudaException – on any error

public static void autoSetDevice() throws CudaException #

Set the device for this thread to the appropriate one.

Java loves threads, but cuda requires each thread to have the device set explicitly or it falls back to CUDA_VISIBLE_DEVICES. Most JNI calls through the cudf API will do this for you, but if you are writing your own JNI calls that extend cudf you might want to call this before calling into your JNI APIs to ensure that the device is set correctly.

Throws:: CudaException – on any error

public static int getDriverVersion() throws CudaException #

Get the CUDA Driver version, which is the latest version of CUDA supported by the driver.

The version is returned as (1000 major + 10 minor). For example, CUDA 9.2 would be represented by 9020. If no driver is installed,then 0 is returned as the driver version.

Throws:: CudaException – on any error
Returns:: the CUDA driver version

public static int getRuntimeVersion() throws CudaException #

Get the CUDA Runtime version of the current CUDA Runtime instance.

The version is returned as (1000 major + 10 minor). For example, CUDA 9.2 would be represented by 9020.

Throws:: CudaException – on any error
Returns:: the CUDA Runtime version

public static int getComputeCapabilityMajor() throws CudaException #

Gets the major CUDA compute capability of the current device.

For reference: https://developer.nvidia.com/cuda/gpus Hardware Generation Compute Capability Ampere 8.x Turing 7.5 Volta 7.0, 7.2 Pascal 6.x Maxwell 5.x Kepler 3.x Fermi 2.x

Throws:: CudaException – on any error
Returns:: The Major compute capability version number of the current CUDA device

public static int getComputeCapabilityMinor() throws CudaException #

Gets the minor CUDA compute capability of the current device.

For reference: https://developer.nvidia.com/cuda/gpus Hardware Generation Compute Capability Ampere 8.x Turing 7.5 Volta 7.0, 7.2 Pascal 6.x Maxwell 5.x Kepler 3.x Fermi 2.x

Throws:: CudaException – on any error
Returns:: The Minor compute capability version number of the current CUDA device

public static void freeZero() throws CudaException #

Calls cudaFree(0).

This can be used to initialize the GPU after a setDevice()

Throws:: CudaException – on any error

public static boolean isEnvCompatibleForTesting()#

This should only be used for tests, to enable or disable tests if the current environment is not compatible with this version of the library.

Currently it only does some very basic checks, but these may be expanded in the future depending on needs.

Returns:: true if it is compatible else false.

public static boolean isPtdsEnabled()#: Whether per-thread default stream is enabled.

public static void multiBufferCopyAsync( long[] destAddrs, long[] srcAddrs, long[] copySizes, Stream stream )#

Copy data from multiple device buffer sources to multiple device buffer destinations.

For each buffer to copy there is a corresponding entry in the destination address, source address, and copy size vectors.

Parameters:

destAddrs – vector of device destination addresses
srcAddrs – vector of device source addresses
copySizes – vector of copy sizes
stream – CUDA stream to use for the copy

public static void profilerStart()#: Begins an Nsight profiling session, if a profiler is currently attached.

Note

if a profiler session has a already started, profilerStart has no effect.

public static void profilerStop()#: Stops an active Nsight profiling session.

Note

if a profiler session isn’t active, profilerStop has no effect.

public static void deviceSynchronize()#: Synchronizes the whole device using cudaDeviceSynchronize.

Note

this is very expensive and should almost never be used

public static final class Stream implements AutoCloseable #

A class representing a CUDA stream.

Constructor Summary

Stream(boolean isNonBlocking)

Create a new CUDA stream.

Method Summary

`close()`
`getStream()`
`sync()`	Block the thread to wait until all pending work on this stream completes.
`toString()`
`waitOn(Event event)`	Have this stream not execute new work until the work recorded in event completes.

Constructors

public Stream(boolean isNonBlocking)#

Create a new CUDA stream.

Parameters:: isNonBlocking – whether stream should be non-blocking with respect to the default stream

Methods

public void waitOn(Event event)#

Have this stream not execute new work until the work recorded in event completes.

Parameters:: event – the event to wait on.

public long getStream()#

public void sync()#

Block the thread to wait until all pending work on this stream completes.

Note that this does not follow any of the java threading standards. Interrupt will not work to wake up the thread.

public String toString()#

public void close()#

public static final class Event implements AutoCloseable #

Constructor Summary

`Event()`	Create an event that is as fast as possible, timing is disabled and no blockingSync.
`Event(boolean enableTiming, boolean blockingSync)`	Create an event to be used for CUDA synchronization.

Method Summary

`close()`
`hasCompleted()`	Check to see if the event has completed or not.
`record()`	Captures the contents of the default stream at the time of this call.
`record(Stream stream)`	Captures the contents of stream at the time of this call.
`sync()`	Block the thread to wait for the event to complete.
`toString()`

Constructors

public Event()#: Create an event that is as fast as possible, timing is disabled and no blockingSync.

public Event(boolean enableTiming, boolean blockingSync)#

Create an event to be used for CUDA synchronization.

Parameters:

enableTiming – true if the event should record timing information.
blockingSync – true if event should use blocking synchronization. A host thread that calls sync() to wait on an event created with this flag will block until the event actually completes.

Methods

public boolean hasCompleted()#

Check to see if the event has completed or not.

This is the equivalent of cudaEventQuery.

Returns:: true it has completed else false.

public void record(Stream stream)#

Captures the contents of stream at the time of this call.

This event and stream must be on the same device. Calls such as hasCompleted() or Stream.waitEvent() will then examine or wait for completion of the work that was captured. Uses of stream after this call do not modify event.

Parameters:: stream – the stream to record the state of.

public void record()#: Captures the contents of the default stream at the time of this call.

public void sync()#

Block the thread to wait for the event to complete.

Note that this does not follow any of the java threading standards. Interrupt will not work to wake up the thread.

public String toString()#

public void close()#