MultiBufferDataSource#

public class MultiBufferDataSource extends DataSource #

This is a DataSource that can take multiple HostMemoryBuffers.

They are treated as if they are all part of a single file connected end to end.

Constructor Summary

`MultiBufferDataSource(HostMemoryAllocator allocator, HostMemoryBuffer ... buffers)`	Create a new data source backed by multiple buffers.
`MultiBufferDataSource(HostMemoryBuffer ... buffers)`	Create a new data source backed by multiple buffers.

Method Summary

`close()`
`deviceRead(long offset, DeviceMemoryBuffer dest, Cuda.Stream stream)`	Read data from the source at the given offset into dest.
`getDevReadBytes()`
`getDevReads()`
`getHostReadBytes()`
`getHostReads()`
`hostRead(long offset, HostMemoryBuffer dest)`	Read data from the source at the given offset into dest.
`hostRead(long offset, long amount)`	Read data from the source at the given offset.
`size()`	Get the size of the source in bytes.
`supportsDeviceRead()`	Return true if this supports reading directly to the device else false.

Methods inherited from class ai.rapids.cudf.DataSource: getDeviceReadCutoff, onHostBufferDone

Constructors

public MultiBufferDataSource(HostMemoryBuffer ... buffers)#

Create a new data source backed by multiple buffers.

Parameters:: buffers – the buffers that will back the data source.

public MultiBufferDataSource( HostMemoryAllocator allocator, HostMemoryBuffer ... buffers )#

Create a new data source backed by multiple buffers.

Parameters:

allocator – the allocator to use for host buffers, if needed.
buffers – the buffers that will back the data source.

Methods

public long size()#: Get the size of the source in bytes.

public HostMemoryBuffer hostRead(long offset, long amount)#

Read data from the source at the given offset.

Return a HostMemoryBuffer for the data that was read.

Parameters:

offset – where to start reading from.
amount – the maximum number of bytes to read.

Throws:

IOException – on any error.

Returns:

a buffer that points to the data.

public long hostRead(long offset, HostMemoryBuffer dest)#

Read data from the source at the given offset into dest.

Note that dest should not be closed, and no reference to it can outlive the call to hostRead. The target amount to read is dest’s length.

Parameters:

offset – the offset to start reading from in the source.
dest – where to write the data.

Returns:

the actual number of bytes written to dest.

public boolean supportsDeviceRead()#

Return true if this supports reading directly to the device else false.

The default is no device support. This cannot change dynamically. It is typically read just once.

public long deviceRead( long offset, DeviceMemoryBuffer dest, Cuda.Stream stream )#

Read data from the source at the given offset into dest.

Note that dest should not be closed, and no reference to it can outlive the call to hostRead. The target amount to read is dest’s length.

Parameters:

offset – the offset to start reading from
dest – where to write the data.
stream – the stream to do the copy on.

Returns:

the actual number of bytes written to dest.

public void close()#

public long getHostReads()#

public long getHostReadBytes()#

public long getDevReads()#

public long getDevReadBytes()#