HostColumnVectorCore#

public class HostColumnVectorCore implements AutoCloseable #

A class that holds Host side Column Vector APIs and the OffHeapState.

Any children of a HostColumnVector will be instantiated via this class.

Subclassed by ai.rapids.cudf.HostColumnVector

Field Summary

`children`
`nullCount`
`offHeap`
`rows`
`type`

Constructor Summary

HostColumnVectorCore(DType type, long rows, Optional<Long> nullCount, HostMemoryBuffer data, HostMemoryBuffer validity, HostMemoryBuffer offsets, List<HostColumnVectorCore> nestedChildren)

Method Summary

`close()`	Close method for the column.
`convertDecimal128FromJavaToCudf(byte[] bytes)`
`getBigDecimal(long index)`	Get the BigDecimal value at index.
`getBoolean(long index)`	Get the boolean value at index.
`getByte(long index)`	Get the value at index.
`getBytesFromList(long rowIndex)`	WARNING: Special case for lists of int8 or uint8, does not support null list values or lists.
`getChildColumnView(int childIndex)`
`getData()`	Returns the data buffer for a given host side column vector.
`getDouble(long index)`	Get the value at index.
`getEndListOffset(long index)`	Get the ending element offset for the list or string at index.
`getFloat(long index)`	Get the value at index.
`getHostBufferFor(BufferType type)`	Get access to the raw host buffer for this column.
`getHostMemorySize()`	Returns the amount of host memory used to store column/validity data (not metadata).
`getInt(long index)`	Get the value at index.
`getJavaString(long index)`	Get the value at index.
`getList(long rowIndex)`	WARNING: Strictly for test only.
`getLong(long index)`	Get the value at index.
`getNullCount()`	Returns the number of nulls in the data.
`getNumChildren()`	Returns the number of children for this column.
`getOffsets()`	Returns the offset buffer.
`getRowCount()`	Returns the number of rows for a given host side column vector.
`getShort(long index)`	Get the value at index.
`getStartListOffset(long index)`	Get the starting element offset for the list or string at index.
`getStruct(int rowIndex)`	WARNING: Strictly for test only.
`getType()`	Returns the type of this vector.
`getUTF8(long index)`	Get the raw UTF8 bytes at index.
`getValidity()`	Returns the validity buffer for a given host side column vector.
`hasNulls()`	Returns if the vector has nulls.
`hasValidityVector()`	Returns if the vector has a validity vector allocated or not.
`isNull(long rowIndex)`	Method that returns a boolean to indicate if the element at a given row index is null.
`toString()`

Protected Fields

protected final OffHeapState offHeap#

protected final DType type#

protected long rows#

protected Optional<Long> nullCount#

protected List<HostColumnVectorCore> children#

Constructors

public HostColumnVectorCore( DType type, long rows, Optional<Long> nullCount, HostMemoryBuffer data, HostMemoryBuffer validity, HostMemoryBuffer offsets, List<HostColumnVectorCore> nestedChildren )#

Methods

public DType getType()#: Returns the type of this vector.

public HostMemoryBuffer getData()#: Returns the data buffer for a given host side column vector.

public HostMemoryBuffer getValidity()#: Returns the validity buffer for a given host side column vector.

public HostMemoryBuffer getOffsets()#: Returns the offset buffer.

public HostColumnVectorCore getChildColumnView(int childIndex)#

public long getNullCount()#

Returns the number of nulls in the data.

Note that this might end up being a very expensive operation because if the null count is not known it will be calculated.

public long getRowCount()#: Returns the number of rows for a given host side column vector.

public int getNumChildren()#: Returns the number of children for this column.

public HostMemoryBuffer getHostBufferFor(BufferType type)#

Get access to the raw host buffer for this column.

This is intended to be used with a lot of caution. The lifetime of the buffer is tied to the lifetime of the column (Do not close the buffer, as the column will take care of it). Do not modify the contents of the buffer or it might negatively impact what happens on the column. The data must be on the host for this to work.

Parameters:: type – the type of buffer to get access to.
Returns:: the underlying buffer or null if no buffer is associated with it for this column. Please note that if the column is empty there may be no buffers at all associated with the column.

public byte getByte(long index)#: Get the value at index.

public final short getShort(long index)#: Get the value at index.

public final int getInt(long index)#: Get the value at index.

public long getStartListOffset(long index)#: Get the starting element offset for the list or string at index.

public long getEndListOffset(long index)#: Get the ending element offset for the list or string at index.

public final long getLong(long index)#: Get the value at index.

public final float getFloat(long index)#: Get the value at index.

public final double getDouble(long index)#: Get the value at index.

public final boolean getBoolean(long index)#: Get the boolean value at index.

public final BigDecimal getBigDecimal(long index)#: Get the BigDecimal value at index.

public byte[] getUTF8(long index)#

Get the raw UTF8 bytes at index.

This API is faster than getJavaString, but still not ideal because it is copying the data onto the heap.

public String getJavaString(long index)#

Get the value at index.

This API is slow as it has to translate the string representation. Please use it with caution.

public byte[] getBytesFromList(long rowIndex)#

WARNING: Special case for lists of int8 or uint8, does not support null list values or lists.

Get array of bytes at index from a list column of int8 or uint8. The column may not be a list of lists and may not have nulls.

public List getList(long rowIndex)#

WARNING: Strictly for test only.

This call is not efficient for production.

public HostColumnVector.StructData getStruct(int rowIndex)#

WARNING: Strictly for test only.

This call is not efficient for production.

public boolean isNull(long rowIndex)#

Method that returns a boolean to indicate if the element at a given row index is null.

Parameters:: rowIndex – the row index
Returns:: true if null else false

public boolean hasValidityVector()#: Returns if the vector has a validity vector allocated or not.

public boolean hasNulls()#

Returns if the vector has nulls.

Note that this might end up being a very expensive operation because if the null count is not known it will be calculated.

public long getHostMemorySize()#: Returns the amount of host memory used to store column/validity data (not metadata).

public void close()#: Close method for the column.

public String toString()#

Protected Static Methods

protected static byte[] convertDecimal128FromJavaToCudf(byte[] bytes)#

protected static final class OffHeapState extends MemoryCleaner.Cleaner #

Holds the off heap state of the column vector so we can clean it up, even if it is leaked.

Field Summary

`data`
`offsets`
`valid`

Method Summary

`cleanImpl(boolean logErrorIfNotClean)`	Clean up any resources not previously released.
`getHostMemorySize()`	This returns total memory allocated on the host for the ColumnVector.
`isClean()`	Check if the underlying memory has been cleaned up or not.
`noWarnLeakExpected()`
`toString()`

Fields inherited from class ai.rapids.cudf.MemoryCleaner.Cleaner: id

Methods inherited from class ai.rapids.cudf.MemoryCleaner.Cleaner: addRef, clean, delRef, isLeakExpected, logRefCountDebug

Fields

public HostMemoryBuffer data#

public HostMemoryBuffer valid#

public HostMemoryBuffer offsets#

Methods

public void noWarnLeakExpected()#

public boolean isClean()#

Check if the underlying memory has been cleaned up or not.

Returns:: true this is clean else false.

public long getHostMemorySize()#: This returns total memory allocated on the host for the ColumnVector.

public String toString()#

Protected Methods

protected boolean cleanImpl(boolean logErrorIfNotClean)#

Clean up any resources not previously released.

Parameters:: logErrorIfNotClean – if true and there are resources to clean up a leak has happened so log it.
Returns:: true if resources were cleaned up else false.