ColumnVector#

public final class ColumnVector extends ColumnView#

This class represents the immutable vector of data.

This class holds references to device(GPU) memory and is reference counted to know when to release it. Call close to decrement the reference count when you are done with the column, and call incRefCount to increment the reference count.

Constructor Summary

`ColumnVector(DType type, long rows, Optional<Long> nullCount, DeviceMemoryBuffer dataBuffer, DeviceMemoryBuffer validityBuffer, DeviceMemoryBuffer offsetBuffer)`	Create a new column vector based off of data already on the device.
`ColumnVector(DType type, long rows, Optional<Long> nullCount, DeviceMemoryBuffer dataBuffer, DeviceMemoryBuffer validityBuffer, DeviceMemoryBuffer offsetBuffer, List<DeviceMemoryBuffer> toClose, long[] childHandles)`	Create a new column vector based off of data already on the device with child columns.
`ColumnVector(long nativePointer)`	Wrap an existing on device cudf::column with the corresponding ColumnVector.

Method Summary

`boolFromBytes(byte... values)`	Create a new vector from the given values.
`build(DType type, int rows, Consumer<Builder> init)`	Create a new vector.
`build(int rows, long stringBufferSize, Consumer<Builder> init)`
`castTo(DType type)`	Generic method to cast ColumnVector When casting from a Date, Timestamp, or Boolean to a numerical type the underlying numerical representation of the data will be used for the cast.
`close()`	Close this Vector and free memory allocated for HostMemoryBuffer and DeviceMemoryBuffer.
`concatenate(ColumnView... columns)`	Create a new vector by concatenating multiple columns together.
`copyToColumnVector()`	For a ColumnVector this is really just incrementing the reference count.
`daysFromInts(int... values)`	Create a new vector from the given values.
`decimalFromBigInt(int scale, BigInteger... values)`	Create a new decimal vector from BigIntegers Compared with scale of java.math.BigDecimal, the scale here represents the opposite meaning.
`decimalFromBoxedInts(int scale, Integer... values)`	Create a new decimal vector from boxed unscaled values (Integer array) and scale.
`decimalFromBoxedLongs(int scale, Long... values)`	Create a new decimal vector from boxed unscaled values (Long array) and scale.
`decimalFromDoubles(DType type, RoundingMode mode, double... values)`	Create a new decimal vector from double floats with specific DecimalType and RoundingMode.
`decimalFromInts(int scale, int... values)`	Create a new decimal vector from unscaled values (int array) and scale.
`decimalFromLongs(int scale, long... values)`	Create a new decimal vector from unscaled values (long array) and scale.
`durationDaysFromBoxedInts(Integer... values)`	Create a new vector from the given values.
`durationDaysFromInts(int... values)`	Create a new vector from the given values.
`durationMicroSecondsFromBoxedLongs(Long... values)`	Create a new vector from the given values.
`durationMicroSecondsFromLongs(long... values)`	Create a new vector from the given values.
`durationMilliSecondsFromBoxedLongs(Long... values)`	Create a new vector from the given values.
`durationMilliSecondsFromLongs(long... values)`	Create a new vector from the given values.
`durationNanoSecondsFromBoxedLongs(Long... values)`	Create a new vector from the given values.
`durationNanoSecondsFromLongs(long... values)`	Create a new vector from the given values.
`durationSecondsFromBoxedLongs(Long... values)`	Create a new vector from the given values.
`durationSecondsFromLongs(long... values)`	Create a new vector from the given values.
`empty(HostColumnVector.DataType colType)`	Creates an empty column according to the data type.
`emptyStructs(HostColumnVector.DataType dataType, long numRows)`	This method is evolving, unstable and currently test only.
`fromArrow(DType type, long numRows, long nullCount, ByteBuffer data, ByteBuffer validity, ByteBuffer offsets)`	Create a ColumnVector from the Apache Arrow byte buffers passed in.
`fromBooleans(boolean... values)`	Create a new vector from the given values.
`fromBoxedBooleans(Boolean... values)`	Create a new vector from the given values.
`fromBoxedBytes(Byte... values)`	Create a new vector from the given values.
`fromBoxedDoubles(Double... values)`	Create a new vector from the given values.
`fromBoxedFloats(Float... values)`	Create a new vector from the given values.
`fromBoxedInts(Integer... values)`	Create a new vector from the given values.
`fromBoxedLongs(Long... values)`	Create a new vector from the given values.
`fromBoxedShorts(Short... values)`	Create a new vector from the given values.
`fromBoxedUnsignedBytes(Byte... values)`	Create a new vector from the given values.
`fromBoxedUnsignedInts(Integer... values)`	Create a new vector from the given values.
`fromBoxedUnsignedLongs(Long... values)`	Create a new vector from the given values.
`fromBoxedUnsignedShorts(Short... values)`	Create a new vector from the given values.
`fromBytes(byte... values)`	Create a new vector from the given values.
`fromDecimals(BigDecimal... values)`	Create a new vector from the given values.
`fromDoubles(double... values)`	Create a new vector from the given values.
`fromFloats(float... values)`	Create a new vector from the given values.
`fromInts(int... values)`	Create a new vector from the given values.
`fromLists(HostColumnVector.DataType dataType, List<T>... lists)`	This method is evolving, unstable and currently test only.
`fromLongs(long... values)`	Create a new vector from the given values.
`fromScalar(Scalar scalar, int rows)`	Create a new vector of length rows, where each row is filled with the Scalar’s value.
`fromShorts(short... values)`	Create a new vector from the given values.
`fromStrings(String... values)`	Create a new string vector from the given values.
`fromStructs(HostColumnVector.DataType dataType, HostColumnVector.StructData... lists)`	This method is evolving, unstable and currently test only.
`fromStructs(HostColumnVector.DataType dataType, List<HostColumnVector.StructData> lists)`	This method is evolving, unstable and currently test only.
`fromUnsignedBytes(byte... values)`	Create a new vector from the given values.
`fromUnsignedInts(int... values)`	Create a new vector from the given values.
`fromUnsignedLongs(long... values)`	Create a new vector from the given values.
`fromUnsignedShorts(short... values)`	Create a new vector from the given values.
`fromUTF8Strings(byte[]... values)`	Create a new string vector from the given values.
`fromViewWithContiguousAllocation(long columnViewAddress, DeviceMemoryBuffer buffer)`	Creates a ColumnVector from a native column_view using a contiguous device allocation.
`getDeviceBufferFor(BufferType type)`	Get access to the raw device buffer for this column.
`getEventHandler()`	Returns the current event handler for this ColumnVector or null if no handler is associated.
`getNullCount()`	Returns the number of nulls in the data.
`getRefCount()`	Returns this column’s current refcount.
`hasNulls()`	Returns if the vector has nulls.
`hasValidityVector()`	Returns if the vector has a validity vector allocated or not.
`incRefCount()`	Increment the reference count for this column.
`listConcatenateByRow(ColumnView... columns)`	Concatenate columns of lists horizontally (row by row), combining a corresponding row from each column into a single list row of a new column.
`listConcatenateByRow(boolean ignoreNull, ColumnView... columns)`	Concatenate columns of lists horizontally (row by row), combining a corresponding row from each column into a single list row of a new column.
`makeList(ColumnView... columns)`	Create a LIST column from the given columns.
`makeList(long rows, DType type, ColumnView... columns)`	Create a LIST column from the given columns.
`makeListFromOffsets(long rows, ColumnView offsets)`	Create a LIST column from the current column and a given offsets column.
`makeStruct(ColumnView... columns)`	Create a new struct vector made up of existing columns.
`makeStruct(long rows, ColumnView... columns)`	Create a new struct vector made up of existing columns.
`md5Hash(ColumnView... columns)`	Create a new vector containing the MD5 hash of each row in the table.
`noWarnLeakExpected()`	This is a really ugly API, but it is possible that the lifecycle of a column of data may not have a clear lifecycle thanks to java and GC.
`sequence(ColumnView start, ColumnView size)`	Create a list column in which each row is a sequence of values starting from a start value, incrementing by one, and its cardinality is specified by a size value.
`sequence(ColumnView start, ColumnView size, ColumnView step)`	Create a list column in which each row is a sequence of values starting from a start value, incrementing by a step value, and its cardinality is specified by a size value.
`sequence(Scalar initialValue, Scalar step, int rows)`	Create a new vector of length rows, starting at the initialValue and going by step each time.
`sequence(Scalar initialValue, int rows)`	Create a new vector of length rows, starting at the initialValue and going by 1 each time.
`setEventHandler(EventHandler newHandler)`	Set an event handler for this vector.
`sha1Hash(ColumnView... columns)`	Create a new column containing the Sha1 hash of each row in the table.
`stringConcatenate(ColumnView[] columns)`	Concatenate columns of strings together, combining a corresponding row from each column into a single string row of a new column with no separator string inserted between each combined string and maintaining null values in combined rows.
`stringConcatenate(ColumnView[] columns, ColumnView sepCol)`	Concatenate columns of strings together using a separator specified for each row and returns the result as a string column.
`stringConcatenate(ColumnView[] columns, ColumnView sepCol, Scalar separatorNarep, Scalar colNarep, boolean separateNulls)`	Concatenate columns of strings together using a separator specified for each row and returns the result as a string column.
`stringConcatenate(Scalar separator, Scalar narep, ColumnView[] columns)`	Concatenate columns of strings together, combining a corresponding row from each column into a single string row of a new column.
`stringConcatenate(Scalar separator, Scalar narep, ColumnView[] columns, boolean separateNulls)`	Concatenate columns of strings together, combining a corresponding row from each column into a single string row of a new column.
`timestampDaysFromBoxedInts(Integer... values)`	Create a new vector from the given values.
`timestampMicroSecondsFromBoxedLongs(Long... values)`	Create a new vector from the given values.
`timestampMicroSecondsFromLongs(long... values)`	Create a new vector from the given values.
`timestampMilliSecondsFromBoxedLongs(Long... values)`	Create a new vector from the given values.
`timestampMilliSecondsFromLongs(long... values)`	Create a new vector from the given values.
`timestampNanoSecondsFromBoxedLongs(Long... values)`	Create a new vector from the given values.
`timestampNanoSecondsFromLongs(long... values)`	Create a new vector from the given values.
`timestampSecondsFromBoxedLongs(Long... values)`	Create a new vector from the given values.
`timestampSecondsFromLongs(long... values)`	Create a new vector from the given values.
`toString()`

Fields inherited from class ai.rapids.cudf.ColumnView: offHeap, rows, type, UNKNOWN_NULL_COUNT, viewHandle

Methods inherited from class ai.rapids.cudf.ColumnView: abs, addCalendricalMonths, all, any, applyBooleanMask, applyRetentionMask, approxPercentile, arccos, arccosh, arcsin, arcsinh, arctan, arctanh, argMax, argMin, asByteList, asBytes, asDoubles, asFloats, asInts, asLongs, asShorts, asStrings, asTimestamp, asTimestampDays, asTimestampMicroseconds, asTimestampMilliseconds, asTimestampNanoseconds, asTimestampSeconds, asUnsignedBytes, asUnsignedInts, asUnsignedLongs, asUnsignedShorts, binaryOp, bitCastTo, bitCount, bitInvert, capitalize, cbrt, ceil, clamp, codePoints, contains, containsRe, copyToHost, copyToHostAsync, cos, cosh, countElements, dateTimeCeil, dateTimeFloor, dateTimeRound, day, dayOfYear, daysInMonth, distinctCount, dropListDuplicates, dropListDuplicatesWithKeysValues, endsWith, exp, extractAllRecord, extractDateTimeComponent, extractListElement, extractRe, findAndReplaceAll, flattenLists, floor, fromDeviceBuffer, generateListOffsets, getByteCount, getCharLengths, getChildColumnView, getChildColumnViews, getData, getDeviceMemorySize, getHostBytesRequired, getJSONObject, getListOffsetsView, getMapKeyExistence, getMapValue, getNativeView, getNumChildren, getOffsets, getRowCount, getScalarElement, getType, getValid, hasNonEmptyNulls, hostPaddingSizeInBytes, hour, ifElse, isFixedPoint, isFloat, isInteger, isLeapYear, isNan, isNotNan, isNotNull, isNull, isTimestamp, joinStrings, lastDayOfMonth, like, listContains, listContainsColumn, listContainsNulls, listIndexOf, listReduce, listsDifferenceDistinct, listsHaveOverlap, listsIntersectDistinct, listSortRows, listsUnionDistinct, log10, log2, logicalCastTo, lower, lstrip, makeStructView, matchesRe, max, mean, mergeAndSetValidity, min, minute, month, nansToNulls, normalizeNANsAndZeros, not, pad, prefixSum, product, purgeNonEmptyNulls, quantile, quarterOfYear, reduce, repeatStrings, replaceChildrenWithViews, replaceListChild, replaceNulls, replaceRegex, reverseStringsOrLists, rint, rollingWindow, rstrip, scan, second, segmentedGather, segmentedReduce, sin, sinh, slice, split, splitAsViews, sqrt, standardDeviation, startsWith, stringConcatenateListElements, stringContains, stringContainsPerRow, stringLocate, stringReplace, stringReplacePerRow, stringReplaceWithBackrefs, stringSplit, stringSplitRecord, strip, substring, subVector, sum, sumOfSquares, tan, tanh, title, toHex, toTitle, transform, unaryOp, upper, urlDecode, urlEncode, variance, weekDay, year, zfill

Methods inherited from interface ai.rapids.cudf.BinaryOperable: add, and, arctan2, bitAnd, bitOr, bitXor, div, equalTo, equalToNullAware, floorDiv, greaterOrEqualTo, greaterThan, lessOrEqualTo, lessThan, log, maxNullAware, minNullAware, mod, mul, notEqualTo, notEqualToNullAware, or, pmod, pow, shiftLeft, shiftRight, shiftRightUnsigned, sub, trueDiv

Constructors

public ColumnVector(long nativePointer)#

Wrap an existing on device cudf::column with the corresponding ColumnVector.

The new ColumnVector takes ownership of the pointer and will free it when the ref count reaches zero.

Parameters:: nativePointer – host address of the cudf::column object which will be owned by this instance.

public ColumnVector( DType type, long rows, Optional<Long> nullCount, DeviceMemoryBuffer dataBuffer, DeviceMemoryBuffer validityBuffer, DeviceMemoryBuffer offsetBuffer )#

Create a new column vector based off of data already on the device.

Parameters:

type – the type of the vector
rows – the number of rows in this vector.
nullCount – the number of nulls in the dataset.
dataBuffer – the data stored on the device. The column vector takes ownership of the buffer. Do not use the buffer after calling this.
validityBuffer – an optional validity buffer. Must be provided if nullCount != 0. The column vector takes ownership of the buffer. Do not use the buffer after calling this.
offsetBuffer – a host buffer required for strings and string categories. The column vector takes ownership of the buffer. Do not use the buffer after calling this.

public ColumnVector( DType type, long rows, Optional<Long> nullCount, DeviceMemoryBuffer dataBuffer, DeviceMemoryBuffer validityBuffer, DeviceMemoryBuffer offsetBuffer, List<DeviceMemoryBuffer> toClose, long[] childHandles )#

Create a new column vector based off of data already on the device with child columns.

Parameters:

type – the type of the vector, typically a nested type
rows – the number of rows in this vector.
nullCount – the number of nulls in the dataset.
dataBuffer – the data stored on the device. The column vector takes ownership of the buffer. Do not use the buffer after calling this.
validityBuffer – an optional validity buffer. Must be provided if nullCount != 0. The column vector takes ownership of the buffer. Do not use the buffer after calling this.
offsetBuffer – a host buffer required for strings and string categories. The column vector takes ownership of the buffer. Do not use the buffer after calling this.
toClose – List of buffers to track and close once done, usually in case of children
childHandles – array of longs for child column view handles.

Methods

public ColumnVector copyToColumnVector()#

For a ColumnVector this is really just incrementing the reference count.

Returns:: this

public EventHandler setEventHandler(EventHandler newHandler)#

Set an event handler for this vector.

This method can be invoked with null to unset the handler.

Parameters:: newHandler – - the EventHandler to use from this point forward
Returns:: the prior event handler, or null if not set.

public EventHandler getEventHandler()#: Returns the current event handler for this ColumnVector or null if no handler is associated.

public void noWarnLeakExpected()#

This is a really ugly API, but it is possible that the lifecycle of a column of data may not have a clear lifecycle thanks to java and GC.

This API informs the leak tracking code that this is expected for this column, and big scary warnings should not be printed when this happens.

public void close()#: Close this Vector and free memory allocated for HostMemoryBuffer and DeviceMemoryBuffer.

public String toString()#

public ColumnVector incRefCount()#

Increment the reference count for this column.

You need to call close on this to decrement the reference count again.

public long getNullCount()#

Returns the number of nulls in the data.

Note that this might end up being a very expensive operation because if the null count is not known it will be calculated.

public int getRefCount()#: Returns this column’s current refcount.

public boolean hasValidityVector()#: Returns if the vector has a validity vector allocated or not.

public boolean hasNulls()#

Returns if the vector has nulls.

Note that this might end up being a very expensive operation because if the null count is not known it will be calculated.

public BaseDeviceMemoryBuffer getDeviceBufferFor(BufferType type)#

Get access to the raw device buffer for this column.

This is intended to be used with a lot of caution. The lifetime of the buffer is tied to the lifetime of the column (Do not close the buffer, as the column will take care of it). Do not modify the contents of the buffer or it might negatively impact what happens on the column. The data must be on the device for this to work. Strings and string categories do not currently work because their underlying device layout is currently hidden.

Parameters:: type – the type of buffer to get access to.
Returns:: the underlying buffer or null if no buffer is associated with it for this column. Please note that if the column is empty there may be no buffers at all associated with the column.

public ColumnVector makeListFromOffsets(long rows, ColumnView offsets)#

Create a LIST column from the current column and a given offsets column.

The output column will contain lists having elements that are copied from the current column and their sizes are determined by the given offsets.

Note that the caller is responsible to make sure the given offsets column is of type INT32 and it contains valid indices to create a LIST column. There will not be any validity check for these offsets during calling to this function. If the given offsets are invalid, we may have bad memory accesses and/or data corruption.

Parameters:

rows – the number of rows to create.
offsets – the offsets pointing to row indices of the current column to create an output LIST column.

public ColumnVector castTo(DType type)#

Generic method to cast ColumnVector When casting from a Date, Timestamp, or Boolean to a numerical type the underlying numerical representation of the data will be used for the cast.

For Strings: Casting strings from/to timestamp isn’t supported atm. Please look at ColumnVector.asTimestamp(DType, String) and ColumnVector.asStrings(String) for casting string to timestamp when the format is known

Float values when converted to String could be different from the expected default behavior in Java e.g. 12.3 => “12.30000019” instead of “12.3” Double.POSITIVE_INFINITY => “Inf” instead of “INFINITY” Double.NEGATIVE_INFINITY => “-Inf” instead of “-INFINITY”

Parameters:: type – type of the resulting ColumnVector
Returns:: A new vector allocated on the GPU

public static<T> ColumnVector fromLists( HostColumnVector.DataType dataType, List<T>... lists )#

This method is evolving, unstable and currently test only.

Please use with caution and expect it to change in the future.

Static Methods

public static ColumnVector fromViewWithContiguousAllocation( long columnViewAddress, DeviceMemoryBuffer buffer )#

Creates a ColumnVector from a native column_view using a contiguous device allocation.

Parameters:

columnViewAddress – address of the native column_view
buffer – device buffer containing the data referenced by the column view

public static ColumnVector fromArrow( DType type, long numRows, long nullCount, ByteBuffer data, ByteBuffer validity, ByteBuffer offsets )#

Create a ColumnVector from the Apache Arrow byte buffers passed in.

Any of the buffers not used for that datatype should be set to null. The buffers are expected to be off heap buffers, but if they are not, it will handle copying them to direct byte buffers. This only supports primitive types. Strings, Decimals and nested types such as list and struct are not supported.

Parameters:

type – - type of the column
numRows – - Number of rows in the arrow column
nullCount – - Null count
data – - ByteBuffer of the Arrow data buffer
validity – - ByteBuffer of the Arrow validity buffer
offsets – - ByteBuffer of the Arrow offsets buffer

Returns:

- new ColumnVector

public static ColumnVector fromScalar(Scalar scalar, int rows)#

Create a new vector of length rows, where each row is filled with the Scalar’s value.

Parameters:

scalar – - Scalar to use to fill rows
rows – - Number of rows in the new ColumnVector

Returns:

- new ColumnVector

public static ColumnVector makeStruct(ColumnView... columns)#

Create a new struct vector made up of existing columns.

Note that this will copy the contents of the input columns to make a new vector. If you only want to do a quick temporary computation you can use ColumnView.makeStructView.

Parameters:: columns – the columns to make the struct from.
Returns:: the new ColumnVector

public static ColumnVector makeStruct( long rows, ColumnView... columns )#

Create a new struct vector made up of existing columns.

Note that this will copy the contents of the input columns to make a new vector. If you only want to do a quick temporary computation you can use ColumnView.makeStructView.

Parameters:

rows – the number of rows in the struct. Used for structs with no children.
columns – the columns to make the struct from.

Returns:

the new ColumnVector

public static ColumnVector makeList(ColumnView... columns)#

Create a LIST column from the given columns.

Each list in the returned column will have the same number of entries in it as columns passed into this method. Be careful about the number of rows passed in as there are limits on the maximum output size supported for column lists.

Parameters:: columns – the columns to make up the list column, in the order they will appear in the resulting lists.
Returns:: the new LIST ColumnVector

public static ColumnVector makeList( long rows, DType type, ColumnView... columns )#

Create a LIST column from the given columns.

Parameters:

rows – the number of rows to create, for the special case of an empty list.
type – the type of the child column, for the special case of an empty list.
columns – the columns to make up the list column, in the order they will appear in the resulting lists.

Returns:

the new LIST ColumnVector

public static ColumnVector sequence( Scalar initialValue, Scalar step, int rows )#

Create a new vector of length rows, starting at the initialValue and going by step each time.

Only numeric types are supported.

Parameters:

initialValue – the initial value to start at.
step – the step to add to each subsequent row.
rows – the total number of rows

Returns:

the new ColumnVector.

public static ColumnVector sequence(Scalar initialValue, int rows)#

Create a new vector of length rows, starting at the initialValue and going by 1 each time.

Only numeric types are supported.

Parameters:

initialValue – the initial value to start at.
rows – the total number of rows

Returns:

the new ColumnVector.

public static ColumnVector sequence(ColumnView start, ColumnView size)#

Create a list column in which each row is a sequence of values starting from a start value, incrementing by one, and its cardinality is specified by a size value.

The start and size values used to generate each list is taken from the corresponding row of the input start and size columns.

Parameters:

start – first values in the result sequences
size – numbers of values in the result sequences

Returns:

the new ColumnVector.

public static ColumnVector sequence( ColumnView start, ColumnView size, ColumnView step )#

Create a list column in which each row is a sequence of values starting from a start value, incrementing by a step value, and its cardinality is specified by a size value.

The values start, step, and size used to generate each list is taken from the corresponding row of the input starts, steps, and sizes columns.

Parameters:

start – first values in the result sequences
size – numbers of values in the result sequences
step – increment values for the result sequences.

Returns:

the new ColumnVector.

public static ColumnVector concatenate(ColumnView... columns)#

Create a new vector by concatenating multiple columns together.

Note that all columns must have the same type.

public static ColumnVector stringConcatenate(ColumnView[] columns)#

Concatenate columns of strings together, combining a corresponding row from each column into a single string row of a new column with no separator string inserted between each combined string and maintaining null values in combined rows.

Parameters:: columns – array of columns containing strings, must be non-empty
Returns:: A new java column vector containing the concatenated strings.

public static ColumnVector stringConcatenate( Scalar separator, Scalar narep, ColumnView[] columns )#

Concatenate columns of strings together, combining a corresponding row from each column into a single string row of a new column.

This version includes the separator for null rows if ‘narep’ is valid.

Parameters:

separator – string scalar inserted between each string being merged.
narep – string scalar indicating null behavior. If set to null and any string in the row is null the resulting string will be null. If not null, null values in any column will be replaced by the specified string.
columns – array of columns containing strings, must be non-empty

Returns:

A new java column vector containing the concatenated strings.

public static ColumnVector stringConcatenate( Scalar separator, Scalar narep, ColumnView[] columns, boolean separateNulls )#

Concatenate columns of strings together, combining a corresponding row from each column into a single string row of a new column.

Parameters:

separator – string scalar inserted between each string being merged.
narep – string scalar indicating null behavior. If set to null and any string in the row is null the resulting string will be null. If not null, null values in any column will be replaced by the specified string.
columns – array of columns containing strings, must be non-empty
separateNulls – if true, then the separator is included for null rows if narep is valid.

Returns:

A new java column vector containing the concatenated strings.

public static ColumnVector stringConcatenate( ColumnView[] columns, ColumnView sepCol )#

Concatenate columns of strings together using a separator specified for each row and returns the result as a string column.

If the row separator for a given row is null, output column for that row is null. Null column values for a given row are skipped.

Parameters:

columns – array of columns containing strings
sepCol – strings column that provides the separator for a given row

Returns:

A new java column vector containing the concatenated strings with separator between.

public static ColumnVector stringConcatenate( ColumnView[] columns, ColumnView sepCol, Scalar separatorNarep, Scalar colNarep, boolean separateNulls )#

Concatenate columns of strings together using a separator specified for each row and returns the result as a string column.

If the row separator for a given row is null, output column for that row is null unless separatorNarep is provided. The separator is applied between two output row values if the separateNulls is YES or only between valid rows if separateNulls is NO.

Parameters:

columns – array of columns containing strings
sepCol – strings column that provides the separator for a given row
separatorNarep – string scalar indicating null behavior when a separator is null. If set to null and the separator is null the resulting string will be null. If not null, this string will be used in place of a null separator.
colNarep – string that should be used in place of any null strings found in any column.
separateNulls – if true, then the separator is included for null rows if colNarep is valid.

Returns:

A new java column vector containing the concatenated strings with separator between.

public static ColumnVector listConcatenateByRow(ColumnView... columns)#

Concatenate columns of lists horizontally (row by row), combining a corresponding row from each column into a single list row of a new column.

NOTICE: Any concatenation involving a null list element will result in a null list.

Parameters:: columns – array of columns containing lists, must be non-empty
Returns:: A new java column vector containing the concatenated lists.

public static ColumnVector listConcatenateByRow( boolean ignoreNull, ColumnView... columns )#

Concatenate columns of lists horizontally (row by row), combining a corresponding row from each column into a single list row of a new column.

Parameters:

ignoreNull – whether to ignore null list element of input columns: If true, null list will be ignored from concatenation; Otherwise, any concatenation involving a null list element will result in a null list
columns – array of columns containing lists, must be non-empty

Returns:

A new java column vector containing the concatenated lists.

public static ColumnVector md5Hash(ColumnView... columns)#

Create a new vector containing the MD5 hash of each row in the table.

Parameters:: columns – array of columns to hash, must have identical number of rows.
Returns:: the new ColumnVector of 32 character hex strings representing each row’s hash value.

public static ColumnVector sha1Hash(ColumnView... columns)#

Create a new column containing the Sha1 hash of each row in the table.

Parameters:: columns – columns to hash
Returns:: the new ColumnVector of 40 character hex strings representing each row’s hash value.

public static ColumnVector build( DType type, int rows, Consumer<Builder> init )#

Create a new vector.

Parameters:

type – the type of vector to build.
rows – maximum number of rows that the vector can hold.
init – what will initialize the vector.

Returns:

the created vector.

public static ColumnVector build( int rows, long stringBufferSize, Consumer<Builder> init )#

public static ColumnVector boolFromBytes(byte... values)#: Create a new vector from the given values.

public static ColumnVector fromStructs( HostColumnVector.DataType dataType, List<HostColumnVector.StructData> lists )#

This method is evolving, unstable and currently test only.

Please use with caution and expect it to change in the future.

public static ColumnVector fromStructs( HostColumnVector.DataType dataType, HostColumnVector.StructData... lists )#

This method is evolving, unstable and currently test only.

Please use with caution and expect it to change in the future.

public static ColumnVector emptyStructs( HostColumnVector.DataType dataType, long numRows )#

This method is evolving, unstable and currently test only.

Please use with caution and expect it to change in the future.

public static ColumnVector fromBooleans(boolean... values)#: Create a new vector from the given values.

public static ColumnVector fromBytes(byte... values)#: Create a new vector from the given values.

public static ColumnVector fromUnsignedBytes(byte... values)#

Create a new vector from the given values.

Java does not have an unsigned byte type, so the values will be treated as if the bits represent an unsigned value.

public static ColumnVector fromShorts(short... values)#: Create a new vector from the given values.

public static ColumnVector fromUnsignedShorts(short... values)#

Create a new vector from the given values.

Java does not have an unsigned short type, so the values will be treated as if the bits represent an unsigned value.

public static ColumnVector fromInts(int... values)#: Create a new vector from the given values.

public static ColumnVector fromUnsignedInts(int... values)#

Create a new vector from the given values.

Java does not have an unsigned int type, so the values will be treated as if the bits represent an unsigned value.

public static ColumnVector fromLongs(long... values)#: Create a new vector from the given values.

public static ColumnVector fromUnsignedLongs(long... values)#

Create a new vector from the given values.

Java does not have an unsigned long type, so the values will be treated as if the bits represent an unsigned value.

public static ColumnVector fromFloats(float... values)#: Create a new vector from the given values.

public static ColumnVector fromDoubles(double... values)#: Create a new vector from the given values.

public static ColumnVector daysFromInts(int... values)#: Create a new vector from the given values.

public static ColumnVector durationSecondsFromLongs(long... values)#: Create a new vector from the given values.

public static ColumnVector timestampSecondsFromLongs(long... values)#: Create a new vector from the given values.

public static ColumnVector durationDaysFromInts(int... values)#: Create a new vector from the given values.

public static ColumnVector durationMilliSecondsFromLongs( long... values )#: Create a new vector from the given values.

public static ColumnVector timestampMilliSecondsFromLongs( long... values )#: Create a new vector from the given values.

public static ColumnVector durationMicroSecondsFromLongs( long... values )#: Create a new vector from the given values.

public static ColumnVector timestampMicroSecondsFromLongs( long... values )#: Create a new vector from the given values.

public static ColumnVector durationNanoSecondsFromLongs( long... values )#: Create a new vector from the given values.

public static ColumnVector timestampNanoSecondsFromLongs( long... values )#: Create a new vector from the given values.

public static ColumnVector decimalFromInts(int scale, int... values)#

Create a new decimal vector from unscaled values (int array) and scale.

The created vector is of type DType.DECIMAL32, whose max precision is 9. Compared with scale of java.math.BigDecimal, the scale here represents the opposite meaning.

public static ColumnVector decimalFromBoxedInts( int scale, Integer... values )#

Create a new decimal vector from boxed unscaled values (Integer array) and scale.

The created vector is of type DType.DECIMAL32, whose max precision is 9. Compared with scale of java.math.BigDecimal, the scale here represents the opposite meaning.

public static ColumnVector decimalFromLongs(int scale, long... values)#

Create a new decimal vector from unscaled values (long array) and scale.

The created vector is of type DType.DECIMAL64, whose max precision is 18. Compared with scale of java.math.BigDecimal, the scale here represents the opposite meaning.

public static ColumnVector decimalFromBoxedLongs( int scale, Long... values )#

Create a new decimal vector from boxed unscaled values (Long array) and scale.

The created vector is of type DType.DECIMAL64, whose max precision is 18. Compared with scale of java.math.BigDecimal, the scale here represents the opposite meaning.

public static ColumnVector decimalFromDoubles( DType type, RoundingMode mode, double... values )#

Create a new decimal vector from double floats with specific DecimalType and RoundingMode.

All doubles will be rescaled if necessary, according to scale of input DecimalType and RoundingMode. If any overflow occurs in extracting integral part, an IllegalArgumentException will be thrown. This API is inefficient because of slow double -> decimal conversion, so it is mainly for testing. Compared with scale of java.math.BigDecimal, the scale here represents the opposite meaning.

public static ColumnVector decimalFromBigInt( int scale, BigInteger... values )#: Create a new decimal vector from BigIntegers Compared with scale of java.math.BigDecimal, the scale here represents the opposite meaning.

public static ColumnVector fromStrings(String... values)#

Create a new string vector from the given values.

This API supports inline nulls. This is really intended to be used only for testing as it is slow and memory intensive to translate between java strings and UTF8 strings.

public static ColumnVector fromUTF8Strings(byte[]... values)#

Create a new string vector from the given values.

This API supports inline nulls.

public static ColumnVector fromDecimals(BigDecimal... values)#

Create a new vector from the given values.

This API supports inline nulls, but is much slower than building from primitive array of unscaledValues. Notice:

All input BigDecimals should share same scale.
The scale will be zero if all input values are null.

public static ColumnVector fromBoxedBooleans(Boolean... values)#

Create a new vector from the given values.

This API supports inline nulls, but is much slower than using a regular array and should really only be used for tests.

public static ColumnVector fromBoxedBytes(Byte... values)#

Create a new vector from the given values.

This API supports inline nulls, but is much slower than using a regular array and should really only be used for tests.

public static ColumnVector fromBoxedUnsignedBytes(Byte... values)#

Create a new vector from the given values.

This API supports inline nulls, but is much slower than using a regular array and should really only be used for tests.

Java does not have an unsigned byte type, so the values will be treated as if the bits represent an unsigned value.

public static ColumnVector fromBoxedShorts(Short... values)#

Create a new vector from the given values.

This API supports inline nulls, but is much slower than using a regular array and should really only be used for tests.

public static ColumnVector fromBoxedUnsignedShorts(Short... values)#

Create a new vector from the given values.

This API supports inline nulls, but is much slower than using a regular array and should really only be used for tests.

Java does not have an unsigned short type, so the values will be treated as if the bits represent an unsigned value.

public static ColumnVector fromBoxedInts(Integer... values)#

Create a new vector from the given values.

This API supports inline nulls, but is much slower than using a regular array and should really only be used for tests.

public static ColumnVector fromBoxedUnsignedInts(Integer... values)#

Create a new vector from the given values.

This API supports inline nulls, but is much slower than using a regular array and should really only be used for tests.

Java does not have an unsigned int type, so the values will be treated as if the bits represent an unsigned value.

public static ColumnVector fromBoxedLongs(Long... values)#

Create a new vector from the given values.

This API supports inline nulls, but is much slower than using a regular array and should really only be used for tests.

public static ColumnVector fromBoxedUnsignedLongs(Long... values)#

Create a new vector from the given values.

This API supports inline nulls, but is much slower than using a regular array and should really only be used for tests.

Java does not have an unsigned long type, so the values will be treated as if the bits represent an unsigned value.

public static ColumnVector fromBoxedFloats(Float... values)#

Create a new vector from the given values.

This API supports inline nulls, but is much slower than using a regular array and should really only be used for tests.

public static ColumnVector fromBoxedDoubles(Double... values)#

Create a new vector from the given values.

This API supports inline nulls, but is much slower than using a regular array and should really only be used for tests.

public static ColumnVector timestampDaysFromBoxedInts( Integer... values )#

Create a new vector from the given values.

This API supports inline nulls, but is much slower than using a regular array and should really only be used for tests.

public static ColumnVector durationDaysFromBoxedInts( Integer... values )#

Create a new vector from the given values.

This API supports inline nulls, but is much slower than using a regular array and should really only be used for tests.

public static ColumnVector durationSecondsFromBoxedLongs( Long... values )#

Create a new vector from the given values.

This API supports inline nulls, but is much slower than using a regular array and should really only be used for tests.

public static ColumnVector timestampSecondsFromBoxedLongs( Long... values )#

Create a new vector from the given values.

This API supports inline nulls, but is much slower than using a regular array and should really only be used for tests.

public static ColumnVector durationMilliSecondsFromBoxedLongs( Long... values )#

Create a new vector from the given values.

This API supports inline nulls, but is much slower than using a regular array and should really only be used for tests.

public static ColumnVector timestampMilliSecondsFromBoxedLongs( Long... values )#

Create a new vector from the given values.

This API supports inline nulls, but is much slower than using a regular array and should really only be used for tests.

public static ColumnVector durationMicroSecondsFromBoxedLongs( Long... values )#

Create a new vector from the given values.

This API supports inline nulls, but is much slower than using a regular array and should really only be used for tests.

public static ColumnVector timestampMicroSecondsFromBoxedLongs( Long... values )#

Create a new vector from the given values.

This API supports inline nulls, but is much slower than using a regular array and should really only be used for tests.

public static ColumnVector durationNanoSecondsFromBoxedLongs( Long... values )#

Create a new vector from the given values.

This API supports inline nulls, but is much slower than using a regular array and should really only be used for tests.

public static ColumnVector timestampNanoSecondsFromBoxedLongs( Long... values )#

Create a new vector from the given values.

This API supports inline nulls, but is much slower than using a regular array and should really only be used for tests.

public static ColumnVector empty(HostColumnVector.DataType colType)#

Creates an empty column according to the data type.

It will create all the nested columns by iterating all the children in the input type object ‘colType’.

The performance is not good, so use it carefully. We may want to move this implementation to the native once figuring out a way to pass the nested data type to the native.

Parameters:: colType – the data type of the empty column
Returns:: an empty ColumnVector with its children. Each children contains zero elements. Users should close the ColumnVector to avoid memory leak.

public interface EventHandler#

Interface to handle events for this ColumnVector.

Only invoked during close, hence onClosed is the only event.

Method Summary

onClosed(ColumnVector cv, int refCount)

onClosed is invoked with the updated refCount during close.

Methods

public void onClosed(ColumnVector cv, int refCount)#

onClosed is invoked with the updated refCount during close.

The last invocation of onClosed will be with refCount=0.

Note

the callback is invoked with this ColumnVector’s lock held.

Parameters:

cv – reference to the ColumnVector we are closing
refCount – the updated ref count for this ColumnVector at the time of invocation

protected static final class OffHeapState extends MemoryCleaner.Cleaner #

Holds the off heap state of the column vector so we can clean it up, even if it is leaked.

Field Summary

UNKNOWN_NULL_COUNT

Constructor Summary

`OffHeapState(DeviceMemoryBuffer data, DeviceMemoryBuffer valid, DeviceMemoryBuffer offsets, List<DeviceMemoryBuffer> buffers, long viewHandle)`	Create from existing cudf::column_view and buffers.
`OffHeapState(long columnHandle)`	Make a column form an existing cudf::column *.
`OffHeapState(long viewHandle, DeviceMemoryBuffer contiguousBuffer)`	Create from existing cudf::column_view and contiguous buffer.

Method Summary

`cleanImpl(boolean logErrorIfNotClean)`	Clean up any resources not previously released.
`getData()`
`getNativeNullCount()`
`getOffsets()`
`getValid()`
`getViewHandle()`
`isClean()`	Check if the underlying memory has been cleaned up or not.
`noWarnLeakExpected()`
`toString()`

Fields inherited from class ai.rapids.cudf.MemoryCleaner.Cleaner: id

Methods inherited from class ai.rapids.cudf.MemoryCleaner.Cleaner: addRef, clean, delRef, isLeakExpected, logRefCountDebug

Static Fields

public static final long UNKNOWN_NULL_COUNT = -1#

Constructors

public OffHeapState(long columnHandle)#: Make a column form an existing cudf::column *.

public OffHeapState( DeviceMemoryBuffer data, DeviceMemoryBuffer valid, DeviceMemoryBuffer offsets, List<DeviceMemoryBuffer> buffers, long viewHandle )#: Create from existing cudf::column_view and buffers.

public OffHeapState( long viewHandle, DeviceMemoryBuffer contiguousBuffer )#: Create from existing cudf::column_view and contiguous buffer.

Methods

public long getViewHandle()#

public long getNativeNullCount()#

public BaseDeviceMemoryBuffer getData()#

public BaseDeviceMemoryBuffer getValid()#

public BaseDeviceMemoryBuffer getOffsets()#

public void noWarnLeakExpected()#

public String toString()#

public boolean isClean()#

Check if the underlying memory has been cleaned up or not.

Returns:: true this is clean else false.

Protected Methods

protected boolean cleanImpl(boolean logErrorIfNotClean)#

Clean up any resources not previously released.

Parameters:: logErrorIfNotClean – if true and there are resources to clean up a leak has happened so log it.
Returns:: true if resources were cleaned up else false.