Table#

public final class Table implements AutoCloseable #

Class to represent a collection of ColumnVectors and operations that can be performed on them collectively.

The refcount on the columns will be increased once they are passed in

Constructor Summary

`Table(ColumnVector... columns)`	Table class makes a copy of the array of ColumnVectors passed to it.
`Table(long[] cudfColumns)`	Create a Table from an array of existing on device cudf::column pointers.

Method Summary

`close()`
`concatenate(Table... tables)`	Concatenate multiple tables together to form a single table.
`conditionalFullJoinGatherMaps(Table rightTable, CompiledExpression condition)`	Computes the gather maps that can be used to manifest the result of a full join between two tables when a conditional expression is true.
`conditionalInnerJoinGatherMaps(Table rightTable, CompiledExpression condition)`	Computes the gather maps that can be used to manifest the result of an inner join between two tables when a conditional expression is true.
`conditionalInnerJoinGatherMaps(Table rightTable, CompiledExpression condition, long outputRowCount)`	Computes the gather maps that can be used to manifest the result of an inner join between two tables when a conditional expression is true.
`conditionalInnerJoinRowCount(Table rightTable, CompiledExpression condition)`	Computes the number of rows from the result of an inner join between two tables when a conditional expression is true.
`conditionalLeftAntiJoinGatherMap(Table rightTable, CompiledExpression condition)`	Computes the gather map that can be used to manifest the result of a left anti join between two tables when a conditional expression is true.
`conditionalLeftAntiJoinGatherMap(Table rightTable, CompiledExpression condition, long outputRowCount)`	Computes the gather map that can be used to manifest the result of a left anti join between two tables when a conditional expression is true.
`conditionalLeftAntiJoinRowCount(Table rightTable, CompiledExpression condition)`	Computes the number of rows from the result of a left anti join between two tables when a conditional expression is true.
`conditionalLeftJoinGatherMaps(Table rightTable, CompiledExpression condition)`	Computes the gather maps that can be used to manifest the result of a left join between two tables when a conditional expression is true.
`conditionalLeftJoinGatherMaps(Table rightTable, CompiledExpression condition, long outputRowCount)`	Computes the gather maps that can be used to manifest the result of a left join between two tables when a conditional expression is true.
`conditionalLeftJoinRowCount(Table rightTable, CompiledExpression condition)`	Computes the number of rows from the result of a left join between two tables when a conditional expression is true.
`conditionalLeftSemiJoinGatherMap(Table rightTable, CompiledExpression condition)`	Computes the gather map that can be used to manifest the result of a left semi join between two tables when a conditional expression is true.
`conditionalLeftSemiJoinGatherMap(Table rightTable, CompiledExpression condition, long outputRowCount)`	Computes the gather map that can be used to manifest the result of a left semi join between two tables when a conditional expression is true.
`conditionalLeftSemiJoinRowCount(Table rightTable, CompiledExpression condition)`	Computes the number of rows from the result of a left semi join between two tables when a conditional expression is true.
`contiguousSplit(int... indices)`	Split a table at given boundaries, but the result of each split has memory that is laid out in a contiguous range of memory.
`crossJoin(Table right)`	Joins two tables all of the left against all of the right.
`distinctCount()`	Count how many rows in the table are distinct from one another.
`distinctCount(NullEquality nullsEqual)`	Count how many rows in the table are distinct from one another.
`dropDuplicates(int[] keyColumns, DuplicateKeepOption keep, boolean nullsEqual)`	Copy rows of the current table to an output table such that duplicate rows in the key columns are ignored (i.e., only one row from the duplicate ones will be copied).
`explode(int index)`	Explodes a list column’s elements.
`explodeOuter(int index)`	Explodes a list column’s elements.
`explodeOuterPosition(int index)`	Explodes a list column’s elements retaining any null entries or empty lists and includes a position column.
`explodePosition(int index)`	Explodes a list column’s elements and includes a position column.
`filter(ColumnView mask)`	Filters this table using a column of boolean values as a mask, returning a new one.
`filterJoinGatherMaps(GatherMap leftGatherMap, GatherMap rightGatherMap, Table leftTable, Table rightTable, CompiledExpression condition, JoinKind joinKind)`	Filters a pair of join gather maps by evaluating a conditional expression on the corresponding rows from the left and right tables.
`fromPackedTable(ByteBuffer metadata, DeviceMemoryBuffer data)`	Construct a table from a packed representation.
`fullJoinGatherMaps(HashJoin rightHash)`	Computes the gather maps that can be used to manifest the result of a full equi-join between two tables.
`fullJoinGatherMaps(HashJoin rightHash, long outputRowCount)`	Computes the gather maps that can be used to manifest the result of a full equi-join between two tables.
`fullJoinGatherMaps(Table rightKeys, boolean compareNullsEqual)`	Computes the gather maps that can be used to manifest the result of an full equi-join between two tables.
`fullJoinRowCount(HashJoin rightHash)`	Computes the number of rows resulting from a full equi-join between two tables.
`gather(ColumnView gatherMap)`	Gathers the rows of this table according to gatherMap such that row “i” in the resulting table’s columns will contain row “gatherMap[i]” from this table.
`gather(ColumnView gatherMap, OutOfBoundsPolicy outOfBoundsPolicy)`	Gathers the rows of this table according to gatherMap such that row “i” in the resulting table’s columns will contain row “gatherMap[i]” from this table.
`getColumn(int index)`	Return the ColumnVector at the specified index.
`getCSVBufferWriter(CSVWriterOptions options, HostBufferConsumer bufferConsumer)`
`getCSVBufferWriter(CSVWriterOptions options, HostBufferConsumer bufferConsumer, HostMemoryAllocator hostMemoryAllocator)`
`getDeviceMemorySize()`	Returns the Device memory buffer size.
`getNativeView()`	Return the native table view handle for this table.
`getNumberOfColumns()`
`getRowCount()`
`groupBy(GroupByOptions groupByOptions, int... indices)`	Returns aggregate operations grouped by columns provided in indices.
`groupBy(int... indices)`	Returns aggregate operations grouped by columns provided in indices with default options as below:
`innerDistinctJoinGatherMaps(DistinctHashJoin rightHash)`	Computes the gather maps that can be used to manifest the result of an inner equi-join between two tables where the right table is guaranteed not to contain any duplicated join keys.
`innerDistinctJoinGatherMaps(Table rightKeys, boolean compareNullsEqual)`	Computes the gather maps that can be used to manifest the result of an inner equi-join between two tables where the right table is guaranteed to not contain any duplicated join keys.
`innerJoinGatherMaps(HashJoin rightHash)`	Computes the gather maps that can be used to manifest the result of an inner equi-join between two tables.
`innerJoinGatherMaps(HashJoin rightHash, long outputRowCount)`	Computes the gather maps that can be used to manifest the result of an inner equi-join between two tables.
`innerJoinGatherMaps(Table rightKeys, boolean compareNullsEqual)`	Computes the gather maps that can be used to manifest the result of an inner equi-join between two tables.
`innerJoinRowCount(HashJoin otherHash)`	Computes the number of rows resulting from an inner equi-join between two tables.
`interleaveColumns()`	Interleave all columns into a single column.
`leftAntiJoinGatherMap(FilteredJoin rightFilter)`	Computes the gather map that can be used to manifest the result of a left anti-join between two tables.
`leftAntiJoinGatherMap(Table rightKeys, boolean compareNullsEqual)`	Computes the gather map that can be used to manifest the result of a left anti-join between two tables.
`leftDistinctJoinGatherMap(DistinctHashJoin rightHash)`	Computes a gather map that can be used to manifest the result of a left equi-join between two tables where the right table is guaranteed not to contain any duplicated join keys.
`leftDistinctJoinGatherMap(Table rightKeys, boolean compareNullsEqual)`	Computes a gather map that can be used to manifest the result of a left equi-join between two tables where the right table is guaranteed to not contain any duplicated join keys.
`leftJoinGatherMaps(HashJoin rightHash)`	Computes the gather maps that can be used to manifest the result of a left equi-join between two tables.
`leftJoinGatherMaps(HashJoin rightHash, long outputRowCount)`	Computes the gather maps that can be used to manifest the result of a left equi-join between two tables.
`leftJoinGatherMaps(Table rightKeys, boolean compareNullsEqual)`	Computes the gather maps that can be used to manifest the result of a left equi-join between two tables.
`leftJoinRowCount(HashJoin rightHash)`	Computes the number of rows resulting from a left equi-join between two tables.
`leftSemiJoinGatherMap(FilteredJoin rightFilter)`	Computes the gather map that can be used to manifest the result of a left semi-join between two tables.
`leftSemiJoinGatherMap(Table rightKeys, boolean compareNullsEqual)`	Computes the gather map that can be used to manifest the result of a left semi-join between two tables.
`lowerBound(Table valueTable, OrderByArg... args)`	Find smallest indices in a sorted table where values should be inserted to maintain order.
`lowerBound(boolean[] areNullsSmallest, Table valueTable, boolean[] descFlags)`	Find smallest indices in a sorted table where values should be inserted to maintain order.
`makeChunkedPack(long bounceBufferSize)`	Create an instance of ChunkedPack which can be used to pack this table contiguously in memory utilizing a bounce buffer of size bounceBufferSize.
`makeChunkedPack(long bounceBufferSize, RmmDeviceMemoryResource tempMemoryResource)`	Create an instance of ChunkedPack which can be used to pack this table contiguously in memory utilizing a bounce buffer of size bounceBufferSize.
`merge(List<Table> tables, OrderByArg... args)`	Merge multiple already sorted tables keeping the sort order the same.
`merge(Table[] tables, OrderByArg... args)`	Merge multiple already sorted tables keeping the sort order the same.
`mixedFullJoinGatherMaps(Table leftKeys, Table rightKeys, Table leftConditional, Table rightConditional, CompiledExpression condition, NullEquality nullEquality)`	Computes the gather maps that can be used to manifest the result of a full join between two tables using a mix of equality and inequality conditions.
`mixedInnerJoinGatherMaps(Table leftKeys, Table rightKeys, Table leftConditional, Table rightConditional, CompiledExpression condition, NullEquality nullEquality)`	Computes the gather maps that can be used to manifest the result of an inner join between two tables using a mix of equality and inequality conditions.
`mixedInnerJoinGatherMaps(Table leftKeys, Table rightKeys, Table leftConditional, Table rightConditional, CompiledExpression condition, NullEquality nullEquality, MixedJoinSize joinSize)`	Computes the gather maps that can be used to manifest the result of an inner join between two tables using a mix of equality and inequality conditions.
`mixedInnerJoinSize(Table leftKeys, Table rightKeys, Table leftConditional, Table rightConditional, CompiledExpression condition, NullEquality nullEquality)`	Computes output size information for an inner join between two tables using a mix of equality and inequality conditions.
`mixedLeftAntiJoinGatherMap(Table leftKeys, Table rightKeys, Table leftConditional, Table rightConditional, CompiledExpression condition, NullEquality nullEquality)`	Computes the gather map that can be used to manifest the result of a left anti join between two tables using a mix of equality and inequality conditions.
`mixedLeftJoinGatherMaps(Table leftKeys, Table rightKeys, Table leftConditional, Table rightConditional, CompiledExpression condition, NullEquality nullEquality)`	Computes the gather maps that can be used to manifest the result of a left join between two tables using a mix of equality and inequality conditions.
`mixedLeftJoinGatherMaps(Table leftKeys, Table rightKeys, Table leftConditional, Table rightConditional, CompiledExpression condition, NullEquality nullEquality, MixedJoinSize joinSize)`	Computes the gather maps that can be used to manifest the result of a left join between two tables using a mix of equality and inequality conditions.
`mixedLeftJoinSize(Table leftKeys, Table rightKeys, Table leftConditional, Table rightConditional, CompiledExpression condition, NullEquality nullEquality)`	Computes output size information for a left join between two tables using a mix of equality and inequality conditions.
`mixedLeftSemiJoinGatherMap(Table leftKeys, Table rightKeys, Table leftConditional, Table rightConditional, CompiledExpression condition, NullEquality nullEquality)`	Computes the gather map that can be used to manifest the result of a left semi join between two tables using a mix of equality and inequality conditions.
`onColumns(int... indices)`
`orderBy(OrderByArg... args)`	Orders the table using the sortkeys returning a new allocated table.
`partition(ColumnView partitionMap, int numberOfPartitions)`	Partition this table using the mapping in partitionMap.
`readAndInferJSON(JSONOptions opts, DataSource ds)`	Read JSON formatted data and infer the column names and schema.
`readArrowIPCChunked(ArrowIPCOptions options, File inputFile)`	Get a reader that will return tables.
`readArrowIPCChunked(ArrowIPCOptions options, HostBufferProvider provider)`
`readArrowIPCChunked(ArrowIPCOptions options, HostBufferProvider provider, HostMemoryAllocator hostMemoryAllocator)`	Get a reader that will return tables.
`readArrowIPCChunked(File inputFile)`	Get a reader that will return tables.
`readArrowIPCChunked(HostBufferProvider provider)`	Get a reader that will return tables.
`readAvro(AvroOptions opts, DataSource ds)`
`readAvro(AvroOptions opts, File path)`	Read an Avro file.
`readAvro(AvroOptions opts, HostMemoryBuffer buffer, long offset, long len)`	Read Avro formatted data.
`readAvro(AvroOptions opts, byte[] buffer)`	Read Avro formatted data.
`readAvro(AvroOptions opts, byte[] buffer, long offset, long len)`
`readAvro(AvroOptions opts, byte[] buffer, long offset, long len, HostMemoryAllocator hostMemoryAllocator)`	Read Avro formatted data.
`readAvro(File path)`	Read an Avro file using the default AvroOptions.
`readAvro(byte[] buffer)`	Read Avro formatted data.
`readCSV(Schema schema, CSVOptions opts, DataSource ds)`
`readCSV(Schema schema, CSVOptions opts, File path)`	Read a CSV file.
`readCSV(Schema schema, CSVOptions opts, HostMemoryBuffer buffer, long offset, long len)`	Read CSV formatted data.
`readCSV(Schema schema, CSVOptions opts, byte[] buffer)`	Read CSV formatted data.
`readCSV(Schema schema, CSVOptions opts, byte[] buffer, long offset, long len)`
`readCSV(Schema schema, CSVOptions opts, byte[] buffer, long offset, long len, HostMemoryAllocator hostMemoryAllocator)`	Read CSV formatted data.
`readCSV(Schema schema, File path)`	Read a CSV file using the default CSVOptions.
`readCSV(Schema schema, byte[] buffer)`	Read CSV formatted data using the default CSVOptions.
`readJSON(JSONOptions opts, HostMemoryBuffer buffer, long offset, long len)`	Read JSON formatted data and infer the column names and schema.
`readJSON(Schema schema, File path)`	Read a JSON file using the default JSONOptions.
`readJSON(Schema schema, JSONOptions opts, DataSource ds)`	Read JSON formatted data.
`readJSON(Schema schema, JSONOptions opts, DataSource ds, int emptyRowCount)`	Deprecated. Read JSON formatted data.
`readJSON(Schema schema, JSONOptions opts, File path)`	Read a JSON file.
`readJSON(Schema schema, JSONOptions opts, HostMemoryBuffer buffer, long offset, long len)`	Read JSON formatted data.
`readJSON(Schema schema, JSONOptions opts, HostMemoryBuffer buffer, long offset, long len, int emptyRowCount)`	Deprecated. Read JSON formatted data.
`readJSON(Schema schema, JSONOptions opts, byte[] buffer)`	Read JSON formatted data.
`readJSON(Schema schema, JSONOptions opts, byte[] buffer, long offset, long len)`
`readJSON(Schema schema, JSONOptions opts, byte[] buffer, long offset, long len, HostMemoryAllocator hostMemoryAllocator)`	Read JSON formatted data.
`readJSON(Schema schema, JSONOptions opts, byte[] buffer, long offset, long len, HostMemoryAllocator hostMemoryAllocator, int emptyRowCount)`	Deprecated. Read JSON formatted data.
`readJSON(Schema schema, JSONOptions opts, byte[] buffer, long offset, long len, int emptyRowCount)`
`readJSON(Schema schema, byte[] buffer)`	Read JSON formatted data using the default JSONOptions.
`readORC(File path)`	Read a ORC file using the default ORCOptions.
`readORC(ORCOptions opts, DataSource ds)`
`readORC(ORCOptions opts, File path)`	Read a ORC file.
`readORC(ORCOptions opts, HostMemoryBuffer buffer, long offset, long len)`	Read ORC formatted data.
`readORC(ORCOptions opts, byte[] buffer)`	Read ORC formatted data.
`readORC(ORCOptions opts, byte[] buffer, long offset, long len)`
`readORC(ORCOptions opts, byte[] buffer, long offset, long len, HostMemoryAllocator hostMemoryAllocator)`	Read ORC formatted data.
`readORC(byte[] buffer)`	Read ORC formatted data.
`readParquet(File path)`	Read a Parquet file using the default ParquetOptions.
`readParquet(ParquetOptions opts, DataSource ds)`	Read parquet formatted data.
`readParquet(ParquetOptions opts, File path)`	Read a Parquet file.
`readParquet(ParquetOptions opts, HostMemoryBuffer buffer, long offset, long len)`	Read parquet formatted data.
`readParquet(ParquetOptions opts, HostMemoryBuffer... buffers)`	Read parquet formatted data.
`readParquet(ParquetOptions opts, byte[] buffer)`	Read parquet formatted data.
`readParquet(ParquetOptions opts, byte[] buffer, long offset, long len)`	Read parquet formatted data.
`readParquet(ParquetOptions opts, byte[] buffer, long offset, long len, HostMemoryAllocator hostMemoryAllocator)`	Read parquet formatted data.
`readParquet(byte[] buffer)`	Read parquet formatted data.
`repeat(ColumnView counts)`	Create a new table by repeating each row of this table.
`repeat(int count)`	Repeat each row of this table count times.
`roundRobinPartition(int numberOfPartitions, int startPartition)`	Round-robin partition a table into the specified number of partitions.
`rowBitCount()`	Returns an approximate cumulative size in bits of all columns in the table_view for each row.
`sample(long n, boolean replacement, long seed)`	Gather n samples from table randomly Note: does not preserve the ordering Example: input: {col1: {1, 2, 3, 4, 5}, col2: {6, 7, 8, 9, 10}} n: 3 replacement: false.
`scatter(ColumnView scatterMap, Table target)`	Scatters values from the source table into the target table out-of-place, returning a new result table.
`scatter(Scalar[] source, ColumnView scatterMap, Table target)`	Scatters values from the source rows into the target table out-of-place, returning a new result table.
`sortOrder(OrderByArg... args)`	Get back a gather map that can be used to sort the data.
`toString()`
`upperBound(Table valueTable, OrderByArg... args)`	Find largest indices in a sorted table where values should be inserted to maintain order.
`upperBound(boolean[] areNullsSmallest, Table valueTable, boolean[] descFlags)`	Find largest indices in a sorted table where values should be inserted to maintain order.
`writeArrowIPCChunked(ArrowIPCWriterOptions options, File outputFile)`	Get a table writer to write arrow IPC data to a file.
`writeArrowIPCChunked(ArrowIPCWriterOptions options, HostBufferConsumer consumer)`
`writeArrowIPCChunked(ArrowIPCWriterOptions options, HostBufferConsumer consumer, HostMemoryAllocator hostMemoryAllocator)`	Get a table writer to write arrow IPC data and handle each chunk with a callback.
`writeColumnViewsToParquet(ParquetWriterOptions options, HostBufferConsumer consumer, ColumnView... columnViews)`
`writeColumnViewsToParquet(ParquetWriterOptions options, HostBufferConsumer consumer, HostMemoryAllocator hostMemoryAllocator, ColumnView... columnViews)`	This is an evolving API and most likely be removed in future releases.
`writeCSVToFile(CSVWriterOptions options, String outputPath)`
`writeORCChunked(ORCWriterOptions options, File outputFile)`	Get a table writer to write ORC data to a file.
`writeORCChunked(ORCWriterOptions options, HostBufferConsumer consumer)`
`writeORCChunked(ORCWriterOptions options, HostBufferConsumer consumer, HostMemoryAllocator hostMemoryAllocator)`	Get a table writer to write ORC data and handle each chunk with a callback.
`writeParquetChunked(ParquetWriterOptions options, File outputFile)`	Get a table writer to write parquet data to a file.
`writeParquetChunked(ParquetWriterOptions options, HostBufferConsumer consumer)`
`writeParquetChunked(ParquetWriterOptions options, HostBufferConsumer consumer, HostMemoryAllocator hostMemoryAllocator)`	Get a table writer to write parquet data and handle each chunk with a callback.

Constructors

public Table(ColumnVector... columns)#

Table class makes a copy of the array of ColumnVectors passed to it.

The class will decrease the refcount on itself and all its contents when closed and free resources if refcount is zero

Parameters:: columns – - Array of ColumnVectors

public Table(long[] cudfColumns)#

Create a Table from an array of existing on device cudf::column pointers.

Ownership of the columns is transferred to the ColumnVectors held by the new Table. In the case of an exception the columns will be deleted.

Parameters:: cudfColumns – - Array of nativeHandles

Methods

public long getNativeView()#: Return the native table view handle for this table.

public ColumnVector getColumn(int index)#

Return the ColumnVector at the specified index.

If you want to keep a reference to the column around past the life time of the table, you will need to increment the reference count on the column yourself.

public final long getRowCount()#

public final int getNumberOfColumns()#

public void close()#

public String toString()#

public long getDeviceMemorySize()#: Returns the Device memory buffer size.

public void writeCSVToFile( CSVWriterOptions options, String outputPath )#

public ColumnVector interleaveColumns()#

Interleave all columns into a single column.

Columns must all have the same data type and length.

Example:

input  = [[A1, A2, A3], [B1, B2, B3]]
return = [A1, B1, A2, B2, A3, B3]

Returns:: The interleaved columns as a single column

public Table repeat(int count)#

Repeat each row of this table count times.

Parameters:: count – the number of times to repeat each row.
Returns:: the new Table.

public Table repeat(ColumnView counts)#

Create a new table by repeating each row of this table.

The number of repetitions of each row is defined by the corresponding value in counts.

Parameters:: counts – the number of times to repeat each row. Cannot have nulls, must be an Integer type, and must have one entry for each row in the table.
Throws:: CudfException – on any error.
Returns:: the new Table.

public PartitionedTable partition( ColumnView partitionMap, int numberOfPartitions )#

Partition this table using the mapping in partitionMap.

partitionMap must be an integer column. The number of rows in partitionMap must be the same as this table. Each row in the map will indicate which partition the rows in the table belong to.

Parameters:

partitionMap – the partitions for each row.
numberOfPartitions – number of partitions

Returns:

PartitionedTable Table that exposes a limited functionality of the Table class

public ColumnVector lowerBound( boolean[] areNullsSmallest, Table valueTable, boolean[] descFlags )#

Find smallest indices in a sorted table where values should be inserted to maintain order.

Example:

 Single column:
     idx            0   1   2   3   4
  inputTable  =   { 10, 20, 20, 30, 50 }
  valuesTable =   { 20 }
  result      =   { 1 }

 Multi Column:
     idx                0    1    2    3    4
  inputTable      = {{  10,  20,  20,  20,  20 },
                     { 5.0,  .5,  .5,  .7,  .7 },
                     {  90,  77,  78,  61,  61 }}
  valuesTable     = {{ 20 },
                     { .7 },
                     { 61 }}
  result          = {  3 }

The input table and the values table need to be non-empty (row count > 0)

Parameters:

areNullsSmallest – per column, true if nulls are assumed smallest
valueTable – the table of values to find insertion locations for
descFlags – per column indicates the ordering, true if descending.

Returns:

ColumnVector with lower bound indices for all rows in valueTable

public ColumnVector lowerBound(Table valueTable, OrderByArg... args)#

Find smallest indices in a sorted table where values should be inserted to maintain order.

This is a convenience method. It pulls out the columns indicated by the args and sets up the ordering properly to call lowerBound.

Parameters:

valueTable – the table of values to find insertion locations for
args – the sort order used to sort this table.

Returns:

ColumnVector with lower bound indices for all rows in valueTable

public ColumnVector upperBound( boolean[] areNullsSmallest, Table valueTable, boolean[] descFlags )#

Find largest indices in a sorted table where values should be inserted to maintain order.

Given a sorted table return the upper bound.

Example:

 Single column:
     idx            0   1   2   3   4
  inputTable  =   { 10, 20, 20, 30, 50 }
  valuesTable =   { 20 }
  result      =   { 3 }

 Multi Column:
     idx                0    1    2    3    4
  inputTable      = {{  10,  20,  20,  20,  20 },
                     { 5.0,  .5,  .5,  .7,  .7 },
                     {  90,  77,  78,  61,  61 }}
  valuesTable     = {{ 20 },
                     { .7 },
                     { 61 }}
  result          = {  5 }

The input table and the values table need to be non-empty (row count > 0)

Parameters:

areNullsSmallest – per column, true if nulls are assumed smallest
valueTable – the table of values to find insertion locations for
descFlags – per column indicates the ordering, true if descending.

Returns:

ColumnVector with upper bound indices for all rows in valueTable

public ColumnVector upperBound(Table valueTable, OrderByArg... args)#

Find largest indices in a sorted table where values should be inserted to maintain order.

This is a convenience method. It pulls out the columns indicated by the args and sets up the ordering properly to call upperBound.

Parameters:

valueTable – the table of values to find insertion locations for
args – the sort order used to sort this table.

Returns:

ColumnVector with upper bound indices for all rows in valueTable

public Table crossJoin(Table right)#

Joins two tables all of the left against all of the right.

Be careful as this gets very big and you can easily use up all of the GPUs memory.

Parameters:: right – the right table
Returns:: the joined table. The order of the columns returned will be left columns, right columns.

public ColumnVector sortOrder(OrderByArg... args)#

Get back a gather map that can be used to sort the data.

This allows you to sort by data that does not appear in the final result and not pay the cost of gathering the data that is only needed for sorting.

Parameters:: args – what order to sort the data by
Returns:: a gather map

public Table orderBy(OrderByArg... args)#

Orders the table using the sortkeys returning a new allocated table.

The caller is responsible for cleaning up the ColumnVector returned as part of the output Table

Example usage: orderBy(true, OrderByArg.asc(0), OrderByArg.desc(3)…);

Parameters:: args – Suppliers to initialize sortKeys.
Returns:: Sorted Table

public GroupByOperation groupBy( GroupByOptions groupByOptions, int... indices )#

Returns aggregate operations grouped by columns provided in indices.

Parameters:

groupByOptions – Options provided in the builder
indices – columns to be considered for groupBy

public GroupByOperation groupBy(int... indices)#

Returns aggregate operations grouped by columns provided in indices with default options as below:

null is considered as key while grouping.
keys are not presorted.
empty key order array.
empty null order array.

Parameters:: indices – columns to be considered for groupBy

public PartitionedTable roundRobinPartition( int numberOfPartitions, int startPartition )#

Round-robin partition a table into the specified number of partitions.

The first row is placed in the specified starting partition, the next row is placed in the next partition, and so on. When the last partition is reached then next partition is partition 0 and the algorithm continues until all rows have been placed in partitions, evenly distributing the rows among the partitions.

Parameters:

numberOfPartitions – - number of partitions to use
startPartition – - starting partition index (i.e.: where first row is placed).

Returns:

- PartitionedTable - Table that exposes a limited functionality of the Table class

public TableOperation onColumns(int... indices)#

public Table filter(ColumnView mask)#

Filters this table using a column of boolean values as a mask, returning a new one.

Given a mask column, each element i from the input columns is copied to the output columns if the corresponding element i in the mask is non-null and true. This operation is stable: the input order is preserved.

This table and mask columns must have the same number of rows.

The output table has size equal to the number of elements in boolean_mask that are both non-null and true.

If the original table row count is zero, there is no error, and an empty table is returned.

Parameters:: mask – column of type DType.BOOL8 used as a mask to filter the input column
Returns:: table containing copy of all elements of this table passing the filter defined by the boolean mask

public Table dropDuplicates( int[] keyColumns, DuplicateKeepOption keep, boolean nullsEqual )#

Copy rows of the current table to an output table such that duplicate rows in the key columns are ignored (i.e., only one row from the duplicate ones will be copied).

These keys columns are a subset of the current table columns and their indices are specified by an input array.

The order of rows in the output table is not specified.

Parameters:

keyColumns – Array of indices representing key columns from the current table.
keep – Option specifying to keep any, first, last, or none of the found duplicates.
nullsEqual – Flag to denote whether nulls are treated as equal when comparing rows of the key columns to check for uniqueness.

Returns:

Table with unique keys.

public int distinctCount(NullEquality nullsEqual)#

Count how many rows in the table are distinct from one another.

Parameters:: nullsEqual – if nulls should be considered equal to each other or not.

public int distinctCount()#

Count how many rows in the table are distinct from one another.

Nulls are considered to be equal to one another.

public ContiguousTable[] contiguousSplit(int... indices)#

Split a table at given boundaries, but the result of each split has memory that is laid out in a contiguous range of memory.

This allows for us to optimize copying the data in a single operation.

Example: input: [{10, 12, 14, 16, 18, 20, 22, 24, 26, 28}, {50, 52, 54, 56, 58, 60, 62, 64, 66, 68}] splits: {2, 5, 9} output: [{{10, 12}, {14, 16, 18}, {20, 22, 24, 26}, {28}}, {{50, 52}, {54, 56, 58}, {60, 62, 64, 66}, {68}}]

Parameters:: indices – A vector of indices where to make the split
Returns:: The tables split at those points. NOTE: It is the responsibility of the caller to close the result. Each table and column holds a reference to the original buffer. But both the buffer and the table must be closed for the memory to be released.

public ChunkedPack makeChunkedPack( long bounceBufferSize, RmmDeviceMemoryResource tempMemoryResource )#

Create an instance of ChunkedPack which can be used to pack this table contiguously in memory utilizing a bounce buffer of size bounceBufferSize.

This version of makeChunkedPack takes a RmmDviceMemoryResource, which can be used to pre-allocate all scratch and temporary space required for the state of cudf::chunked_pack.

The caller is responsible for calling close on the returned ChunkedPack object.

Parameters:

bounceBufferSize – The size of bounce buffer that will be utilized to pack into
tempMemoryResource – A memory resource that is used to satisfy allocations for temporary and thrust scratch space.

Returns:

An instance of ChunkedPack that the caller must use to finish the operation.

public ChunkedPack makeChunkedPack(long bounceBufferSize)#

Create an instance of ChunkedPack which can be used to pack this table contiguously in memory utilizing a bounce buffer of size bounceBufferSize.

This version of makeChunkedPack makes use of the default per-device memory resource, for scratch and temporary space required for the state of cudf::chunked_pack.

The caller is responsible for calling close on the returned ChunkedPack object.

Parameters:: bounceBufferSize – The size of bounce buffer that will be utilized to pack into
Returns:: An instance of ChunkedPack that the caller must use to finish the operation.

public Table explode(int index)#

Explodes a list column’s elements.

Any list is exploded, which means the elements of the list in each row are expanded into new rows in the output. The corresponding rows for other columns in the input are duplicated.

Example: input: [[5,10,15], 100], [[20,25], 200], [[30], 300] index: 0 output: [5, 100], [10, 100], [15, 100], [20, 200], [25, 200], [30, 300]

Nulls propagate in different ways depending on what is null. input: [[5,null,15], 100], [null, 200] index: 0 output: [5, 100], [null, 100], [15, 100] Note that null lists are completely removed from the output and nulls inside lists are pulled out and remain.

Parameters:: index – Column index to explode inside the table.
Returns:: A new table with explode_col exploded.

public Table explodePosition(int index)#

Explodes a list column’s elements and includes a position column.

Any list is exploded, which means the elements of the list in each row are expanded into new rows in the output. The corresponding rows for other columns in the input are duplicated. A position column is added that has the index inside the original list for each row. Example: input: [[5,10,15], 100], [[20,25], 200], [[30], 300] index: 0 output: [0, 5, 100], [1, 10, 100], [2, 15, 100], [0, 20, 200], [1, 25, 200], [0, 30, 300]

Nulls and empty lists propagate in different ways depending on what is null or empty. input: [[5,null,15], 100], [null, 200] index: 0 output: [5, 100], [null, 100], [15, 100]

Note that null lists are not included in the resulting table, but nulls inside lists and empty lists will be represented with a null entry for that column in that row.

Parameters:: index – Column index to explode inside the table.
Returns:: A new table with exploded value and position. The column order of return table is [cols before explode_input, explode_position, explode_value, cols after explode_input].

public Table explodeOuter(int index)#

Explodes a list column’s elements.

Any list is exploded, which means the elements of the list in each row are expanded into new rows in the output. The corresponding rows for other columns in the input are duplicated.

Example: input: [[5,10,15], 100], [[20,25], 200], [[30], 300], index: 0 output: [5, 100], [10, 100], [15, 100], [20, 200], [25, 200], [30, 300]

Nulls propagate in different ways depending on what is null. input: [[5,null,15], 100], [null, 200] index: 0 output: [5, 100], [null, 100], [15, 100], [null, 200] Note that null lists are completely removed from the output and nulls inside lists are pulled out and remain.

Parameters:: index – Column index to explode inside the table.
Returns:: A new table with explode_col exploded.

public Table explodeOuterPosition(int index)#

Explodes a list column’s elements retaining any null entries or empty lists and includes a position column.

Example: input: [[5,10,15], 100], [[20,25], 200], [[30], 300], index: 0 output: [0, 5, 100], [1, 10, 100], [2, 15, 100], [0, 20, 200], [1, 25, 200], [0, 30, 300]

Nulls and empty lists propagate as null entries in the result. input: [[5,null,15], 100], [null, 200], [[], 300] index: 0 output: [0, 5, 100], [1, null, 100], [2, 15, 100], [0, null, 200], [0, null, 300]

returns

Parameters:: index – Column index to explode inside the table.
Returns:: A new table with exploded value and position. The column order of return table is [cols before explode_input, explode_position, explode_value, cols after explode_input].

public ColumnVector rowBitCount()#

Returns an approximate cumulative size in bits of all columns in the table_view for each row.

This function counts bits instead of bytes to account for the null mask which only has one bit per row. Each row in the returned column is the sum of the per-row bit size for each column in the table.

In some cases, this is an inexact approximation. Specifically, columns of lists and strings require N+1 offsets to represent N rows. It is up to the caller to calculate the small additional overhead of the terminating offset for any group of rows being considered.

This function returns the per-row bit sizes as the columns are currently formed. This can end up being larger than the number you would get by gathering the rows. Specifically, the push-down of struct column validity masks can nullify rows that contain data for string or list columns. In these cases, the size returned is conservative such that: row_bit_count(column(x)) >= row_bit_count(gather(column(x)))

Returns:: INT32 column of bit size per row of the table

public Table gather(ColumnView gatherMap)#

Gathers the rows of this table according to gatherMap such that row “i” in the resulting table’s columns will contain row “gatherMap[i]” from this table.

The number of rows in the result table will be equal to the number of elements in gatherMap.

A negative value i in the gatherMap is interpreted as i+n, where n is the number of rows in this table.

Parameters:: gatherMap – the map of indexes. Must be non-nullable and integral type.
Returns:: the resulting Table.

public Table gather( ColumnView gatherMap, OutOfBoundsPolicy outOfBoundsPolicy )#

Gathers the rows of this table according to gatherMap such that row “i” in the resulting table’s columns will contain row “gatherMap[i]” from this table.

The number of rows in the result table will be equal to the number of elements in gatherMap.

A negative value i in the gatherMap is interpreted as i+n, where n is the number of rows in this table.

Parameters:

gatherMap – the map of indexes. Must be non-nullable and integral type.
outOfBoundsPolicy – policy to use when an out-of-range value is in gatherMap.

Returns:

the resulting Table.

public Table scatter(ColumnView scatterMap, Table target)#

Scatters values from the source table into the target table out-of-place, returning a new result table.

The scatter is performed according to a scatter map such that row scatterMap[i] of the destination table gets row i of the source table. All other rows of the destination table equal corresponding rows of the target table.

The number of columns in source must match the number of columns in target and their corresponding data types must be the same.

If the same index appears more than once in the scatter map, the result is undefined.

A negative value i in the scatterMap is interpreted as i + n, where n is the number of rows in the target table.

Parameters:

scatterMap – The map of indexes. Must be non-nullable and integral type.
target – The table into which rows from the current table are to be scattered out-of-place.

Returns:

A new table which is the result of out-of-place scattering the source table into the target table.

public GatherMap[] leftJoinGatherMaps( Table rightKeys, boolean compareNullsEqual )#

Computes the gather maps that can be used to manifest the result of a left equi-join between two tables.

It is assumed this table instance holds the key columns from the left table, and the table argument represents the key columns from the right table. Two GatherMap instances will be returned that can be used to gather the left and right tables, respectively, to produce the result of the left join.

It is the responsibility of the caller to close the resulting gather map instances.

Parameters:

rightKeys – join key columns from the right table
compareNullsEqual – true if null key values should match otherwise false

Returns:

left and right table gather maps

public GatherMap leftDistinctJoinGatherMap( Table rightKeys, boolean compareNullsEqual )#

Computes a gather map that can be used to manifest the result of a left equi-join between two tables where the right table is guaranteed to not contain any duplicated join keys.

The left table can be used as-is to produce the left table columns resulting from the join, i.e.: left table ordering is preserved in the join result, so no gather map is required for the left table. The resulting gather map can be applied to the right table to produce the right table columns resulting from the join. It is assumed this table instance holds the key columns from the left table, and the table argument represents the key columns from the right table. A GatherMap instance will be returned that can be used to gather the right table and that result combined with the left table to produce a left outer join result.

It is the responsibility of the caller to close the resulting gather map instance.

Parameters:

rightKeys – join key columns from the right table
compareNullsEqual – true if null key values should match otherwise false

Returns:

right table gather map

public GatherMap leftDistinctJoinGatherMap(DistinctHashJoin rightHash)#

Computes a gather map that can be used to manifest the result of a left equi-join between two tables where the right table is guaranteed not to contain any duplicated join keys.

The left table can be used as-is to produce the left table columns resulting from the join, i.e.: left table ordering is preserved in the join result, so no gather map is required for the left table. The resulting gather map can be applied to the right table to produce the right table columns resulting from the join. It is assumed this table instance holds the key columns from the left table, and the DistinctHashJoin argument has been constructed from the key columns from the right table. A GatherMap instance will be returned that can be used to gather the right table and that result combined with the left table to produce a left outer join result.

It is the responsibility of the caller to close the resulting gather map instance.

Parameters:: rightHash – hash table built from distinct join key columns from the right table
Returns:: right table gather map

public long leftJoinRowCount(HashJoin rightHash)#

Computes the number of rows resulting from a left equi-join between two tables.

It is assumed this table instance holds the key columns from the left table, and the HashJoin argument has been constructed from the key columns from the right table.

Parameters:: rightHash – hash table built from join key columns from the right table
Returns:: row count of the join result

public GatherMap[] leftJoinGatherMaps(HashJoin rightHash)#

Computes the gather maps that can be used to manifest the result of a left equi-join between two tables.

It is assumed this table instance holds the key columns from the left table, and the HashJoin argument has been constructed from the key columns from the right table. Two GatherMap instances will be returned that can be used to gather the left and right tables, respectively, to produce the result of the left join.

It is the responsibility of the caller to close the resulting gather map instances.

Parameters:: rightHash – hash table built from join key columns from the right table
Returns:: left and right table gather maps

public GatherMap[] leftJoinGatherMaps( HashJoin rightHash, long outputRowCount )#

Computes the gather maps that can be used to manifest the result of a left equi-join between two tables.

It is the responsibility of the caller to close the resulting gather map instances.

This interface allows passing an output row count that was previously computed from leftJoinRowCount(HashJoin). Doing so no longer avoids any work: the output size is always computed internally and the supplied count is only validated against it. Prefer leftJoinGatherMaps(HashJoin); this overload will be deprecated in a future release.

Throws:

CudfException – if outputRowCount does not equal the actual output row count

Parameters:

rightHash – hash table built from join key columns from the right table
outputRowCount – number of output rows in the join result

Returns:

left and right table gather maps

public long conditionalLeftJoinRowCount( Table rightTable, CompiledExpression condition )#

Computes the number of rows from the result of a left join between two tables when a conditional expression is true.

It is assumed this table instance holds the columns from the left table, and the table argument represents the columns from the right table.

Parameters:

rightTable – the right side table of the join in the join
condition – conditional expression to evaluate during the join

Returns:

row count for the join result

public GatherMap[] conditionalLeftJoinGatherMaps( Table rightTable, CompiledExpression condition )#

Computes the gather maps that can be used to manifest the result of a left join between two tables when a conditional expression is true.

It is assumed this table instance holds the columns from the left table, and the table argument represents the columns from the right table. Two GatherMap instances will be returned that can be used to gather the left and right tables, respectively, to produce the result of the left join.

It is the responsibility of the caller to close the resulting gather map instances.

Parameters:

rightTable – the right side table of the join in the join
condition – conditional expression to evaluate during the join

Returns:

left and right table gather maps

public GatherMap[] conditionalLeftJoinGatherMaps( Table rightTable, CompiledExpression condition, long outputRowCount )#

Computes the gather maps that can be used to manifest the result of a left join between two tables when a conditional expression is true.

It is the responsibility of the caller to close the resulting gather map instances.

This interface allows passing an output row count that was previously computed from conditionalLeftJoinRowCount(Table, CompiledExpression).

WARNING: Passing a row count that is smaller than the actual row count will result in undefined behavior.

Parameters:

rightTable – the right side table of the join in the join
condition – conditional expression to evaluate during the join
outputRowCount – number of output rows in the join result

Returns:

left and right table gather maps

public GatherMap[] innerJoinGatherMaps( Table rightKeys, boolean compareNullsEqual )#

Computes the gather maps that can be used to manifest the result of an inner equi-join between two tables.

It is the responsibility of the caller to close the resulting gather map instances.

Parameters:

rightKeys – join key columns from the right table
compareNullsEqual – true if null key values should match otherwise false

Returns:

left and right table gather maps

public GatherMap[] innerDistinctJoinGatherMaps( Table rightKeys, boolean compareNullsEqual )#

Computes the gather maps that can be used to manifest the result of an inner equi-join between two tables where the right table is guaranteed to not contain any duplicated join keys.

It is the responsibility of the caller to close the resulting gather map instances.

Parameters:

rightKeys – join key columns from the right table
compareNullsEqual – true if null key values should match otherwise false

Returns:

left and right table gather maps

public GatherMap[] innerDistinctJoinGatherMaps( DistinctHashJoin rightHash )#

Computes the gather maps that can be used to manifest the result of an inner equi-join between two tables where the right table is guaranteed not to contain any duplicated join keys.

It is assumed this table instance holds the key columns from the left table, and the DistinctHashJoin argument has been constructed from the key columns from the right table. Two GatherMap instances will be returned that can be used to gather the left and right tables, respectively, to produce the result of the inner join.

It is the responsibility of the caller to close the resulting gather map instances.

Parameters:: rightHash – hash table built from distinct join key columns from the right table
Returns:: left and right table gather maps

public long innerJoinRowCount(HashJoin otherHash)#

Computes the number of rows resulting from an inner equi-join between two tables.

Parameters:: otherHash – hash table built from join key columns from the other table
Returns:: row count of the join result

public GatherMap[] innerJoinGatherMaps(HashJoin rightHash)#

Computes the gather maps that can be used to manifest the result of an inner equi-join between two tables.

It is the responsibility of the caller to close the resulting gather map instances.

Parameters:: rightHash – hash table built from join key columns from the right table
Returns:: left and right table gather maps

public GatherMap[] innerJoinGatherMaps( HashJoin rightHash, long outputRowCount )#

Computes the gather maps that can be used to manifest the result of an inner equi-join between two tables.

It is the responsibility of the caller to close the resulting gather map instances.

This interface allows passing an output row count that was previously computed from innerJoinRowCount(HashJoin). Doing so no longer avoids any work: the output size is always computed internally and the supplied count is only validated against it. Prefer innerJoinGatherMaps(HashJoin); this overload will be deprecated in a future release.

Throws:

CudfException – if outputRowCount does not equal the actual output row count

Parameters:

rightHash – hash table built from join key columns from the right table
outputRowCount – number of output rows in the join result

Returns:

left and right table gather maps

public long conditionalInnerJoinRowCount( Table rightTable, CompiledExpression condition )#

Computes the number of rows from the result of an inner join between two tables when a conditional expression is true.

It is assumed this table instance holds the columns from the left table, and the table argument represents the columns from the right table.

Parameters:

rightTable – the right side table of the join in the join
condition – conditional expression to evaluate during the join

Returns:

row count for the join result

public GatherMap[] conditionalInnerJoinGatherMaps( Table rightTable, CompiledExpression condition )#

Computes the gather maps that can be used to manifest the result of an inner join between two tables when a conditional expression is true.

It is the responsibility of the caller to close the resulting gather map instances.

Parameters:

rightTable – the right side table of the join
condition – conditional expression to evaluate during the join

Returns:

left and right table gather maps

public GatherMap[] conditionalInnerJoinGatherMaps( Table rightTable, CompiledExpression condition, long outputRowCount )#

Computes the gather maps that can be used to manifest the result of an inner join between two tables when a conditional expression is true.

It is the responsibility of the caller to close the resulting gather map instances.

This interface allows passing an output row count that was previously computed from conditionalInnerJoinRowCount(Table, CompiledExpression).

WARNING: Passing a row count that is smaller than the actual row count will result in undefined behavior.

Parameters:

rightTable – the right side table of the join in the join
condition – conditional expression to evaluate during the join
outputRowCount – number of output rows in the join result

Returns:

left and right table gather maps

public GatherMap[] fullJoinGatherMaps( Table rightKeys, boolean compareNullsEqual )#

Computes the gather maps that can be used to manifest the result of an full equi-join between two tables.

It is the responsibility of the caller to close the resulting gather map instances.

Parameters:

rightKeys – join key columns from the right table
compareNullsEqual – true if null key values should match otherwise false

Returns:

left and right table gather maps

public long fullJoinRowCount(HashJoin rightHash)#

Computes the number of rows resulting from a full equi-join between two tables.

It is assumed this table instance holds the key columns from the left table, and the HashJoin argument has been constructed from the key columns from the right table. Like leftJoinRowCount(HashJoin) and innerJoinRowCount(HashJoin), this repeats work that fullJoinGatherMaps(HashJoin) performs anyway, so only call it when the row count is needed before manifesting the gather maps.

Parameters:: rightHash – hash table built from join key columns from the right table
Returns:: row count of the join result

public GatherMap[] fullJoinGatherMaps(HashJoin rightHash)#

Computes the gather maps that can be used to manifest the result of a full equi-join between two tables.

It is the responsibility of the caller to close the resulting gather map instances.

Parameters:: rightHash – hash table built from join key columns from the right table
Returns:: left and right table gather maps

public GatherMap[] fullJoinGatherMaps( HashJoin rightHash, long outputRowCount )#

Computes the gather maps that can be used to manifest the result of a full equi-join between two tables.

It is the responsibility of the caller to close the resulting gather map instances.

This interface allows passing an output row count that was previously computed from fullJoinRowCount(HashJoin). Doing so no longer avoids any work: the output size is always computed internally and the supplied count is only validated against it. Prefer fullJoinGatherMaps(HashJoin); this overload will be deprecated in a future release.

Throws:

CudfException – if outputRowCount does not equal the actual output row count

Parameters:

rightHash – hash table built from join key columns from the right table
outputRowCount – number of output rows in the join result

Returns:

left and right table gather maps

public GatherMap[] conditionalFullJoinGatherMaps( Table rightTable, CompiledExpression condition )#

Computes the gather maps that can be used to manifest the result of a full join between two tables when a conditional expression is true.

It is the responsibility of the caller to close the resulting gather map instances.

Parameters:

rightTable – the right side table of the join
condition – conditional expression to evaluate during the join

Returns:

left and right table gather maps

public GatherMap leftSemiJoinGatherMap( Table rightKeys, boolean compareNullsEqual )#

Computes the gather map that can be used to manifest the result of a left semi-join between two tables.

It is assumed this table instance holds the key columns from the left table, and the table argument represents the key columns from the right table. The GatherMap instance returned can be used to gather the left table to produce the result of the left semi-join. It is the responsibility of the caller to close the resulting gather map instance.

Parameters:

rightKeys – join key columns from the right table
compareNullsEqual – true if null key values should match otherwise false

Returns:

left table gather map

public GatherMap leftSemiJoinGatherMap(FilteredJoin rightFilter)#

Computes the gather map that can be used to manifest the result of a left semi-join between two tables.

It is assumed this table instance holds the key columns from the left table, and the FilteredJoin argument represents a reusable lookup built from the key columns from the right table. The GatherMap instance returned can be used to gather the left table to produce the result of the left semi-join. It is the responsibility of the caller to close the resulting gather map instance.

Parameters:: rightFilter – reusable lookup built from join key columns from the right table
Returns:: left table gather map

public long conditionalLeftSemiJoinRowCount( Table rightTable, CompiledExpression condition )#

Computes the number of rows from the result of a left semi join between two tables when a conditional expression is true.

It is assumed this table instance holds the columns from the left table, and the table argument represents the columns from the right table.

Parameters:

rightTable – the right side table of the join in the join
condition – conditional expression to evaluate during the join

Returns:

row count for the join result

public GatherMap conditionalLeftSemiJoinGatherMap( Table rightTable, CompiledExpression condition )#

Computes the gather map that can be used to manifest the result of a left semi join between two tables when a conditional expression is true.

Parameters:

rightTable – the right side table of the join
condition – conditional expression to evaluate during the join

Returns:

left table gather map

public GatherMap conditionalLeftSemiJoinGatherMap( Table rightTable, CompiledExpression condition, long outputRowCount )#

Computes the gather map that can be used to manifest the result of a left semi join between two tables when a conditional expression is true.

It is assumed this table instance holds the columns from the left table, and the table argument represents the columns from the right table. The GatherMap instance returned can be used to gather the left table to produce the result of the left semi join. It is the responsibility of the caller to close the resulting gather map instance. This interface allows passing an output row count that was previously computed from conditionalLeftSemiJoinRowCount(Table, CompiledExpression). WARNING: Passing a row count that is smaller than the actual row count will result in undefined behavior.

Parameters:

rightTable – the right side table of the join
condition – conditional expression to evaluate during the join
outputRowCount – number of output rows in the join result

Returns:

left table gather map

public GatherMap leftAntiJoinGatherMap( Table rightKeys, boolean compareNullsEqual )#

Computes the gather map that can be used to manifest the result of a left anti-join between two tables.

It is assumed this table instance holds the key columns from the left table, and the table argument represents the key columns from the right table. The GatherMap instance returned can be used to gather the left table to produce the result of the left anti-join. It is the responsibility of the caller to close the resulting gather map instance.

Parameters:

rightKeys – join key columns from the right table
compareNullsEqual – true if null key values should match otherwise false

Returns:

left table gather map

public GatherMap leftAntiJoinGatherMap(FilteredJoin rightFilter)#

Computes the gather map that can be used to manifest the result of a left anti-join between two tables.

It is assumed this table instance holds the key columns from the left table, and the FilteredJoin argument represents a reusable lookup built from the key columns from the right table. The GatherMap instance returned can be used to gather the left table to produce the result of the left anti-join. It is the responsibility of the caller to close the resulting gather map instance.

Parameters:: rightFilter – reusable lookup built from join key columns from the right table
Returns:: left table gather map

public long conditionalLeftAntiJoinRowCount( Table rightTable, CompiledExpression condition )#

Computes the number of rows from the result of a left anti join between two tables when a conditional expression is true.

It is assumed this table instance holds the columns from the left table, and the table argument represents the columns from the right table.

Parameters:

rightTable – the right side table of the join in the join
condition – conditional expression to evaluate during the join

Returns:

row count for the join result

public GatherMap conditionalLeftAntiJoinGatherMap( Table rightTable, CompiledExpression condition )#

Computes the gather map that can be used to manifest the result of a left anti join between two tables when a conditional expression is true.

Parameters:

rightTable – the right side table of the join
condition – conditional expression to evaluate during the join

Returns:

left table gather map

public GatherMap conditionalLeftAntiJoinGatherMap( Table rightTable, CompiledExpression condition, long outputRowCount )#

Computes the gather map that can be used to manifest the result of a left anti join between two tables when a conditional expression is true.

It is assumed this table instance holds the columns from the left table, and the table argument represents the columns from the right table. The GatherMap instance returned can be used to gather the left table to produce the result of the left anti join. It is the responsibility of the caller to close the resulting gather map instance. This interface allows passing an output row count that was previously computed from conditionalLeftAntiJoinRowCount(Table, CompiledExpression). WARNING: Passing a row count that is smaller than the actual row count will result in undefined behavior.

Parameters:

rightTable – the right side table of the join
condition – conditional expression to evaluate during the join
outputRowCount – number of output rows in the join result

Returns:

left table gather map

public Table sample(long n, boolean replacement, long seed)#

Gather n samples from table randomly Note: does not preserve the ordering Example: input: {col1: {1, 2, 3, 4, 5}, col2: {6, 7, 8, 9, 10}} n: 3 replacement: false.

output: {col1: {3, 1, 4}, col2: {8, 6, 9}}

replacement: true

output: {col1: {3, 1, 1}, col2: {8, 6, 6}}

throws “logic_error” if n > table rows and replacement == FALSE. throws “logic_error” if n < 0.

Parameters:

n – non-negative number of samples expected from table
replacement – Allow or disallow sampling of the same row more than once.
seed – Seed value to initiate random number generator.

Returns:

Table containing samples

Static Methods

public static Table readCSV(Schema schema, File path)#

Read a CSV file using the default CSVOptions.

Parameters:

schema – the schema of the file. You may use Schema.INFERRED to infer the schema.
path – the local file to read.

Returns:

the file parsed as a table on the GPU.

public static Table readCSV(Schema schema, CSVOptions opts, File path)#

Read a CSV file.

Parameters:

schema – the schema of the file. You may use Schema.INFERRED to infer the schema.
opts – various CSV parsing options.
path – the local file to read.

Returns:

the file parsed as a table on the GPU.

public static Table readCSV(Schema schema, byte[] buffer)#

Read CSV formatted data using the default CSVOptions.

Parameters:

schema – the schema of the data. You may use Schema.INFERRED to infer the schema.
buffer – raw UTF8 formatted bytes.

Returns:

the data parsed as a table on the GPU.

public static Table readCSV( Schema schema, CSVOptions opts, byte[] buffer )#

Read CSV formatted data.

Parameters:

schema – the schema of the data. You may use Schema.INFERRED to infer the schema.
opts – various CSV parsing options.
buffer – raw UTF8 formatted bytes.

Returns:

the data parsed as a table on the GPU.

public static Table readCSV( Schema schema, CSVOptions opts, byte[] buffer, long offset, long len, HostMemoryAllocator hostMemoryAllocator )#

Read CSV formatted data.

Parameters:

schema – the schema of the data. You may use Schema.INFERRED to infer the schema.
opts – various CSV parsing options.
buffer – raw UTF8 formatted bytes.
offset – the starting offset into buffer.
len – the number of bytes to parse.
hostMemoryAllocator – allocator for host memory buffers

Returns:

the data parsed as a table on the GPU.

public static Table readCSV( Schema schema, CSVOptions opts, byte[] buffer, long offset, long len )#

public static Table readCSV( Schema schema, CSVOptions opts, HostMemoryBuffer buffer, long offset, long len )#

Read CSV formatted data.

Parameters:

schema – the schema of the data. You may use Schema.INFERRED to infer the schema.
opts – various CSV parsing options.
buffer – raw UTF8 formatted bytes.
offset – the starting offset into buffer.
len – the number of bytes to parse.

Returns:

the data parsed as a table on the GPU.

public static Table readCSV( Schema schema, CSVOptions opts, DataSource ds )#

public static TableWriter getCSVBufferWriter( CSVWriterOptions options, HostBufferConsumer bufferConsumer, HostMemoryAllocator hostMemoryAllocator )#

public static TableWriter getCSVBufferWriter( CSVWriterOptions options, HostBufferConsumer bufferConsumer )#

public static Table readJSON(Schema schema, File path)#

Read a JSON file using the default JSONOptions.

Parameters:

schema – the schema of the file. You may use Schema.INFERRED to infer the schema.
path – the local file to read.

Returns:

the file parsed as a table on the GPU.

public static Table readJSON(Schema schema, byte[] buffer)#

Read JSON formatted data using the default JSONOptions.

Parameters:

schema – the schema of the data. You may use Schema.INFERRED to infer the schema.
buffer – raw UTF8 formatted bytes.

Returns:

the data parsed as a table on the GPU.

public static Table readJSON( Schema schema, JSONOptions opts, byte[] buffer )#

Read JSON formatted data.

Parameters:

schema – the schema of the data. You may use Schema.INFERRED to infer the schema.
opts – various JSON parsing options.
buffer – raw UTF8 formatted bytes.

Returns:

the data parsed as a table on the GPU.

public static Table readJSON( Schema schema, JSONOptions opts, File path )#

Read a JSON file.

Parameters:

schema – the schema of the file. You may use Schema.INFERRED to infer the schema.
opts – various JSON parsing options.
path – the local file to read.

Returns:

the file parsed as a table on the GPU.

public static Table readJSON( Schema schema, JSONOptions opts, byte[] buffer, long offset, long len, HostMemoryAllocator hostMemoryAllocator )#

Read JSON formatted data.

Parameters:

schema – the schema of the data. You may use Schema.INFERRED to infer the schema.
opts – various JSON parsing options.
buffer – raw UTF8 formatted bytes.
offset – the starting offset into buffer.
len – the number of bytes to parse.
hostMemoryAllocator – allocator for host memory buffers

Returns:

the data parsed as a table on the GPU.

public static Table readJSON( Schema schema, JSONOptions opts, byte[] buffer, long offset, long len, HostMemoryAllocator hostMemoryAllocator, int emptyRowCount ) Deprecated#

Read JSON formatted data.

Deprecated:: This method is deprecated since emptyRowCount is not used. Use the method without emptyRowCount instead.

Parameters:

schema – the schema of the data. You may use Schema.INFERRED to infer the schema.
opts – various JSON parsing options.
buffer – raw UTF8 formatted bytes.
offset – the starting offset into buffer.
len – the number of bytes to parse.
hostMemoryAllocator – allocator for host memory buffers
emptyRowCount – the number of rows to return if no columns were read.

Returns:

the data parsed as a table on the GPU.

public static Table readJSON( Schema schema, JSONOptions opts, byte[] buffer, long offset, long len, int emptyRowCount )#

public static Table readJSON( Schema schema, JSONOptions opts, byte[] buffer, long offset, long len )#

public static TableWithMeta readJSON( JSONOptions opts, HostMemoryBuffer buffer, long offset, long len )#

Read JSON formatted data and infer the column names and schema.

Parameters:

opts – various JSON parsing options.
buffer – raw UTF8 formatted bytes.
offset – the starting offset into buffer.
len – the number of bytes to parse.

Returns:

the data parsed as a table on the GPU and the metadata for the table returned.

public static TableWithMeta readAndInferJSON( JSONOptions opts, DataSource ds )#

Read JSON formatted data and infer the column names and schema.

Parameters:: opts – various JSON parsing options.
Returns:: the data parsed as a table on the GPU and the metadata for the table returned.

public static Table readJSON( Schema schema, JSONOptions opts, HostMemoryBuffer buffer, long offset, long len )#

Read JSON formatted data.

Parameters:

schema – the schema of the data. You may use Schema.INFERRED to infer the schema.
opts – various JSON parsing options.
buffer – raw UTF8 formatted bytes.
offset – the starting offset into buffer.
len – the number of bytes to parse.

Returns:

the data parsed as a table on the GPU.

public static Table readJSON( Schema schema, JSONOptions opts, HostMemoryBuffer buffer, long offset, long len, int emptyRowCount ) Deprecated#

Read JSON formatted data.

Deprecated:: This method is deprecated since emptyRowCount is not used. Use the method without emptyRowCount instead.

Parameters:

schema – the schema of the data. You may use Schema.INFERRED to infer the schema.
opts – various JSON parsing options.
buffer – raw UTF8 formatted bytes.
offset – the starting offset into buffer.
len – the number of bytes to parse.
emptyRowCount – the number of rows to use if no columns were found.

Returns:

the data parsed as a table on the GPU.

public static Table readJSON( Schema schema, JSONOptions opts, DataSource ds )#

Read JSON formatted data.

Parameters:

schema – the schema of the data. You may use Schema.INFERRED to infer the schema.
opts – various JSON parsing options.
ds – the DataSource to read from.

Returns:

the data parsed as a table on the GPU.

public static Table readJSON( Schema schema, JSONOptions opts, DataSource ds, int emptyRowCount ) Deprecated#

Read JSON formatted data.

Deprecated:: This method is deprecated since emptyRowCount is not used. Use the method without emptyRowCount instead.

Parameters:

schema – the schema of the data. You may use Schema.INFERRED to infer the schema.
opts – various JSON parsing options.
ds – the DataSource to read from.
emptyRowCount – the number of rows to return if no columns were read.

Returns:

the data parsed as a table on the GPU.

public static Table readParquet(File path)#

Read a Parquet file using the default ParquetOptions.

Parameters:: path – the local file to read.
Returns:: the file parsed as a table on the GPU.

public static Table readParquet(ParquetOptions opts, File path)#

Read a Parquet file.

Parameters:

opts – various parquet parsing options.
path – the local file to read.

Returns:

the file parsed as a table on the GPU.

public static Table readParquet(byte[] buffer)#

Read parquet formatted data.

Parameters:: buffer – raw parquet formatted bytes.
Returns:: the data parsed as a table on the GPU.

public static Table readParquet(ParquetOptions opts, byte[] buffer)#

Read parquet formatted data.

Parameters:

opts – various parquet parsing options.
buffer – raw parquet formatted bytes.

Returns:

the data parsed as a table on the GPU.

public static Table readParquet( ParquetOptions opts, byte[] buffer, long offset, long len, HostMemoryAllocator hostMemoryAllocator )#

Read parquet formatted data.

Parameters:

opts – various parquet parsing options.
buffer – raw parquet formatted bytes.
offset – the starting offset into buffer.
len – the number of bytes to parse.
hostMemoryAllocator – allocator for host memory buffers

Returns:

the data parsed as a table on the GPU.

public static Table readParquet( ParquetOptions opts, byte[] buffer, long offset, long len )#

Read parquet formatted data.

Parameters:

opts – various parquet parsing options.
buffer – raw parquet formatted bytes.
offset – the starting offset into buffer.
len – the number of bytes to parse.

Returns:

the data parsed as a table on the GPU.

public static Table readParquet( ParquetOptions opts, HostMemoryBuffer buffer, long offset, long len )#

Read parquet formatted data.

Parameters:

opts – various parquet parsing options.
buffer – raw parquet formatted bytes.
offset – the starting offset into buffer.
len – the number of bytes to parse.

Returns:

the data parsed as a table on the GPU.

public static Table readParquet( ParquetOptions opts, HostMemoryBuffer... buffers )#

Read parquet formatted data.

Parameters:

opts – various parquet parsing options.
buffers – Buffers containing the Parquet data. The buffers are logically concatenated in order to construct the file being read.

Returns:

the data parsed as a table on the GPU.

public static Table readParquet(ParquetOptions opts, DataSource ds)#

Read parquet formatted data.

Parameters:

opts – various parquet parsing options.
ds – custom datasource to provide the Parquet file data

Returns:

the data parsed as a table on the GPU.

public static Table readAvro(File path)#

Read an Avro file using the default AvroOptions.

Parameters:: path – the local file to read.
Returns:: the file parsed as a table on the GPU.

public static Table readAvro(AvroOptions opts, File path)#

Read an Avro file.

Parameters:

opts – various Avro parsing options.
path – the local file to read.

Returns:

the file parsed as a table on the GPU.

public static Table readAvro(byte[] buffer)#

Read Avro formatted data.

Parameters:: buffer – raw Avro formatted bytes.
Returns:: the data parsed as a table on the GPU.

public static Table readAvro(AvroOptions opts, byte[] buffer)#

Read Avro formatted data.

Parameters:

opts – various Avro parsing options.
buffer – raw Avro formatted bytes.

Returns:

the data parsed as a table on the GPU.

public static Table readAvro( AvroOptions opts, byte[] buffer, long offset, long len, HostMemoryAllocator hostMemoryAllocator )#

Read Avro formatted data.

Parameters:

opts – various Avro parsing options.
buffer – raw Avro formatted bytes.
offset – the starting offset into buffer.
len – the number of bytes to parse.
hostMemoryAllocator – allocator for host memory buffers

Returns:

the data parsed as a table on the GPU.

public static Table readAvro( AvroOptions opts, byte[] buffer, long offset, long len )#

public static Table readAvro( AvroOptions opts, HostMemoryBuffer buffer, long offset, long len )#

Read Avro formatted data.

Parameters:

opts – various Avro parsing options.
buffer – raw Avro formatted bytes.
offset – the starting offset into buffer.
len – the number of bytes to parse.

Returns:

the data parsed as a table on the GPU.

public static Table readAvro(AvroOptions opts, DataSource ds)#

public static Table readORC(File path)#

Read a ORC file using the default ORCOptions.

Parameters:: path – the local file to read.
Returns:: the file parsed as a table on the GPU.

public static Table readORC(ORCOptions opts, File path)#

Read a ORC file.

Parameters:

opts – ORC parsing options.
path – the local file to read.

Returns:

the file parsed as a table on the GPU.

public static Table readORC(byte[] buffer)#

Read ORC formatted data.

Parameters:: buffer – raw ORC formatted bytes.
Returns:: the data parsed as a table on the GPU.

public static Table readORC(ORCOptions opts, byte[] buffer)#

Read ORC formatted data.

Parameters:

opts – various ORC parsing options.
buffer – raw ORC formatted bytes.

Returns:

the data parsed as a table on the GPU.

public static Table readORC( ORCOptions opts, byte[] buffer, long offset, long len, HostMemoryAllocator hostMemoryAllocator )#

Read ORC formatted data.

Parameters:

opts – various ORC parsing options.
buffer – raw ORC formatted bytes.
offset – the starting offset into buffer.
len – the number of bytes to parse.
hostMemoryAllocator – allocator for host memory buffers

Returns:

the data parsed as a table on the GPU.

public static Table readORC( ORCOptions opts, byte[] buffer, long offset, long len )#

public static Table readORC( ORCOptions opts, HostMemoryBuffer buffer, long offset, long len )#

Read ORC formatted data.

Parameters:

opts – various ORC parsing options.
buffer – raw ORC formatted bytes.
offset – the starting offset into buffer.
len – the number of bytes to parse.

Returns:

the data parsed as a table on the GPU.

public static Table readORC(ORCOptions opts, DataSource ds)#

public static ParquetTableWriter writeParquetChunked( ParquetWriterOptions options, File outputFile )#

Get a table writer to write parquet data to a file.

Parameters:

options – the parquet writer options.
outputFile – where to write the file.

Returns:

a Parquet table writer to use for writing out multiple tables.

public static ParquetTableWriter writeParquetChunked( ParquetWriterOptions options, HostBufferConsumer consumer, HostMemoryAllocator hostMemoryAllocator )#

Get a table writer to write parquet data and handle each chunk with a callback.

Parameters:

options – the parquet writer options.
consumer – a class that will be called when host buffers are ready with parquet formatted data in them.
hostMemoryAllocator – allocator for host memory buffers

Returns:

a Parquet table writer to use for writing out multiple tables.

public static ParquetTableWriter writeParquetChunked( ParquetWriterOptions options, HostBufferConsumer consumer )#

public static void writeColumnViewsToParquet( ParquetWriterOptions options, HostBufferConsumer consumer, HostMemoryAllocator hostMemoryAllocator, ColumnView... columnViews )#

This is an evolving API and most likely be removed in future releases.

Please use with the caveat that this will not exist in the near future.

Parameters:

options – the Parquet writer options.
consumer – a class that will be called when host buffers are ready with Parquet formatted data in them.
hostMemoryAllocator – allocator for host memory buffers
columnViews – ColumnViews to write to Parquet

public static void writeColumnViewsToParquet( ParquetWriterOptions options, HostBufferConsumer consumer, ColumnView... columnViews )#

public static TableWriter writeORCChunked( ORCWriterOptions options, File outputFile )#

Get a table writer to write ORC data to a file.

Parameters:

options – the ORC writer options.
outputFile – where to write the file.

Returns:

a table writer to use for writing out multiple tables.

public static TableWriter writeORCChunked( ORCWriterOptions options, HostBufferConsumer consumer, HostMemoryAllocator hostMemoryAllocator )#

Get a table writer to write ORC data and handle each chunk with a callback.

Parameters:

options – the ORC writer options.
consumer – a class that will be called when host buffers are ready with ORC formatted data in them.
hostMemoryAllocator – allocator for host memory buffers

Returns:

a table writer to use for writing out multiple tables.

public static TableWriter writeORCChunked( ORCWriterOptions options, HostBufferConsumer consumer )#

public static TableWriter writeArrowIPCChunked( ArrowIPCWriterOptions options, File outputFile )#

Get a table writer to write arrow IPC data to a file.

Parameters:

options – the arrow IPC writer options.
outputFile – where to write the file.

Returns:

a table writer to use for writing out multiple tables.

public static TableWriter writeArrowIPCChunked( ArrowIPCWriterOptions options, HostBufferConsumer consumer, HostMemoryAllocator hostMemoryAllocator )#

Get a table writer to write arrow IPC data and handle each chunk with a callback.

Parameters:

options – the arrow IPC writer options.
consumer – a class that will be called when host buffers are ready with arrow IPC formatted data in them.
hostMemoryAllocator – allocator for host memory buffers

Returns:

a table writer to use for writing out multiple tables.

public static TableWriter writeArrowIPCChunked( ArrowIPCWriterOptions options, HostBufferConsumer consumer )#

public static StreamedTableReader readArrowIPCChunked( ArrowIPCOptions options, File inputFile )#

Get a reader that will return tables.

Parameters:

options – options for reading.
inputFile – the file to read the Arrow IPC formatted data from

Returns:

a reader.

public static StreamedTableReader readArrowIPCChunked(File inputFile)#

Get a reader that will return tables.

Parameters:: inputFile – the file to read the Arrow IPC formatted data from
Returns:: a reader.

public static StreamedTableReader readArrowIPCChunked( ArrowIPCOptions options, HostBufferProvider provider, HostMemoryAllocator hostMemoryAllocator )#

Get a reader that will return tables.

Parameters:

options – options for reading.
provider – what will provide the data being read.

Returns:

a reader.

public static StreamedTableReader readArrowIPCChunked( ArrowIPCOptions options, HostBufferProvider provider )#

public static StreamedTableReader readArrowIPCChunked( HostBufferProvider provider )#

Get a reader that will return tables.

Parameters:: provider – what will provide the data being read.
Returns:: a reader.

public static Table concatenate(Table... tables)#

Concatenate multiple tables together to form a single table.

The schema of each table (i.e.: number of columns and types of each column) must be equal across all tables and will determine the schema of the resulting table.

public static Table merge(Table[] tables, OrderByArg... args)#

Merge multiple already sorted tables keeping the sort order the same.

This is a more efficient version of concatenate followed by orderBy, but requires that the input already be sorted.

Parameters:

tables – the tables that should be merged.
args – the ordering of the tables. Should match how they were sorted initially.

Returns:

a combined sorted table.

public static Table merge(List<Table> tables, OrderByArg... args)#

Merge multiple already sorted tables keeping the sort order the same.

This is a more efficient version of concatenate followed by orderBy, but requires that the input already be sorted.

Parameters:

tables – the tables that should be merged.
args – the ordering of the tables. Should match how they were sorted initially.

Returns:

a combined sorted table.

public static Table scatter( Scalar[] source, ColumnView scatterMap, Table target )#

Scatters values from the source rows into the target table out-of-place, returning a new result table.

The scatter is performed according to a scatter map such that row scatterMap[i] of the destination table is replaced by the source row i. All other rows of the destination table equal corresponding rows of the target table.

The number of elements in source must match the number of columns in target and their corresponding data types must be the same.

If the same index appears more than once in the scatter map, the result is undefined.

A negative value i in the scatterMap is interpreted as i + n, where n is the number of rows in the target table.

Parameters:

source – The input scalars containing values to be scattered into the target table.
scatterMap – The map of indexes. Must be non-nullable and integral type.
target – The table into which the values from source are to be scattered out-of-place.

Returns:

A new table which is the result of out-of-place scattering the source values into the target table.

public static GatherMap[] filterJoinGatherMaps( GatherMap leftGatherMap, GatherMap rightGatherMap, Table leftTable, Table rightTable, CompiledExpression condition, JoinKind joinKind )#

Filters a pair of join gather maps by evaluating a conditional expression on the corresponding rows from the left and right tables.

The maps must be the paired results of an equality join of the same kind as joinKind: INNER maps for JoinKind.INNER, LEFT maps for JoinKind.LEFT, and FULL maps for JoinKind.FULL. For example, maps from leftJoinGatherMaps(HashJoin) can be filtered with JoinKind.LEFT. Equivalent equality-join maps from other producers are also supported. Each conditional table must have the same row count and row numbering as its corresponding equality-join source table; its columns may differ. The maps must have the same length, and entries at the same position identify a candidate row pair. The join origin and index validity are not checked, and converting maps between join kinds is unsupported.

Integer.MIN_VALUE denotes an unmatched row in an outer-join map. Such pairs pass through without evaluating the condition. For pairs with two valid indices, the condition must produce a Boolean result; false or null means no match. LEFT and FULL retain one unmatched entry for each retained-side row with no passing candidate. Empty input maps produce empty output maps; this method does not complete an outer join from empty INNER maps. In particular, LEFT maps for a nonempty left table and an empty right table must already contain the unmatched left rows.

The input gather maps are not modified or closed. Two new GatherMap instances with independent storage are returned for the left and right tables, respectively. The outputs remain valid after closing the inputs, and closing the outputs does not prevent reusing the inputs. Output row order is unspecified.

It is the responsibility of the caller to close the resulting gather map instances.

Parameters:

leftGatherMap – input gather map for the left table
rightGatherMap – input gather map for the right table
leftTable – left table containing the columns referenced by the condition
rightTable – right table containing the columns referenced by the condition
condition – Boolean conditional expression to evaluate for each valid pair
joinKind – kind of the input equality join and the filtered output join

Throws:

IllegalArgumentException – if the input gather maps have different lengths

Returns:

filtered left and right table gather maps

public static MixedJoinSize mixedLeftJoinSize( Table leftKeys, Table rightKeys, Table leftConditional, Table rightConditional, CompiledExpression condition, NullEquality nullEquality )#

Computes output size information for a left join between two tables using a mix of equality and inequality conditions.

The entire join condition is assumed to be a logical AND of the equality condition and inequality condition. NOTE: It is the responsibility of the caller to close the resulting size information object or native resources can be leaked!

Parameters:

leftKeys – the left table’s key columns for the equality condition
rightKeys – the right table’s key columns for the equality condition
leftConditional – the left table’s columns needed to evaluate the inequality condition
rightConditional – the right table’s columns needed to evaluate the inequality condition
condition – the inequality condition of the join
nullEquality – whether nulls should compare as equal

Returns:

size information for the join

public static GatherMap[] mixedLeftJoinGatherMaps( Table leftKeys, Table rightKeys, Table leftConditional, Table rightConditional, CompiledExpression condition, NullEquality nullEquality )#

Computes the gather maps that can be used to manifest the result of a left join between two tables using a mix of equality and inequality conditions.

The entire join condition is assumed to be a logical AND of the equality condition and inequality condition. Two GatherMap instances will be returned that can be used to gather the left and right tables, respectively, to produce the result of the left join.

It is the responsibility of the caller to close the resulting gather map instances.

Parameters:

leftKeys – the left table’s key columns for the equality condition
rightKeys – the right table’s key columns for the equality condition
leftConditional – the left table’s columns needed to evaluate the inequality condition
rightConditional – the right table’s columns needed to evaluate the inequality condition
condition – the inequality condition of the join
nullEquality – whether nulls should compare as equal

Returns:

left and right table gather maps

Computes the gather maps that can be used to manifest the result of a left join between two tables using a mix of equality and inequality conditions.

It is the responsibility of the caller to close the resulting gather map instances.

This interface allows passing the size result from mixedLeftJoinSize(Table, Table, Table, Table, CompiledExpression, NullEquality) when the output size was computed previously.

Parameters:

leftKeys – the left table’s key columns for the equality condition
rightKeys – the right table’s key columns for the equality condition
leftConditional – the left table’s columns needed to evaluate the inequality condition
rightConditional – the right table’s columns needed to evaluate the inequality condition
condition – the inequality condition of the join
nullEquality – whether nulls should compare as equal
joinSize – mixed join size result

Returns:

left and right table gather maps

public static MixedJoinSize mixedInnerJoinSize( Table leftKeys, Table rightKeys, Table leftConditional, Table rightConditional, CompiledExpression condition, NullEquality nullEquality )#

Computes output size information for an inner join between two tables using a mix of equality and inequality conditions.

Parameters:

leftKeys – the left table’s key columns for the equality condition
rightKeys – the right table’s key columns for the equality condition
leftConditional – the left table’s columns needed to evaluate the inequality condition
rightConditional – the right table’s columns needed to evaluate the inequality condition
condition – the inequality condition of the join
nullEquality – whether nulls should compare as equal

Returns:

size information for the join

public static GatherMap[] mixedInnerJoinGatherMaps( Table leftKeys, Table rightKeys, Table leftConditional, Table rightConditional, CompiledExpression condition, NullEquality nullEquality )#

Computes the gather maps that can be used to manifest the result of an inner join between two tables using a mix of equality and inequality conditions.

It is the responsibility of the caller to close the resulting gather map instances.

Parameters:

leftKeys – the left table’s key columns for the equality condition
rightKeys – the right table’s key columns for the equality condition
leftConditional – the left table’s columns needed to evaluate the inequality condition
rightConditional – the right table’s columns needed to evaluate the inequality condition
condition – the inequality condition of the join
nullEquality – whether nulls should compare as equal

Returns:

left and right table gather maps

Computes the gather maps that can be used to manifest the result of an inner join between two tables using a mix of equality and inequality conditions.

It is the responsibility of the caller to close the resulting gather map instances.

This interface allows passing the size result from mixedInnerJoinSize(Table, Table, Table, Table, CompiledExpression, NullEquality) when the output size was computed previously.

Parameters:

leftKeys – the left table’s key columns for the equality condition
rightKeys – the right table’s key columns for the equality condition
leftConditional – the left table’s columns needed to evaluate the inequality condition
rightConditional – the right table’s columns needed to evaluate the inequality condition
condition – the inequality condition of the join
nullEquality – whether nulls should compare as equal
joinSize – mixed join size result

Returns:

left and right table gather maps

public static GatherMap[] mixedFullJoinGatherMaps( Table leftKeys, Table rightKeys, Table leftConditional, Table rightConditional, CompiledExpression condition, NullEquality nullEquality )#

Computes the gather maps that can be used to manifest the result of a full join between two tables using a mix of equality and inequality conditions.

It is the responsibility of the caller to close the resulting gather map instances.

Parameters:

leftKeys – the left table’s key columns for the equality condition
rightKeys – the right table’s key columns for the equality condition
leftConditional – the left table’s columns needed to evaluate the inequality condition
rightConditional – the right table’s columns needed to evaluate the inequality condition
condition – the inequality condition of the join
nullEquality – whether nulls should compare as equal

Returns:

left and right table gather maps

public static GatherMap mixedLeftSemiJoinGatherMap( Table leftKeys, Table rightKeys, Table leftConditional, Table rightConditional, CompiledExpression condition, NullEquality nullEquality )#

Computes the gather map that can be used to manifest the result of a left semi join between two tables using a mix of equality and inequality conditions.

The entire join condition is assumed to be a logical AND of the equality condition and inequality condition. A GatherMap instance will be returned that can be used to gather the left table to produce the result of the left semi join.

It is the responsibility of the caller to close the resulting gather map instances.

Parameters:

leftKeys – the left table’s key columns for the equality condition
rightKeys – the right table’s key columns for the equality condition
leftConditional – the left table’s columns needed to evaluate the inequality condition
rightConditional – the right table’s columns needed to evaluate the inequality condition
condition – the inequality condition of the join
nullEquality – whether nulls should compare as equal

Returns:

left and right table gather maps

public static GatherMap mixedLeftAntiJoinGatherMap( Table leftKeys, Table rightKeys, Table leftConditional, Table rightConditional, CompiledExpression condition, NullEquality nullEquality )#

Computes the gather map that can be used to manifest the result of a left anti join between two tables using a mix of equality and inequality conditions.

It is the responsibility of the caller to close the resulting gather map instances.

Parameters:

leftKeys – the left table’s key columns for the equality condition
rightKeys – the right table’s key columns for the equality condition
leftConditional – the left table’s columns needed to evaluate the inequality condition
rightConditional – the right table’s columns needed to evaluate the inequality condition
condition – the inequality condition of the join
nullEquality – whether nulls should compare as equal

Returns:

left and right table gather maps

public static Table fromPackedTable( ByteBuffer metadata, DeviceMemoryBuffer data )#

Construct a table from a packed representation.

Parameters:

metadata – host-based metadata for the table
data – GPU data buffer for the table

Returns:

table which is zero-copy reconstructed from the packed-form

public enum DuplicateKeepOption#

Enum to specify which of duplicate rows/elements will be copied to the output.

Enum Constant Summary

`KEEP_ANY`
`KEEP_FIRST`
`KEEP_LAST`
`KEEP_NONE`

Enum Constants

KEEP_ANY#

KEEP_FIRST#

KEEP_LAST#

KEEP_NONE#

public static final class GroupByOperation#

Class representing groupby operations.

Method Summary

`aggregate(GroupByAggregationOnColumn... aggregates)`	Aggregates the group of columns represented by indices Usage: aggregate(count(), max(2),…); example: input : 1, 1, 1 1, 2, 1 2, 4, 5.
`aggregateWindows(AggregationOverWindow... windowAggregates)`	Computes row-based window aggregation functions on the Table/projection, based on windows specified in the argument.
`aggregateWindowsOverRanges(AggregationOverWindow... windowAggregates)`	Computes range-based window aggregation functions on the Table/projection, based on windows specified in the argument.
`contiguousSplitGroups()`	Splits the groups in a single table into separate tables according to the grouping keys.
`contiguousSplitGroupsAndGenUniqKeys()`	Similar to contiguousSplitGroups, return an extra uniq key table in which each row is corresponding to a group split.
`contiguousSplitGroupsAndGenUniqKeys(int[] projectionColumnIndices)`	Similar to the above contiguousSplitGroupsAndGenUniqKeys.
`replaceNulls(ReplacePolicyWithColumn... replacements)`
`scan(GroupByScanAggregationOnColumn... aggregates)`

Methods

public Table aggregate(GroupByAggregationOnColumn... aggregates)#

Aggregates the group of columns represented by indices Usage: aggregate(count(), max(2),…); example: input : 1, 1, 1 1, 2, 1 2, 4, 5.

table.groupBy(0, 2).count()

    col0, col1

output: 1, 1 1, 2 2, 1 ==> aggregated count

public Table aggregateWindows( AggregationOverWindow... windowAggregates )#

Computes row-based window aggregation functions on the Table/projection, based on windows specified in the argument.

The rows must already be sorted by the columns passed to Table.groupBy; this method does not sort them, and the GroupByOptions sort settings have no effect here. Groups are taken to be runs of adjacent rows with equal values in those columns, so rows with equal values that are not adjacent are aggregated as separate groups. Within a group, the preceding and following rows are counted in the order the rows appear in the table, so if the window should follow some other order (the ORDER BY of the SQL example below), the rows must be sorted by it within each group as well. Unsorted input is not detected and might silently produce incorrect results.

This method enables queries such as the following SQL:

SELECT user_id, MAX(sales_amt) OVER(PARTITION BY user_id ORDER BY date

ROWS BETWEEN 1 PRECEDING and 1 FOLLOWING) FROM my_sales_table WHERE …

Each window-aggregation is represented by a different AggregationOverWindow argument, indicating:

the Aggregation.Kind,
the number of rows preceding and following the current row, within a window,
the minimum number of observations within the defined window

This method returns a Table instance, with one result column for each specified window aggregation.

In this example, for the following input:

[ // user_id, sales_amt { “user1”, 10 }, { “user2”, 20 }, { “user1”, 20 }, { “user1”, 10 }, { “user2”, 30 }, { “user2”, 80 }, { “user1”, 50 }, { “user1”, 60 }, { “user2”, 40 } ]

The input must first be sorted by user_id, which yields the following sales_amt vector (with 2 groups, one for each distinct user_id):

[ 10, 20, 10, 50, 60, 20, 30, 80, 40 ] <—-—user1——–>|<——user2——->

The SUM aggregation is applied with 1 preceding and 1 following row, with a minimum of 1 period. The aggregation window is thus 3 rows wide, yielding the following column:

[ 30, 40, 80, 120, 110, 50, 130, 150, 120 ]

Parameters:: windowAggregates – the window-aggregations to be performed
Throws:: IllegalArgumentException – if the window arguments are not of type WindowOptions.FrameType.ROWS, i.e. a timestamp column is specified for a window-aggregation.
Returns:: Table instance, with each column containing the result of each aggregation.

public Table aggregateWindowsOverRanges( AggregationOverWindow... windowAggregates )#

Computes range-based window aggregation functions on the Table/projection, based on windows specified in the argument.

The rows must already be sorted by the columns passed to Table.groupBy and, within each group, by the order-by columns of the WindowOptions; this method does not sort them, and the GroupByOptions sort settings have no effect here. Groups are taken to be runs of adjacent rows with equal values in the group-by columns, as in aggregateWindows. Within a group, the window bounds are found by searching the order-by columns, so they must be sorted in the direction the WindowOptions gives. For a single order-by column, the placement of nulls is deduced from the data, so the nulls must be at the same end, either the start or the end, of every group; for multiple order-by columns, the nulls must be placed as WindowOptions.Builder.orderByColumns states. Unsorted input is not detected and might silently produce incorrect results.

This method enables queries such as the following SQL:

SELECT user_id, MAX(sales_amt) OVER(PARTITION BY user_id ORDER BY date

RANGE BETWEEN INTERVAL 1 DAY PRECEDING and CURRENT ROW) FROM my_sales_table WHERE …

Each window-aggregation is represented by a different AggregationOverWindow argument, indicating:

the Aggregation.Kind,
the index for the timestamp column to base the window definitions on

the number of DAYS preceding and following the current row’s date, to consider in the window
the minimum number of observations within the defined window

This method returns a Table instance, with one result column for each specified window aggregation.

In this example, for the following input:

[ // user, sales_amt, YYYYMMDD (date) { “user1”, 10, 20200101 }, { “user2”, 20, 20200101 }, { “user1”, 20, 20200102 }, { “user1”, 10, 20200103 }, { “user2”, 30, 20200101 }, { “user2”, 80, 20200102 }, { “user1”, 50, 20200107 }, { “user1”, 60, 20200107 }, { “user2”, 40, 20200104 } ]

The input must first be sorted by user_id and then by date, which yields the following sales_amt vector (with 2 groups, one for each distinct user_id):

Date :(202001-) [ 01, 02, 03, 07, 07, 01, 01, 02, 04 ] Input: [ 10, 20, 10, 50, 60, 20, 30, 80, 40 ] <—-—user1——–>|<———user2———>

The SUM aggregation is applied, with 1 day preceding, and 1 day following, with a minimum of 1 period. The aggregation window is thus 3 days wide, yielding the following output column:

Results: [ 30, 40, 30, 110, 110, 130, 130, 130, 40 ]

Parameters:: windowAggregates – the window-aggregations to be performed
Throws:: IllegalArgumentException – if the window arguments are not of type WindowOptions.FrameType.RANGE or the orderBys are not of (Boolean-exclusive) integral type i.e. the timestamp-column was not specified for the aggregation.
Returns:: Table instance, with each column containing the result of each aggregation.

public Table scan(GroupByScanAggregationOnColumn... aggregates)#

public Table replaceNulls(ReplacePolicyWithColumn... replacements)#

public ContiguousTable[] contiguousSplitGroups()#

Splits the groups in a single table into separate tables according to the grouping keys.

Each split table represents a single group.

This API will be used by some grouping related operators to process the data group by group.

Example: Grouping column index: 0 Input: A table of 3 rows (two groups) a 1 b 2 b 3

Result: Two tables, one group one table. Result[0]: a 1

Result[1]: b 2 b 3

Note, the order of the groups returned is NOT always the same with that in the input table. The split is done in native to avoid copying the offset array to JVM.

Returns:: The tables split according to the groups in the table. NOTE: It is the responsibility of the caller to close the result. Each table and column holds a reference to the original buffer. But both the buffer and the table must be closed for the memory to be released.

public ContigSplitGroupByResult contiguousSplitGroupsAndGenUniqKeys()#

Similar to contiguousSplitGroups, return an extra uniq key table in which each row is corresponding to a group split.

Splits the groups in a single table into separate tables according to the grouping keys. Each split table represents a single group.

Example, see the example in contiguousSplitGroups The uniqKeysTable in ContigSplitGroupByResult is: a b Note: only 2 rows because of only has 2 split groups

Returns:: The split groups and uniq key table.

public ContigSplitGroupByResult contiguousSplitGroupsAndGenUniqKeys( int[] projectionColumnIndices )#

Similar to the above contiguousSplitGroupsAndGenUniqKeys.

The diff with the above method is:

Provide an extra input projectionColumnIndices which defines the columns to output.
The above method outputs keys columns in the split tables, but this method does not except projectionColumnIndices includes key columns.

The split tables only contain the columns defined in the projectionColumnIndices

Parameters:: projectionColumnIndices – Defines the output columns.
Returns:: The split groups and uniq key table.

public static final class TableOperation#

Method Summary

`hashPartition(HashType type, int numberOfPartitions)`	Hash partition a table into the specified number of partitions.
`hashPartition(HashType type, int numberOfPartitions, int seed)`	Hash partition a table into the specified number of partitions.
`hashPartition(int numberOfPartitions)`	Hash partition a table into the specified number of partitions.

Methods

public PartitionedTable hashPartition(int numberOfPartitions)#

Hash partition a table into the specified number of partitions.

Uses the default MURMUR3 hashing.

Parameters:: numberOfPartitions – - number of partitions to use
Returns:: - PartitionedTable - Table that exposes a limited functionality of the Table class

public PartitionedTable hashPartition( HashType type, int numberOfPartitions )#

Hash partition a table into the specified number of partitions.

Parameters:

type – the type of hash to use. Depending on the type of hash different restrictions on the hash column(s) may exist. Not all hash functions are guaranteed to work besides IDENTITY and MURMUR3.
numberOfPartitions – - number of partitions to use

Returns:

PartitionedTable - Table that exposes a limited functionality of the Table class

public PartitionedTable hashPartition( HashType type, int numberOfPartitions, int seed )#

Hash partition a table into the specified number of partitions.

Parameters:

type – the type of hash to use. Depending on the type of hash different restrictions on the hash column(s) may exist. Not all hash functions are guaranteed to work besides IDENTITY and MURMUR3.
numberOfPartitions – number of partitions to use
seed – the seed value for hashing

Returns:

Table that exposes a limited functionality of the Table class

public static final class TestBuilder#

Create a table on the GPU with data from the CPU.

This is not fast and intended mostly for tests.

Method Summary

`build()`
`column(Boolean... values)`
`column(Boolean[]... values)`
`column(Byte... values)`
`column(Byte[]... values)`
`column(Double... values)`
`column(Double[]... values)`
`column(Float... values)`
`column(Float[]... values)`
`column(Integer... values)`
`column(Integer[]... values)`
`column(ListType dataType, List<?>... values)`
`column(Long... values)`
`column(Long[]... values)`
`column(Short... values)`
`column(Short[]... values)`
`column(String... values)`
`column(String[]... values)`
`column(StructType dataType, StructData... values)`
`column(StructType dataType, StructData[]... values)`
`decimal128Column(int scale, RoundingMode mode, BigInteger... values)`
`decimal32Column(int scale, Integer... unscaledValues)`
`decimal32Column(int scale, RoundingMode mode, Double... values)`
`decimal32Column(int scale, RoundingMode mode, String... values)`
`decimal64Column(int scale, Long... unscaledValues)`
`decimal64Column(int scale, RoundingMode mode, Double... values)`
`decimal64Column(int scale, RoundingMode mode, String... values)`
`timestampDayColumn(Integer... values)`
`timestampMicrosecondsColumn(Long... values)`
`timestampMillisecondsColumn(Long... values)`
`timestampNanosecondsColumn(Long... values)`
`timestampSecondsColumn(Long... values)`

Methods

public TestBuilder column(String... values)#

public TestBuilder column(Boolean... values)#

public TestBuilder column(Byte... values)#

public TestBuilder column(Short... values)#

public TestBuilder column(Integer... values)#

public TestBuilder column(Long... values)#

public TestBuilder column(Float... values)#

public TestBuilder column(Double... values)#

public TestBuilder column(ListType dataType, List<?>... values)#

public TestBuilder column(String[]... values)#

public TestBuilder column(Boolean[]... values)#

public TestBuilder column(Byte[]... values)#

public TestBuilder column(Short[]... values)#

public TestBuilder column(Integer[]... values)#

public TestBuilder column(Long[]... values)#

public TestBuilder column(Float[]... values)#

public TestBuilder column(Double[]... values)#

public TestBuilder column(StructType dataType, StructData... values)#

public TestBuilder column(StructType dataType, StructData[]... values)#

public TestBuilder timestampDayColumn(Integer... values)#

public TestBuilder timestampNanosecondsColumn(Long... values)#

public TestBuilder timestampMillisecondsColumn(Long... values)#

public TestBuilder timestampMicrosecondsColumn(Long... values)#

public TestBuilder timestampSecondsColumn(Long... values)#

public TestBuilder decimal32Column( int scale, Integer... unscaledValues )#

public TestBuilder decimal32Column( int scale, RoundingMode mode, Double... values )#

public TestBuilder decimal32Column( int scale, RoundingMode mode, String... values )#

public TestBuilder decimal64Column(int scale, Long... unscaledValues)#

public TestBuilder decimal64Column( int scale, RoundingMode mode, Double... values )#

public TestBuilder decimal64Column( int scale, RoundingMode mode, String... values )#

public TestBuilder decimal128Column( int scale, RoundingMode mode, BigInteger... values )#

public Table build()#