reduce

class pylibcudf.reduce.ApproxDistinctCount(

Table input,

int32_t precision=12,

null_policy null_handling=null_policy.EXCLUDE,

nan_policy nan_handling=nan_policy.NAN_IS_NULL,

stream=None,

DeviceMemoryResource mr=None,

)

HyperLogLog sketch for approximate distinct counting.

For details, see cudf::approx_distinct_count.

Parameters:

inputTable

Table whose rows will be added to the sketch.

precisionint

The HyperLogLog precision parameter (4-18). Higher precision gives better accuracy but uses more memory. Default is 12.

null_handlingnull_policy

Whether to include or exclude rows with nulls (default: EXCLUDE).

nan_handlingnan_policy

Whether to treat NaNs as null or valid elements (default: NAN_IS_NULL).

streamStream | None

CUDA stream on which to perform the operation.

Methods

add(self, Table input[, stream])	Add rows from a table to the sketch.
estimate(self[, stream])	Estimate the approximate number of distinct rows in the sketch.
merge(self, ApproxDistinctCount other[, stream])	Merge another sketch into this sketch.
nan_handling(self)	Return the NaN handling policy for this sketch.
null_handling(self)	Return the null handling policy for this sketch.
precision(self)	Return the precision parameter for this sketch.
sketch_alignment()	Return the alignment required for sketch storage.
sketch_bytes(int32_t precision)	Return the bytes required for sketch storage at a given precision.
standard_error(self)	Return the standard error (error tolerance) for this sketch.

add(self, Table input, stream=None) → void

Add rows from a table to the sketch.

Parameters:

inputTable

Table whose rows will be added.

streamStream | None

CUDA stream on which to perform the operation.

estimate(self, stream=None) → size_t

Estimate the approximate number of distinct rows in the sketch.

Parameters:

streamStream | None

CUDA stream on which to perform the operation.

Returns:

int

The approximate number of distinct rows.

merge(

self,

ApproxDistinctCount other,

stream=None,

) → void

Merge another sketch into this sketch.

Parameters:

otherApproxDistinctCount

The sketch to merge into this sketch.

streamStream | None

CUDA stream on which to perform the operation.

nan_handling(self) → nan_policy

Return the NaN handling policy for this sketch.

null_handling(self) → null_policy

Return the null handling policy for this sketch.

precision(self) → int32_t

Return the precision parameter for this sketch.

static sketch_alignment()

Return the alignment required for sketch storage.

Returns:

int

The required alignment in bytes.

static sketch_bytes(int32_t precision)

Return the bytes required for sketch storage at a given precision.

Parameters:

precisionint

The HLL precision parameter (4-18).

Returns:

int

The number of bytes required for the sketch.

standard_error(self) → double

Return the standard error (error tolerance) for this sketch.

pylibcudf.reduce.ScanType

See also scan_type.

Enum members

• INCLUSIVE

• EXCLUSIVE

pylibcudf.reduce.distinct_count(

Column source,

null_policy null_handling,

nan_policy nan_handling,

stream: CudaStreamLike | None = None,

) → size_type

Returns the number of distinct elements in the input column.

For details, see cudf::distinct_count().

Parameters:

sourceColumn

The input column to count the unique elements of.

null_handlingnull_policy

Flag to include or exclude nulls from the count. If included, all nulls compare equal.

nan_handlingnan_policy

Whether to treat NaNs as null, or valid elements. If valid all NaNs compare equal.

Returns:

size_type

The number of distinct elements in the input column.

pylibcudf.reduce.is_valid_reduce_aggregation(DataType source, Aggregation agg) → bool

Return if an aggregation is supported for a given datatype.

Parameters:

source

The type of the column the aggregation is being performed on.

agg

The aggregation.

Returns:

True if the aggregation is supported.

pylibcudf.reduce.minmax(

Column col,

stream: CudaStreamLike | None = None,

DeviceMemoryResource mr=None,

) → tuple

Compute the minimum and maximum of a column

For details, see cudf::minmax documentation.

Parameters:

colColumn

The column to compute the minimum and maximum of.

streamStream | None

CUDA stream on which to perform the operation.

mrDeviceMemoryResource | None

Device memory resource used to allocate the returned scalars’ device memory.

Returns:

tuple

A tuple of two Scalars, the first being the minimum and the second being the maximum.

pylibcudf.reduce.reduce(

Column col,

Aggregation agg,

DataType data_type,

Scalar init=None,

stream: CudaStreamLike | None = None,

DeviceMemoryResource mr=None,

) → Scalar

Perform a reduction on a column

For details, see cudf::reduce documentation.

Parameters:

colColumn

The column to perform the reduction on.

aggAggregation

The aggregation to perform.

data_typeDataType

The data type of the result.

initScalar | None

The initial value for the reduction.

streamStream | None

CUDA stream on which to perform the operation.

mrDeviceMemoryResource | None

Device memory resource used to allocate the returned scalar’s device memory.

Returns:

Scalar

The result of the reduction.

pylibcudf.reduce.scan(

Column col,

Aggregation agg,

scan_type inclusive,

stream: CudaStreamLike | None = None,

DeviceMemoryResource mr=None,

) → Column

Perform a scan on a column

For details, see cudf::scan documentation.

Parameters:

colColumn

The column to perform the scan on.

aggAggregation

The aggregation to perform.

inclusivescan_type

The type of scan to perform.

streamStream | None

CUDA stream on which to perform the operation.

mrDeviceMemoryResource | None

Device memory resource used to allocate the returned column’s device memory.

Returns:

Column

The result of the scan.

pylibcudf.reduce.unique_count(

Column source,

null_policy null_handling,

nan_policy nan_handling,

stream: CudaStreamLike | None = None,

) → size_type

Returns the number of unique consecutive elements in the input column.

For details, see cudf::unique_count().

Parameters:

sourceColumn

The input column to count the unique elements of.

null_handlingnull_policy

Flag to include or exclude nulls from the count. If included, all nulls compare equal.

nan_handlingnan_policy

Whether to treat NaNs as null, or valid elements. If valid all NaNs compare equal.

Returns:

size_type

The number of unique consecutive elements in the input column.

Notes

If the input column is sorted, then unique_count can produce the same result as distinct_count, but faster.