Is this page helpful?

Column Hash#

group Hashing

Typedefs

using hash_value_type = uint32_t#

Type of hash value.

Functions

std::unique_ptr<column> murmurhash3_x86_32(
table_view const &input,
uint32_t seed = DEFAULT_HASH_SEED,
cuda::stream_ref stream = cudf::get_default_stream(),
rmm::device_async_resource_ref mr = cudf::get_current_device_resource_ref()
)#

Computes the MurmurHash3 32-bit hash value of each row in the given table.

This function hashes each column using the same initial seed, then combines the column hashes into a single uint32 value for each row.

Parameters:

  • input – The table of columns to hash

  • seed – Optional seed value to use for the hash function

  • stream – CUDA stream used for device memory operations and kernel launches

  • mr – Device memory resource used to allocate the returned column’s device memory

Returns:

A column where each row is the hash of a row from the input

std::unique_ptr<column> spark_murmurhash3_x86_32(
table_view const &input,
uint32_t seed = DEFAULT_HASH_SEED,
cuda::stream_ref stream = cudf::get_default_stream(),
rmm::device_async_resource_ref mr = cudf::get_current_device_resource_ref()
)#

Computes the Apache Spark-compatible MurmurHash3 32-bit hash of each row.

This function follows Apache Spark value hashing and row traversal semantics. Each non-null value is hashed using the preceding value hash as its seed. Null values leave the current hash unchanged. Spark-specific handling is applied to strings, narrow integral types, fixed-point values, lists, and structs. Floating-point values follow Spark’s rule that -0.0 hashes as +0.0 and every NaN is canonicalized to a single bit pattern, so callers do not need to normalize their input beforehand. MurmurHash3 packs the bytes left over after the last full four-byte block into a single partial block. Spark instead sign-extends each leftover byte and mixes it as a complete block.

Chrono columns are hashed by their stored count, without unit conversion, so the caller must supply Spark’s units: TIMESTAMP_MICROSECONDS for TimestampType, TIMESTAMP_DAYS for DateType, DURATION_MICROSECONDS for a day-time interval. Non-empty tables with other resolutions are rejected.

LIST columns whose child is a STRUCT are not supported yet, and a non-empty table containing one is rejected.

Parameters:

  • input – The table of columns to hash

  • seed – Optional initial seed value, interpreted as unsigned. Defaults to DEFAULT_HASH_SEED (0), matching other libcudf hash functions and cudf-spark-jni. Pass 42 explicitly to match Spark’s SQL hash(). Negative Spark seeds are passed as unsigned.

  • stream – CUDA stream used for device memory operations and kernel launches

  • mr – Device memory resource used to allocate the returned column device memory

Throws:

cudf::logic_error – If non-empty input contains an unsupported chrono resolution or a LIST column whose child is a STRUCT

Returns:

A non-nullable INT32 column containing one Spark-compatible hash per input row

std::unique_ptr<table> murmurhash3_x64_128(
table_view const &input,
uint64_t seed = DEFAULT_HASH_SEED,
cuda::stream_ref stream = cudf::get_default_stream(),
rmm::device_async_resource_ref mr = cudf::get_current_device_resource_ref()
)#

Computes the MurmurHash3 64-bit hash value of each row in the given table.

This function takes a 64-bit seed value and returns hash values using the MurmurHash3_x64_128 algorithm. The hash produces in two uint64 values per row.

Parameters:

  • input – The table of columns to hash

  • seed – Optional seed value to use for the hash function

  • stream – CUDA stream used for device memory operations and kernel launches

  • mr – Device memory resource used to allocate the returned column’s device memory

Returns:

A table of two UINT64 columns

std::unique_ptr<column> md5(
table_view const &input,
cuda::stream_ref stream = cudf::get_default_stream(),
rmm::device_async_resource_ref mr = cudf::get_current_device_resource_ref()
)#

Computes the MD5 hash value of each row in the given table.

Parameters:

  • input – The table of columns to hash

  • stream – CUDA stream used for device memory operations and kernel launches

  • mr – Device memory resource used to allocate the returned column’s device memory

Returns:

A column where each row is the hash of a row from the input

std::unique_ptr<column> sha1(
table_view const &input,
cuda::stream_ref stream = cudf::get_default_stream(),
rmm::device_async_resource_ref mr = cudf::get_current_device_resource_ref()
)#

Computes the SHA-1 hash value of each row in the given table.

Parameters:

  • input – The table of columns to hash

  • stream – CUDA stream used for device memory operations and kernel launches

  • mr – Device memory resource used to allocate the returned column’s device memory

Returns:

A column where each row is the hash of a row from the input

std::unique_ptr<column> sha224(
table_view const &input,
cuda::stream_ref stream = cudf::get_default_stream(),
rmm::device_async_resource_ref mr = cudf::get_current_device_resource_ref()
)#

Computes the SHA-224 hash value of each row in the given table.

Parameters:

  • input – The table of columns to hash

  • stream – CUDA stream used for device memory operations and kernel launches

  • mr – Device memory resource used to allocate the returned column’s device memory

Returns:

A column where each row is the hash of a row from the input

std::unique_ptr<column> sha256(
table_view const &input,
cuda::stream_ref stream = cudf::get_default_stream(),
rmm::device_async_resource_ref mr = cudf::get_current_device_resource_ref()
)#

Computes the SHA-256 hash value of each row in the given table.

Parameters:

  • input – The table of columns to hash

  • stream – CUDA stream used for device memory operations and kernel launches

  • mr – Device memory resource used to allocate the returned column’s device memory

Returns:

A column where each row is the hash of a row from the input

std::unique_ptr<column> sha384(
table_view const &input,
cuda::stream_ref stream = cudf::get_default_stream(),
rmm::device_async_resource_ref mr = cudf::get_current_device_resource_ref()
)#

Computes the SHA-384 hash value of each row in the given table.

Parameters:

  • input – The table of columns to hash

  • stream – CUDA stream used for device memory operations and kernel launches

  • mr – Device memory resource used to allocate the returned column’s device memory

Returns:

A column where each row is the hash of a row from the input

std::unique_ptr<column> sha512(
table_view const &input,
cuda::stream_ref stream = cudf::get_default_stream(),
rmm::device_async_resource_ref mr = cudf::get_current_device_resource_ref()
)#

Computes the SHA-512 hash value of each row in the given table.

Parameters:

  • input – The table of columns to hash

  • stream – CUDA stream used for device memory operations and kernel launches

  • mr – Device memory resource used to allocate the returned column’s device memory

Returns:

A column where each row is the hash of a row from the input

std::unique_ptr<column> xxhash_32(
table_view const &input,
uint32_t seed = DEFAULT_HASH_SEED,
cuda::stream_ref stream = cudf::get_default_stream(),
rmm::device_async_resource_ref mr = cudf::get_current_device_resource_ref()
)#

Computes the XXHash_32 hash value of each row in the given table.

This function computes the hash of each column using the seed for the first column and the resulting hash as a seed for the next column and so on. The result is a uint32 value for each row.

Parameters:

  • input – The table of columns to hash

  • seed – Optional seed value to use for the hash function

  • stream – CUDA stream used for device memory operations and kernel launches

  • mr – Device memory resource used to allocate the returned column’s device memory

Returns:

A column where each row is the hash of a row from the input

std::unique_ptr<column> xxhash_64(
table_view const &input,
uint64_t seed = DEFAULT_HASH_SEED,
cuda::stream_ref stream = cudf::get_default_stream(),
rmm::device_async_resource_ref mr = cudf::get_current_device_resource_ref()
)#

Computes the XXHash_64 hash value of each row in the given table.

This function takes a 64-bit seed value and returns a column of type UINT64.

Parameters:

  • input – The table of columns to hash

  • seed – Optional seed value to use for the hash function

  • stream – CUDA stream used for device memory operations and kernel launches

  • mr – Device memory resource used to allocate the returned column’s device memory

Returns:

A column where each row is the hash of a row from the input

Variables

static constexpr uint32_t DEFAULT_HASH_SEED = 0#

The default seed value for hash functions.