BatchedZstdCompressor#
- public class BatchedZstdCompressor extends BatchedCompressor#
Multi-buffer ZSTD compressor.
Constructor Summary
BatchedZstdCompressor(long chunkSize, long maxIntermediateBufferSize)Construct a batched ZSTD compressor instance.
Method Summary
Asynchronously compress a batch of buffers.
batchedCompressGetTempSize(long batchSize, long maxChunkSize, long totalSize, long stream)Get the temporary workspace size required to perform compression of an entire batch.
Methods inherited from class
ai.rapids.cudf.nvcomp.BatchedCompressor:compressConstructors
- public BatchedZstdCompressor(
- long chunkSize,
- long maxIntermediateBufferSize
Construct a batched ZSTD compressor instance.
- Parameters:
chunkSize – maximum amount of uncompressed data to compress as a single chunk. Inputs larger than this will be compressed in multiple chunks.
maxIntermediateBufferSize – desired maximum size of intermediate device buffers used during compression.
Protected Methods
- protected long batchedCompressGetTempSize(
- long batchSize,
- long maxChunkSize,
- long totalSize,
- long stream
Get the temporary workspace size required to perform compression of an entire batch.
- Parameters:
batchSize – number of chunks in the batch
maxChunkSize – maximum size of an uncompressed chunk in bytes
totalSize – Upper bound on the total uncompressed size of all chunks
stream – CUDA stream to use
- Returns:
The size of required temporary workspace in bytes to compress the batch.
- protected void batchedCompressAsync(
- long devInPtrs,
- long devInSizes,
- long chunkSize,
- long batchSize,
- long tempPtr,
- long tempSize,
- long devOutPtrs,
- long compressedSizesOutPtr,
- long stream
Asynchronously compress a batch of buffers.
Note that compressedSizesOutPtr must point to pinned memory for this operation to be asynchronous.
- Parameters:
devInPtrs – device address of uncompressed buffer addresses vector
devInSizes – device address of uncompressed buffer sizes vector
chunkSize – maximum size of an uncompressed chunk in bytes
batchSize – number of chunks in the batch
tempPtr – device address of the temporary workspace buffer
tempSize – size of the temporary workspace buffer in bytes
devOutPtrs – device address of output buffer addresses vector
compressedSizesOutPtr – device address where to write the sizes of the compressed data written to the corresponding output buffers. Must point to a buffer with at least 8 bytes of memory per output buffer in the batch.
stream – CUDA stream to use