C++ API#

This is the C++ API reference for the NVIDIA® nvCOMP library.

Generic#

namespace nvcomp#

class NVCompException#

#include <nvcomp.hpp>

The top-level exception thrown by nvcomp C++ methods.

Public Functions

inline NVCompException(nvcompStatus_t err, const std::string &msg)#

Create a new NVCompException.

Parameters:

err – [in] The error associated with the exception.
msg – [in] The error message.

inline nvcompStatus_t get_error() const noexcept#

Private Members

nvcompStatus_t m_err#

namespace nvcomp

Enums

enum BitstreamKind#

Enumeration that defines how a buffer gets compressed by an nvCOMP manager.

Values:

enumerator NVCOMP_NATIVE#: Each input buffer is chunked according to manager setting and compressed in parallel. Allows computation of checksums. Adds custom header with nvCOMP metadata at the beginning of the compressed data.

enumerator RAW#: Compresses input data as is, just using underlying compression algorithm. Does not add header with nvCOMP metadata.

enumerator WITH_UNCOMPRESSED_SIZE#: Similar to RAW, but adds custom header with just uncompressed size at the beginning of the compressed data.

enum ChecksumPolicy#

Enumeration that defines the checksum policy used by an nvCOMP manager.

Values:

enumerator NoComputeNoVerify#: During compression, do not compute checksums. During decompression, do not verify checksums.

enumerator ComputeAndNoVerify#: During compression, compute checksums. During decompression, do not attempt to verify checksums.

enumerator NoComputeAndVerifyIfPresent#: During compression, do not compute checksums. During decompression, verify checksums if they were included.

enumerator ComputeAndVerifyIfPresent#: During compression, compute checksums. During decompression, verify checksums if they were included.

enumerator ComputeAndVerify#: During compression, compute checksums. During decompression, verify checksums. A runtime error will be thrown upon configure_decompression if checksums were not included in the compressed buffer.

struct CompressionConfig#

#include <nvcompManager.hpp>

Configuration used to aggregate information about the compression of a particular buffer.

Public Functions

CompressionConfig()#: Default constructor that initializes members with default values.

Note

CompressionConfig should be retrieved via the configure_compression member function, and shouldn’t be constructed directly.

CompressionConfig( detail::PinnedPtrPool<nvcompStatus_t> &pool, size_t uncompressed_buffer_size, )#: Construct the CompressionConfig using an nvcompStatus_t memory pool and the uncompressed buffer size.

Note

CompressionConfig should be retrieved via the configure_compression member function, and shouldn’t be constructed directly.

nvcompStatus_t *get_status() const#: Get the status of the overall compression.

Note

The data resides in pinned host memory.

CompressionConfig(CompressionConfig &&other)#

CompressionConfig(const CompressionConfig &other)#

CompressionConfig &operator=(CompressionConfig &&other)#

CompressionConfig &operator=(const CompressionConfig &other)#

~CompressionConfig() noexcept#

Public Members

std::shared_ptr<CompressionConfigImpl> impl#

size_t uncompressed_buffer_size#

size_t max_compressed_buffer_size#

size_t num_chunks#

bool compute_checksums#

struct DecompressionConfig#

#include <nvcompManager.hpp>

Configuration used to aggregate information about a particular decompression.

Public Functions

DecompressionConfig()#: Default constructor that initializes members with default values.

Note

DecompressionConfig should be retrieved via the configure_decompression member function, and shouldn’t be constructed directly.

DecompressionConfig(detail::PinnedPtrPool<nvcompStatus_t> &pool)#: Construct the config given an nvcompStatus_t memory pool.

Note

DecompressionConfig should be retrieved via the configure_decompression member function, and shouldn’t be constructed directly.

nvcompStatus_t *get_status() const#: Get the status of the overall decompression.

Note

The data resides in pinned host memory.

DecompressionConfig(DecompressionConfig &&other)#

DecompressionConfig(const DecompressionConfig &other)#

DecompressionConfig &operator=(DecompressionConfig &&other)#

DecompressionConfig &operator=(const DecompressionConfig &other)#

~DecompressionConfig() noexcept#

Public Members

std::shared_ptr<DecompressionConfigImpl> impl#

size_t decomp_data_size#

size_t num_chunks#

bool checksums_present#

struct nvcompManagerBase#

#include <nvcompManager.hpp>

Abstract base class that defines the nvCOMP high-level interface.

Note

This base class should not be directly constructed. One should rely on the explicit manager constructors (e.g., ANSManager), or the create_manager factory function.

Public Functions

virtual CompressionConfig configure_compression( const size_t uncomp_buffer_size, ) = 0#

Configure the compression of a single buffer.

This routine computes the size of the required result buffer. The result config also contains the nvcompStatus_t* that allows error checking.

Parameters:: uncomp_buffer_size – [in] The uncompressed input data size (in bytes).
Returns:: CompressionConfig for the size provided.

virtual std::vector<CompressionConfig> configure_compression( const std::vector<size_t> &uncomp_buffer_sizes, ) = 0#

Configure the compression of a batch of buffers.

This routine computes the size of the required result buffer for each element of the batch. The result config also contains the nvcompStatus_t* that allows error checking.

Parameters:: uncomp_buffer_sizes – [in] The vector of uncompressed input data sizes (in bytes) for each element of the batch.
Returns:: A vector with CompressionConfig for each of the size provided.

virtual void compress( const uint8_t *uncomp_buffer, uint8_t *comp_buffer, const CompressionConfig &comp_config, size_t *comp_size = nullptr, ) = 0#

Perform compression asynchronously for a single buffer.

Parameters:

uncomp_buffer – [in] The uncompressed input data. (a pointer to device continuous memory).
comp_buffer – [out] The location to output the compressed data to. (a pointer to device continuous memory) Size requirement is provided in CompressionConfig.
comp_config – [in] Generated for the current uncomp_buffer with configure_compression.
comp_size – [out] The location to output size in bytes after compression. (a pointer to a single size_t variable on device) Optional when bitstream kind is NVCOMP_NATIVE.

virtual void compress( const uint8_t *const *uncomp_buffers, uint8_t *const *comp_buffers, const std::vector<CompressionConfig> &comp_configs, size_t *comp_sizes = nullptr, ) = 0#

Perform compression asynchronously for a batch of buffers. Batch size is inferred from comp_configs size.

Parameters:

uncomp_buffers – [in] The uncompressed input data. (a pointer to a host array of pointers to device continuous memory)
comp_buffers – [out] The location to output the compressed data to. (a pointer to a host array of pointers to device continuous memory) Size requirement is provided in CompressionConfig.
comp_configs – [in] Generated for the current uncomp_buffers with configure_compression.
comp_sizes – [out] The location to output size in bytes after compression. (a pointer to a device array, with size enough to contain batch_size elements of type size_t) Optional when bitstream kind is NVCOMP_NATIVE.

virtual DecompressionConfig configure_decompression( const uint8_t *comp_buffer, const size_t *comp_size = nullptr, ) = 0#

Configure the decompression for a single buffer using a compressed buffer.

Synchronizes the user stream.

If bitstream kind is NVCOMP_NATIVE, it will parse the header in comp_buffer.
If bitstream kind is RAW, it may be required (e.g for LZ4) to parse the whole comp_buffer, which could be significantly slower that other options.
If bitstream kind is WITH_UNCOMPRESSED_SIZE, it will read the size from the beginning of the comp_buffer.

Parameters:

comp_buffer – [in] The compressed input data. (a pointer to device continuous memory)
comp_size – [in] Size of the compressed input data. This is required only for RAW format. (a pointer to device variable with compressed size)

Returns:

DecompressionConfig for the comp_buffer provided.

virtual std::vector<DecompressionConfig> configure_decompression( const uint8_t *const *comp_buffers, size_t batch_size, const size_t *comp_sizes = nullptr, ) = 0#

Configure the decompression for a batch of buffers using a compressed buffer.

Synchronizes the user stream.

If bitstream kind is NVCOMP_NATIVE, it will parse the header in comp_buffers.
If bitstream kind is RAW, it may be required (e.g for LZ4) to parse the whole comp_buffers, which could be significantly slower that other options.
If bitstream kind is WITH_UNCOMPRESSED_SIZE, it will read the size from the beginning of the comp_buffers.

Parameters:

comp_buffers – [in] The compressed input data. (a pointer to host arrays of pointers to device continuous memory)
batch_size – [in] The size of the batch.
comp_sizes – [in] Size of the compressed input data. (a pointer to device array) This is required only for RAW format.

Returns:

A vector of DecompressionConfig for each of the comp_buffer provided.

virtual DecompressionConfig configure_decompression( const CompressionConfig &comp_config, ) = 0#

Configure the decompression for a single buffer using a CompressionConfig object.

Does not synchronize the user stream.

Parameters:: comp_config – [in] The config used to compress a buffer.
Returns:: DecompressionConfig based on compression config provided.

virtual std::vector<DecompressionConfig> configure_decompression( const std::vector<CompressionConfig> &comp_configs, ) = 0#

Configure the decompression for a batch of buffers using a CompressionConfig objects.

Does not synchronize the user stream.

Parameters:: comp_configs – [in] A vector of configs used to compress a batch of buffers.
Returns:: A vector of DecompressionConfig based on compression configs provided.

virtual void decompress( uint8_t *decomp_buffer, const uint8_t *comp_buffer, const DecompressionConfig &decomp_config, size_t *comp_size = nullptr, ) = 0#

Perform decompression asynchronously of a single buffer.

Parameters:

decomp_buffer – [out] The location to output the decompressed data to. (a pointer to device continuous memory) Size requirement is provided in DecompressionConfig.
comp_buffer – [in] The compressed input data. (a pointer to device continuous memory)
decomp_config – [in] Resulted from configure_decompression given this comp_buffer. Contains nvcompStatus_t* in host/device-accessible memory to allow error checking.
comp_size – [in] The size of compressed input data passed. (a pointer to a single size_t variable on device) Optional when bitstream kind is NVCOMP_NATIVE.

virtual void decompress( uint8_t *const *decomp_buffers, const uint8_t *const *comp_buffers, const std::vector<DecompressionConfig> &decomp_configs, const size_t *comp_sizes = nullptr, ) = 0#

Perform decompression asynchronously of a batch of buffers.

Parameters:

decomp_buffers – [out] The location to output the decompressed data to. (a pointer to a host array of pointers to device continuous memory) Size requirement is provided in DecompressionConfig.
comp_buffers – [in] The compressed input data. (a pointer to a host array of pointers to device continuous memory with uncompressed data)
decomp_configs – [in] Resulted from configure_decompression given this comp_buffers. Contains nvcompStatus_t* in host/device-accessible memory to allow error checking.
comp_sizes – [in] The size of compressed input data passed. (a pointer to a device array, with size enough to contain batch_size elements of type size_t) Optional when bitstream kind is NVCOMP_NATIVE.

virtual void set_scratch_allocators( const AllocFn_t &alloc_fn, const DeAllocFn_t &dealloc_fn, ) = 0#

Allows the user to provide a function for allocating / deallocating memory.

The manager requires scratch memory to perform its operations. By default, it will use internal allocators which make use of cudaMallocAsync / cudaFreeAsync The user can override the allocation functions with this API. The required signatures are void* alloc_fn(size_t alloc_size) and void dealloc_fn(void* buffer, size_t alloc_size)

This API copies the allocation functions. The copied functions must be valid until either 1) deallocate_gpu_mem() is called or 2) the nvcompManager instance is destroyed

If a scratch buffer was previously allocated, it is first deallocated using the prior dealloc_fn (or cudaFreeAsync if one wasn’t previously provided)

Parameters:

alloc_fn – [in] The host function to use to alloc a new scratch result buffer.
dealloc_fn – [in] The host function to use to dealloc a scratch result buffer.

virtual size_t get_compressed_output_size( const uint8_t *comp_buffer, ) = 0#

Computes the compressed output size (in bytes) of a given buffer.

Synchronously copies the size of the compressed buffer to a host variable for return.

Can only be used with NVCOMP_NATIVE bitstream kind.

To obtain compressed sizes one can also cudaMemcpy sizes from comp_sizes passed to compress function.

Parameters:: comp_buffer – [in] The compressed input data. (a pointer to device continuous memory)
Returns:: Size of the compressed buffer.

virtual std::vector<size_t> get_compressed_output_size( const uint8_t *const *comp_buffers, size_t batch_size, ) = 0#

Computes the compressed output size (in bytes) of a given batch of buffers.

Synchronously copies the size of the compressed buffer to a host variable for return.

Can only be used with NVCOMP_NATIVE bitstream kind.

To obtain compressed sizes one can also cudaMemcpy sizes from comp_sizes passed to compress function.

Parameters:: comp_buffers – [in] The compressed input data. (a pointer host array of pointers to device continuous memory)
Returns:: A vector with sizes of each compressed buffer in the batch.

virtual void deallocate_gpu_mem() = 0#

Frees any internal GPU memory used by the nvCOMP HLIF.

Returns this memory back through the deallocator if one was specified through set_scratch_allocators()

virtual ~nvcompManagerBase() noexcept#: Destructor of the base classs.

namespace nvcomp

Functions

std::shared_ptr<nvcompManagerBase> create_manager( const uint8_t *comp_buffer, cudaStream_t stream = 0, ChecksumPolicy checksum_policy = NoComputeNoVerify, nvcompDecompressBackend_t backend = NVCOMP_DECOMPRESS_BACKEND_DEFAULT, bool use_de_sort = false, )#

Construct a ManagerBase from a given compressed buffer.

Note

This operation synchronizes the host with the stream.

Parameters:

comp_buffer – [in] The HLIF compressed buffer from which we intend to create the manager.
stream – [in] The CUDA stream to perform the operation on.
checksum_policy – [in] The checksum policy to use.
backend – [in] The backend (CUDA / hardware decompress engine) to use.
use_de_sort – [in] Whether to sort before hardware decompression for load balancing (for LZ4, Snappy, Deflate, and Gzip).

Returns:

The constructed manager instance.

LZ4#

namespace nvcomp

struct LZ4FormatSpecHeader#

Public Members

nvcompType_t data_type#: LZ4 data type to use.

struct LZ4Manager#

#include <lz4.hpp>

High-level interface class for the LZ4 compressor.

Note

If user_stream is specified, the lifetime of the LZ4Manager must not extend beyond that of the user_stream.

Warning

Any uncompressed data buffer to be compressed MUST be a size that is a multiple of the data type size, else compression may crash or result in invalid output.

Public Functions

LZ4Manager( size_t uncomp_chunk_size, const nvcompBatchedLZ4CompressOpts_t &compress_opts = nvcompBatchedLZ4CompressDefaultOpts, const nvcompBatchedLZ4DecompressOpts_t &decompress_opts = nvcompBatchedLZ4DecompressDefaultOpts, cudaStream_t user_stream = 0, ChecksumPolicy checksum_policy = NoComputeNoVerify, BitstreamKind bitstream_kind = BitstreamKind::NVCOMP_NATIVE, )#

Constructor of LZ4Manager.

Parameters:

uncomp_chunk_size – [in] Internal chunk size used to partition the input data.
compress_opts – [in] Compression options to use.
decompress_opts – [in] Decompression options to use.
user_stream – [in] The CUDA stream to operate on.
checksum_policy – [in] The checksum policy to use during compression and decompression.
bitstream_kind – [in] Setting to configure how the manager compresses the input.

~LZ4Manager() noexcept#: Destructor of LZ4Manager.

Snappy#

namespace nvcomp

struct SnappyFormatSpecHeader#: #include <snappy.hpp>

Format specification for Snappy compression.

struct SnappyManager#

#include <snappy.hpp>

High-level interface class for the Snappy compressor.

Note

If user_stream is specified, the lifetime of the SnappyManager instance must not extend beyond that of the user_stream.

Public Functions

SnappyManager( size_t uncomp_chunk_size, const nvcompBatchedSnappyCompressOpts_t &compress_opts = nvcompBatchedSnappyCompressDefaultOpts, const nvcompBatchedSnappyDecompressOpts_t &decompress_opts = nvcompBatchedSnappyDecompressDefaultOpts, cudaStream_t user_stream = 0, ChecksumPolicy checksum_policy = NoComputeNoVerify, BitstreamKind bitstream_kind = BitstreamKind::NVCOMP_NATIVE, )#

Constructor of SnappyManager.

Parameters:

uncomp_chunk_size – [in] Internal chunk size used to partition the input data.
compress_opts – [in] Compression options to use.
decompress_opts – [in] Decompression options to use.
user_stream – [in] The CUDA stream to operate on.
checksum_policy – [in] The checksum policy to use during compression and decompression.
bitstream_kind – [in] Setting to configure how the manager compresses the input.

~SnappyManager() noexcept#: Destructor of SnappyManager.

Deflate#

namespace nvcomp

struct DeflateFormatSpecHeader#

#include <deflate.hpp>

Format specification for Deflate compression.

Public Members

int algorithm#

Compression algorithm to use.

0: highest-throughput, entropy-only compression (use for symmetric compression/decompression performance)
1: high-throughput, low compression ratio (default)
2: medium-througput, medium compression ratio, beat Zlib level 1 on the compression ratio
3: placeholder for further compression level support, will fall into MEDIUM_COMPRESSION at this point
4: lower-throughput, higher compression ratio, beat Zlib level 6 on the compression ratio
5: lowest-throughput, highest compression ratio

struct DeflateManager#

#include <deflate.hpp>

High-level interface class for the Deflate compressor.

Note

If user_stream is specified, the lifetime of the DeflateManager instance must not extend beyond that of the user_stream.

Public Functions

DeflateManager( size_t uncomp_chunk_size, const nvcompBatchedDeflateCompressOpts_t &compress_opts = nvcompBatchedDeflateCompressDefaultOpts, const nvcompBatchedDeflateDecompressOpts_t &decompress_opts = nvcompBatchedDeflateDecompressDefaultOpts, cudaStream_t user_stream = 0, ChecksumPolicy checksum_policy = NoComputeNoVerify, BitstreamKind bitstream_kind = BitstreamKind::NVCOMP_NATIVE, )#

Constructor of DeflateManager.

Parameters:

uncomp_chunk_size – [in] Internal chunk size used to partition the input data.
compress_opts – [in] Compression options to use.
decompress_opts – [in] Decompression options to use.
user_stream – [in] The CUDA stream to operate on.
checksum_policy – [in] The checksum policy to use during compression and decompression.
bitstream_kind – [in] Setting to configure how the manager compresses the input.

~DeflateManager() noexcept#: Destructor of DeflateManager.

GDeflate#

namespace nvcomp

struct GdeflateFormatSpecHeader#

#include <gdeflate.hpp>

Format specification for GDeflate compression.

Public Members

int algorithm#

Compression algorithm to use.

0: highest-throughput, entropy-only compression (use for symmetric compression/decompression performance)
1: high-throughput, low compression ratio (default)
2: medium-througput, medium compression ratio, beat Zlib level 1 on the compression ratio
3: placeholder for further compression level support, will fall into MEDIUM_COMPRESSION at this point
4: lower-throughput, higher compression ratio, beat Zlib level 6 on the compression ratio
5: lowest-throughput, highest compression ratio

struct GdeflateManager#

#include <gdeflate.hpp>

High-level interface class for the GDeflate compressor.

Note

If user_stream is specified, the lifetime of the GdeflateManager instance must not extend beyond that of the user_stream.

Public Functions

GdeflateManager( size_t uncomp_chunk_size, const nvcompBatchedGdeflateCompressOpts_t &compress_opts = nvcompBatchedGdeflateCompressDefaultOpts, const nvcompBatchedGdeflateDecompressOpts_t &decompress_opts = nvcompBatchedGdeflateDecompressDefaultOpts, cudaStream_t user_stream = 0, ChecksumPolicy checksum_policy = NoComputeNoVerify, BitstreamKind bitstream_kind = BitstreamKind::NVCOMP_NATIVE, )#

Constructor of GdeflateManager.

Parameters:

uncomp_chunk_size – [in] Internal chunk size used to partition the input data.
compress_opts – [in] Compression options to use.
decompress_opts – [in] Decompression options to use.
user_stream – [in] The CUDA stream to operate on.
checksum_policy – [in] The checksum policy to use during compression and decompression.
bitstream_kind – [in] Setting to configure how the manager compresses the input.

~GdeflateManager() noexcept#: Destructor of GdeflateManager.

ZSTD#

namespace nvcomp

struct ZstdFormatSpecHeader#: #include <zstd.hpp>

Format specification for Zstd compression.

struct ZstdManager#

#include <zstd.hpp>

High-level interface class for the Zstd compressor.

Note

uncomp_chunk_size must be <= nvcompZstdCompressionMaxAllowedChunkSize. Use 64-128 KB for best performance.

If user_stream is specified, the lifetime of the ZstdManager instance must not extend beyond that of the user_stream.

Public Functions

ZstdManager( size_t uncomp_chunk_size, const nvcompBatchedZstdCompressOpts_t &compress_opts = nvcompBatchedZstdCompressDefaultOpts, const nvcompBatchedZstdDecompressOpts_t &decompress_opts = nvcompBatchedZstdDecompressDefaultOpts, cudaStream_t user_stream = 0, ChecksumPolicy checksum_policy = NoComputeNoVerify, BitstreamKind bitstream_kind = BitstreamKind::NVCOMP_NATIVE, )#

Constructor of ZstdManager.

Parameters:

uncomp_chunk_size – [in] Internal chunk size used to partition the input data.
compress_opts – [in] Compression options to use.
decompress_opts – [in] Decompression options to use.
user_stream – [in] The CUDA stream to operate on.
checksum_policy – [in] The checksum policy to use during compression and decompression.
bitstream_kind – [in] Setting to configure how the manager compresses the input.

~ZstdManager() noexcept#: Destructor of ZstdManager.

ANS#

namespace nvcomp

struct ANSFormatSpecHeader#: #include <ans.hpp>

Format specification for ANS compression.

struct ANSManager#

#include <ans.hpp>

High-level interface class for the ANS compressor.

Note

If user_stream is specified, the lifetime of the ANSManager instance must not extend beyond that of the user_stream.

Public Functions

ANSManager( size_t uncomp_chunk_size, const nvcompBatchedANSCompressOpts_t &compress_opts = nvcompBatchedANSCompressDefaultOpts, const nvcompBatchedANSDecompressOpts_t &decompress_opts = nvcompBatchedANSDecompressDefaultOpts, cudaStream_t user_stream = 0, ChecksumPolicy checksum_policy = NoComputeNoVerify, BitstreamKind bitstream_kind = BitstreamKind::NVCOMP_NATIVE, )#

Constructor of ANSManager.

Parameters:

uncomp_chunk_size – [in] Internal chunk size used to partition the input data.
compress_opts – [in] Compression options to use.
decompress_opts – [in] Decompression options to use.
user_stream – [in] The CUDA stream to operate on.
checksum_policy – [in] The checksum policy to use during compression and decompression.
bitstream_kind – [in] Setting to configure how the manager compresses the input.

~ANSManager() noexcept#: Destructor of ANSManager.

Bitcomp#

namespace nvcomp

struct BitcompFormatSpecHeader#

#include <bitcomp.hpp>

Format specification for Bitcomp compression

Public Members

int algorithm#

Bitcomp algorithm options.

0 : Default algorithm, usually gives the best compression ratios
1 : “Sparse” algorithm, works well on sparse data (with lots of zeroes). and is usually a faster than the default algorithm.

nvcompType_t data_type#: One of nvcomp’s possible data types.

struct BitcompManager#

#include <bitcomp.hpp>

High-level interface class for the Bitcomp compressor.

Note

If user_stream is specified, the lifetime of the BitcompManager instance must not extend beyond that of the user_stream.

Warning

Any uncompressed data buffer to be compressed MUST be a size that is a multiple of the data type size, else compression may crash or result in invalid output.

Public Functions

BitcompManager( size_t uncomp_chunk_size, const nvcompBatchedBitcompCompressOpts_t &compress_opts = nvcompBatchedBitcompCompressDefaultOpts, const nvcompBatchedBitcompDecompressOpts_t &decompress_opts = nvcompBatchedBitcompDecompressDefaultOpts, cudaStream_t user_stream = 0, ChecksumPolicy checksum_policy = NoComputeNoVerify, BitstreamKind bitstream_kind = BitstreamKind::NVCOMP_NATIVE, )#

Constructor of BitcompManager.

Parameters:

uncomp_chunk_size – [in] Internal chunk size used to partition the input data.
compress_opts – [in] Compression options to use.
decompress_opts – [in] Decompression options to use.
user_stream – [in] The CUDA stream to operate on.
checksum_policy – [in] The checksum policy to use during compression and decompression.
bitstream_kind – [in] Setting to configure how the manager compresses the input.

~BitcompManager() noexcept#: Destructor of BitcompManager.

Cascaded#

namespace nvcomp

struct CascadedFormatSpecHeader#

#include <cascaded.hpp>

Format specification for Cascased compression.

Public Members

size_t internal_chunk_bytes#

The size of each internal chunk of data to decompress independently with.

Cascaded compression. The value should be in the range of [512, 16384] depending on the datatype of the input and the shared memory size of the GPU being used. This is not the size of chunks passed into the API. Recommended size is 4096.

Note

Not currently used and a default of 4096 is just used.

nvcompType_t type#: The datatype used to define the bit-width for compression.

int num_RLEs#: The number of Run Length Encodings to perform.

int num_deltas#: The number of Delta Encodings to perform.

int use_bp#: Whether or not to bitpack the final layers.

struct CascadedManager#

#include <cascaded.hpp>

High-level interface class for the Cascaded compressor.

Note

If user_stream is specified, the lifetime of the CascadedManager instance must not extend beyond that of the user_stream.

Warning

Any uncompressed data buffer to be compressed MUST be a size that is a multiple of the data type size, else compression may crash or result in invalid output.

Public Functions

CascadedManager( size_t uncomp_chunk_size, const nvcompBatchedCascadedCompressOpts_t &compress_opts = nvcompBatchedCascadedCompressDefaultOpts, const nvcompBatchedCascadedDecompressOpts_t &decompress_opts = nvcompBatchedCascadedDecompressDefaultOpts, cudaStream_t user_stream = 0, ChecksumPolicy checksum_policy = NoComputeNoVerify, BitstreamKind bitstream_kind = BitstreamKind::NVCOMP_NATIVE, )#

Constructor of CascadedManager.

Parameters:

uncomp_chunk_size – [in] Internal chunk size used to partition the input data.
compress_opts – [in] Compression options to use.
decompress_opts – [in] Decompression options to use.
user_stream – [in] The CUDA stream to operate on.
checksum_policy – [in] The checksum policy to use during compression and decompression.
bitstream_kind – [in] Setting to configure how the manager compresses the input.

~CascadedManager() noexcept#: Destructor of CascadedManager.