C++ API

Generic

namespace nvcomp

class NVCompException : public runtime_error

#include <nvcomp.hpp>

The top-level exception throw by nvcomp C++ methods.

Public Functions

inline NVCompException(nvcompStatus_t err, const std::string &msg)

Create a new NVCompException.

Parameters:

• err – [in] The error associated with the exception.

• msg – [in] The error message.

inline nvcompStatus_t get_error() const

Private Members

nvcompStatus_t m_err

namespace nvcomp

Typedefs

typedef std::function<void*(size_t)> AllocFn_t

typedef std::function<void(void*, size_t)> DeAllocFn_t

Enums

enum class BitstreamKind

Defines how buffer will be compressed in nvcomp Manager.

Values:

enumerator NVCOMP_NATIVE

Each input buffer is chunked according to manager setting and compressed in parallel. Allows computation of checksums. Adds custom header with nvCOMP metadata at the beginning of the compressed data.

enumerator RAW

Compresses input data as is, just using underlying compression algorithm. Does not add header with nvCOMP metadata.

enumerator WITH_UNCOMPRESSED_SIZE

Similar to RAW, but adds custom header with just uncompressed size at the beginning of the compressed data.

enum ChecksumPolicy

Values:

enumerator NoComputeNoVerify

During compression, do not compute checksums. During decompression, do not verify checksums.

enumerator ComputeAndNoVerify

During compression, compute checksums. During decompression, do not attempt to verify checksums.

enumerator NoComputeAndVerifyIfPresent

During compression, do not compute checksums. During decompression, verify checksums if they were included.

enumerator ComputeAndVerifyIfPresent

During compression, compute checksums. During decompression, verify checksums if they were included.

enumerator ComputeAndVerify

During compression, compute checksums. During decompression, verify checksums. A runtime error will be thrown upon configure_decompression if checksums were not included in the compressed buffer.

struct CompressionConfig

#include <nvcompManager.hpp>

Config used to aggregate information about the compression of a particular buffer.

Contains a “PinnedPtrHandle” to an nvcompStatus. After the compression is complete, the user can check the result status which resides in pinned host memory.

Public Functions

CompressionConfig(PinnedPtrPool<nvcompStatus_t> &pool, size_t uncompressed_buffer_size)

Construct the config given an nvcompStatus_t memory pool.

nvcompStatus_t *get_status() const

Get the raw nvcompStatus_t*.

CompressionConfig(CompressionConfig &&other)

CompressionConfig(const CompressionConfig &other)

CompressionConfig &operator=(CompressionConfig &&other)

CompressionConfig &operator=(const CompressionConfig &other)

~CompressionConfig()

Public Members

size_t uncompressed_buffer_size

size_t max_compressed_buffer_size

size_t num_chunks

bool compute_checksums

Private Members

std::shared_ptr<CompressionConfigImpl> impl

struct DecompressionConfig

#include <nvcompManager.hpp>

Config used to aggregate information about a particular decompression.

Contains a “PinnedPtrHandle” to an nvcompStatus. After the decompression is complete, the user can check the result status which resides in pinned host memory.

Public Functions

DecompressionConfig(PinnedPtrPool<nvcompStatus_t> &pool)

Construct the config given an nvcompStatus_t memory pool.

nvcompStatus_t *get_status() const

Get the nvcompStatus_t*.

DecompressionConfig(DecompressionConfig &&other)

DecompressionConfig(const DecompressionConfig &other)

DecompressionConfig &operator=(DecompressionConfig &&other)

DecompressionConfig &operator=(const DecompressionConfig &other)

~DecompressionConfig()

Public Members

size_t decomp_data_size

size_t num_chunks

bool checksums_present

Private Members

std::shared_ptr<DecompressionConfigImpl> impl

struct nvcompManagerBase

#include <nvcompManager.hpp>

Abstract base class that defines the nvCOMP high-level interface.

Subclassed by nvcomp::PimplManager

Public Functions

virtual CompressionConfig configure_compression(const size_t uncomp_buffer_size) = 0

Configure the compression of a single buffer.

This routine computes the size of the required result buffer. The result config also contains the nvcompStatus* that allows error checking.

Parameters:

uncomp_buffer_size – [in] The uncompressed input data size (in bytes).

Returns:

CompressionConfig for the size provided.

virtual std::vector<CompressionConfig> configure_compression(const std::vector<size_t> &uncomp_buffer_sizes) = 0

Configure the compression of a batch of buffers.

This routine computes the size of the required result buffer for each element of the batch. The result config also contains the nvcompStatus* that allows error checking.

Parameters:

uncomp_buffer_sizes – [in] The vector of uncompressed input data sizes (in bytes) for each element of the batch.

Returns:

A vector with CompressionConfig for each of the size provided.

virtual void compress(const uint8_t *uncomp_buffer, uint8_t *comp_buffer, const CompressionConfig &comp_config, size_t *comp_size = nullptr) = 0

Perform compression asynchronously for a single buffer.

Parameters:

• uncomp_buffer – [in] The uncompressed input data. (a pointer to device continuous memory).

• comp_buffer – [out] The location to output the compressed data to. (a pointer to device continuous memory) Size requirement is provided in CompressionConfig.

• comp_config – [in] Generated for the current uncomp_buffer with configure_compression.

• comp_size – [out] The location to output size in bytes after compression. (a pointer to a single size_t variable on device) Optional when bitstream kind is NVCOMP_NATIVE.

virtual void compress(const uint8_t *const *uncomp_buffers, uint8_t *const *comp_buffers, const std::vector<CompressionConfig> &comp_configs, size_t *comp_sizes = nullptr) = 0

Perform compression asynchronously for a batch of buffers. Batch size is inferred from comp_configs size.

Parameters:

• uncomp_buffers – [in] The uncompressed input data. (a pointer to a host array of pointers to device continuous memory)

• comp_buffers – [out] The location to output the compressed data to. (a pointer to a host array of pointers to device continuous memory) Size requirement is provided in CompressionConfig.

• comp_configs – [in] Generated for the current uncomp_buffers with configure_compression.

• comp_sizes – [out] The location to output size in bytes after compression. (a pointer to a device array, with size enough to contain batch_size elements of type size_t) Optional when bitstream kind is NVCOMP_NATIVE.

virtual DecompressionConfig configure_decompression(const uint8_t *comp_buffer, const size_t *comp_size = nullptr) = 0

Configure the decompression for a single buffer using a compressed buffer.

Synchronizes the user stream.

• If bitstream kind is NVCOMP_NATIVE, it will parse the header in comp_buffer.

• If bitstream kind is RAW, it may be required (e.g for LZ4) to parse the whole comp_buffer, which could be significantly slower that other options.

• If bitstream kind is WITH_UNCOMPRESSED_SIZE, it will read the size from the beginning of the comp_buffer.

Parameters:

• comp_buffer – [in] The compressed input data. (a pointer to device continuous memory)

• comp_size – [in] Size of the compressed input data. This is required only for RAW format. (a pointer to device variable with compressed size)

Returns:

DecompressionConfig for the comp_buffer provided.

virtual std::vector<DecompressionConfig> configure_decompression(const uint8_t *const *comp_buffers, size_t batch_size, const size_t *comp_sizes = nullptr) = 0

Configure the decompression for a batch of buffers using a compressed buffer.

Synchronizes the user stream.

• If bitstream kind is NVCOMP_NATIVE, it will parse the header in comp_buffers.

• If bitstream kind is RAW, it may be required (e.g for LZ4) to parse the whole comp_buffers, which could be significantly slower that other options.

• If bitstream kind is WITH_UNCOMPRESSED_SIZE, it will read the size from the beginning of the comp_buffers.

Parameters:

• comp_buffers – [in] The compressed input data. (a pointer to host arrays of pointers to device continuous memory)

• batch_size – [in] The size of the batch.

• comp_sizes – [in] Size of the compressed input data. (a pointer to device array) This is required only for RAW format.

Returns:

A vector of DecompressionConfig for each of the comp_buffer provided.

virtual DecompressionConfig configure_decompression(const CompressionConfig &comp_config) = 0

Configure the decompression for a single buffer using a CompressionConfig object.

Does not synchronize the user stream.

Parameters:

comp_config – [in] The config used to compress a buffer.

Returns:

DecompressionConfig based on compression config provided.

virtual std::vector<DecompressionConfig> configure_decompression(const std::vector<CompressionConfig> &comp_configs) = 0

Configure the decompression for a batch of buffers using a CompressionConfig objects.

Does not synchronize the user stream.

Parameters:

comp_configs – [in] A vector of configs used to compress a batch of buffers.

Returns:

A vector of DecompressionConfig based on compression configs provided.

virtual void decompress(uint8_t *decomp_buffer, const uint8_t *comp_buffer, const DecompressionConfig &decomp_config, size_t *comp_size = nullptr) = 0

Perform decompression asynchronously of a single buffer.

Parameters:

• decomp_buffer – [out] The location to output the decompressed data to. (a pointer to device continuous memory) Size requirement is provided in DecompressionConfig.

• comp_buffer – [in] The compressed input data. (a pointer to device continuous memory)

• decomp_config – [in] Resulted from configure_decompression given this comp_buffer. Contains nvcompStatus* in host/device-accessible memory to allow error checking.

• comp_size – [in] The size of compressed input data passed. (a pointer to a single size_t variable on device) Optional when bitstream kind is NVCOMP_NATIVE.

virtual void decompress(uint8_t *const *decomp_buffers, const uint8_t *const *comp_buffers, const std::vector<DecompressionConfig> &decomp_configs, const size_t *comp_sizes = nullptr) = 0

Perform decompression asynchronously of a batch of buffers.

Parameters:

• decomp_buffers – [out] The location to output the decompressed data to. (a pointer to a host array of pointers to device continuous memory) Size requirement is provided in DecompressionConfig.

• comp_buffers – [in] The compressed input data. (a pointer to a host array of pointers to device continuous memory with uncompressed data)

• decomp_configs – [in] Resulted from configure_decompression given this comp_buffers. Contains nvcompStatus* in host/device-accessible memory to allow error checking.

• comp_sizes – [in] The size of compressed input data passed. (a pointer to a device array, with size enough to contain batch_size elements of type size_t) Optional when bitstream kind is NVCOMP_NATIVE.

virtual void set_scratch_allocators(const AllocFn_t &alloc_fn, const DeAllocFn_t &dealloc_fn) = 0

Allows the user to provide a function for allocating / deallocating memory.

The manager requires scratch memory to perform its operations. By default, it will use internal allocators which make use of cudaMallocAsync / cudaFreeAsync The user can override the allocation functions with this API. The required signatures are void* alloc_fn(size_t alloc_size) and void dealloc_fn(void* buffer, size_t alloc_size)

This API copies the allocation functions. The copied functions must be valid until either 1) deallocate_gpu_mem() is called or 2) the nvcompManager instance is destroyed

If a scratch buffer was previously allocated, it is first deallocated using the prior dealloc_fn (or cudaFreeAsync if one wasn’t previously provided)

Parameters:

• alloc_fn – [in] The host function to use to alloc a new scratch result buffer.

• dealloc_fn – [in] The host function to use to dealloc a scratch result buffer.

virtual size_t get_compressed_output_size(const uint8_t *comp_buffer) = 0

Computes the compressed output size (in bytes) of a given buffer.

Synchronously copies the size of the compressed buffer to a host variable for return.

Can only be used with NVCOMP_NATIVE bitstream kind.

To obtain compressed sizes one can also cudaMemcpy sizes from comp_sizes passed to compress function.

Parameters:

comp_buffer – [in] The compressed input data. (a pointer to device continuous memory)

Returns:

Size of the compressed buffer.

virtual std::vector<size_t> get_compressed_output_size(const uint8_t *const *comp_buffers, size_t batch_size) = 0

Computes the compressed output size (in bytes) of a given batch of buffers.

Synchronously copies the size of the compressed buffer to a host variable for return.

Can only be used with NVCOMP_NATIVE bitstream kind.

To obtain compressed sizes one can also cudaMemcpy sizes from comp_sizes passed to compress function.

Parameters:

comp_buffers – [in] The compressed input data. (a pointer host array of pointers to device continuous memory)

Returns:

A vector with sizes of each compressed buffer in the batch.

virtual void deallocate_gpu_mem() = 0

Frees any internal GPU memory used by the nvCOMP HLIF.

Returns this memory back through the deallocator if one was specified through set_scratch_allocators()

virtual ~nvcompManagerBase() = default

struct PimplManager : public nvcomp::nvcompManagerBase

#include <nvcompManager.hpp>

Interface class between nvcompManagerBase and algorithm specific implementation class.

Subclassed by nvcomp::ANSManager, nvcomp::BitcompManager, nvcomp::CascadedManager, nvcomp::DeflateManager, nvcomp::GdeflateManager, nvcomp::LZ4Manager, nvcomp::SnappyManager, nvcomp::ZstdManager

Public Functions

inline virtual ~PimplManager()

inline PimplManager()

PimplManager(const PimplManager&) = delete

PimplManager &operator=(const PimplManager&) = delete

inline virtual CompressionConfig configure_compression(const size_t uncomp_buffer_size) override

Configure the compression of a single buffer.

This routine computes the size of the required result buffer. The result config also contains the nvcompStatus* that allows error checking.

Parameters:

uncomp_buffer_size – [in] The uncompressed input data size (in bytes).

Returns:

CompressionConfig for the size provided.

inline virtual std::vector<CompressionConfig> configure_compression(const std::vector<size_t> &uncomp_buffer_sizes) override

Configure the compression of a batch of buffers.

This routine computes the size of the required result buffer for each element of the batch. The result config also contains the nvcompStatus* that allows error checking.

Parameters:

uncomp_buffer_sizes – [in] The vector of uncompressed input data sizes (in bytes) for each element of the batch.

Returns:

A vector with CompressionConfig for each of the size provided.

inline virtual void compress(const uint8_t *uncomp_buffer, uint8_t *comp_buffer, const CompressionConfig &comp_config, size_t *comp_size = nullptr) override

Perform compression asynchronously for a single buffer.

Parameters:

• uncomp_buffer – [in] The uncompressed input data. (a pointer to device continuous memory).

• comp_buffer – [out] The location to output the compressed data to. (a pointer to device continuous memory) Size requirement is provided in CompressionConfig.

• comp_config – [in] Generated for the current uncomp_buffer with configure_compression.

• comp_size – [out] The location to output size in bytes after compression. (a pointer to a single size_t variable on device) Optional when bitstream kind is NVCOMP_NATIVE.

inline virtual void compress(const uint8_t *const *uncomp_buffers, uint8_t *const *comp_buffers, const std::vector<CompressionConfig> &comp_configs, size_t *comp_sizes = nullptr) override

Perform compression asynchronously for a batch of buffers. Batch size is inferred from comp_configs size.

Parameters:

• uncomp_buffers – [in] The uncompressed input data. (a pointer to a host array of pointers to device continuous memory)

• comp_buffers – [out] The location to output the compressed data to. (a pointer to a host array of pointers to device continuous memory) Size requirement is provided in CompressionConfig.

• comp_configs – [in] Generated for the current uncomp_buffers with configure_compression.

• comp_sizes – [out] The location to output size in bytes after compression. (a pointer to a device array, with size enough to contain batch_size elements of type size_t) Optional when bitstream kind is NVCOMP_NATIVE.

inline virtual DecompressionConfig configure_decompression(const uint8_t *comp_buffer, const size_t *comp_size = nullptr) override

Configure the decompression for a single buffer using a compressed buffer.

Synchronizes the user stream.

• If bitstream kind is NVCOMP_NATIVE, it will parse the header in comp_buffer.

• If bitstream kind is RAW, it may be required (e.g for LZ4) to parse the whole comp_buffer, which could be significantly slower that other options.

• If bitstream kind is WITH_UNCOMPRESSED_SIZE, it will read the size from the beginning of the comp_buffer.

Parameters:

• comp_buffer – [in] The compressed input data. (a pointer to device continuous memory)

• comp_size – [in] Size of the compressed input data. This is required only for RAW format. (a pointer to device variable with compressed size)

Returns:

DecompressionConfig for the comp_buffer provided.

inline virtual std::vector<DecompressionConfig> configure_decompression(const uint8_t *const *comp_buffers, size_t batch_size, const size_t *comp_sizes = nullptr) override

Configure the decompression for a batch of buffers using a compressed buffer.

Synchronizes the user stream.

• If bitstream kind is NVCOMP_NATIVE, it will parse the header in comp_buffers.

• If bitstream kind is RAW, it may be required (e.g for LZ4) to parse the whole comp_buffers, which could be significantly slower that other options.

• If bitstream kind is WITH_UNCOMPRESSED_SIZE, it will read the size from the beginning of the comp_buffers.

Parameters:

• comp_buffers – [in] The compressed input data. (a pointer to host arrays of pointers to device continuous memory)

• batch_size – [in] The size of the batch.

• comp_sizes – [in] Size of the compressed input data. (a pointer to device array) This is required only for RAW format.

Returns:

A vector of DecompressionConfig for each of the comp_buffer provided.

inline virtual DecompressionConfig configure_decompression(const CompressionConfig &comp_config) override

Configure the decompression for a single buffer using a CompressionConfig object.

Does not synchronize the user stream.

Parameters:

comp_config – [in] The config used to compress a buffer.

Returns:

DecompressionConfig based on compression config provided.

inline virtual std::vector<DecompressionConfig> configure_decompression(const std::vector<CompressionConfig> &comp_configs) override

Configure the decompression for a batch of buffers using a CompressionConfig objects.

Does not synchronize the user stream.

Parameters:

comp_configs – [in] A vector of configs used to compress a batch of buffers.

Returns:

A vector of DecompressionConfig based on compression configs provided.

inline virtual void decompress(uint8_t *decomp_buffer, const uint8_t *comp_buffer, const DecompressionConfig &decomp_config, size_t *comp_size = nullptr) override

Perform decompression asynchronously of a single buffer.

Parameters:

• decomp_buffer – [out] The location to output the decompressed data to. (a pointer to device continuous memory) Size requirement is provided in DecompressionConfig.

• comp_buffer – [in] The compressed input data. (a pointer to device continuous memory)

• decomp_config – [in] Resulted from configure_decompression given this comp_buffer. Contains nvcompStatus* in host/device-accessible memory to allow error checking.

• comp_size – [in] The size of compressed input data passed. (a pointer to a single size_t variable on device) Optional when bitstream kind is NVCOMP_NATIVE.

inline virtual void decompress(uint8_t *const *decomp_buffers, const uint8_t *const *comp_buffers, const std::vector<DecompressionConfig> &decomp_configs, const size_t *comp_sizes = nullptr) override

Perform decompression asynchronously of a batch of buffers.

Parameters:

• decomp_buffers – [out] The location to output the decompressed data to. (a pointer to a host array of pointers to device continuous memory) Size requirement is provided in DecompressionConfig.

• comp_buffers – [in] The compressed input data. (a pointer to a host array of pointers to device continuous memory with uncompressed data)

• decomp_configs – [in] Resulted from configure_decompression given this comp_buffers. Contains nvcompStatus* in host/device-accessible memory to allow error checking.

• comp_sizes – [in] The size of compressed input data passed. (a pointer to a device array, with size enough to contain batch_size elements of type size_t) Optional when bitstream kind is NVCOMP_NATIVE.

inline virtual void set_scratch_allocators(const AllocFn_t &alloc_fn, const DeAllocFn_t &dealloc_fn) override

Allows the user to provide a function for allocating / deallocating memory.

The manager requires scratch memory to perform its operations. By default, it will use internal allocators which make use of cudaMallocAsync / cudaFreeAsync The user can override the allocation functions with this API. The required signatures are void* alloc_fn(size_t alloc_size) and void dealloc_fn(void* buffer, size_t alloc_size)

This API copies the allocation functions. The copied functions must be valid until either 1) deallocate_gpu_mem() is called or 2) the nvcompManager instance is destroyed

If a scratch buffer was previously allocated, it is first deallocated using the prior dealloc_fn (or cudaFreeAsync if one wasn’t previously provided)

Parameters:

• alloc_fn – [in] The host function to use to alloc a new scratch result buffer.

• dealloc_fn – [in] The host function to use to dealloc a scratch result buffer.

inline virtual size_t get_compressed_output_size(const uint8_t *comp_buffer) override

Computes the compressed output size (in bytes) of a given buffer.

Synchronously copies the size of the compressed buffer to a host variable for return.

Can only be used with NVCOMP_NATIVE bitstream kind.

To obtain compressed sizes one can also cudaMemcpy sizes from comp_sizes passed to compress function.

Parameters:

comp_buffer – [in] The compressed input data. (a pointer to device continuous memory)

Returns:

Size of the compressed buffer.

inline virtual std::vector<size_t> get_compressed_output_size(const uint8_t *const *comp_buffers, size_t batch_size) override

Computes the compressed output size (in bytes) of a given batch of buffers.

Synchronously copies the size of the compressed buffer to a host variable for return.

Can only be used with NVCOMP_NATIVE bitstream kind.

To obtain compressed sizes one can also cudaMemcpy sizes from comp_sizes passed to compress function.

Parameters:

comp_buffers – [in] The compressed input data. (a pointer host array of pointers to device continuous memory)

Returns:

A vector with sizes of each compressed buffer in the batch.

inline virtual void deallocate_gpu_mem() override

Frees any internal GPU memory used by the nvCOMP HLIF.

Returns this memory back through the deallocator if one was specified through set_scratch_allocators()

Protected Attributes

std::unique_ptr<nvcompManagerBase> impl

template<typename T>
struct PinnedPtrPool

#include <nvcompManager.hpp>

Internal memory pool used for compression / decompression configs

namespace nvcomp

Functions

std::shared_ptr<nvcompManagerBase> create_manager(const uint8_t *comp_buffer, cudaStream_t stream = 0, ChecksumPolicy checksum_policy = NoComputeNoVerify)

Construct a ManagerBase from a buffer.

This synchronizes the stream

std::shared_ptr<nvcompManagerBase> create_manager(const uint8_t *comp_buffer, cudaStream_t stream, const int device_id, ChecksumPolicy checksum_policy = NoComputeNoVerify)

Construct a ManagerBase from a buffer.

Deprecated:

This signature is deprecated, in favour of the one that does not accept a device_id, and instead gets the device from the stream.

LZ4

namespace nvcomp

struct LZ4FormatSpecHeader

Public Members

nvcompType_t data_type

struct LZ4Manager : public nvcomp::PimplManager

#include <lz4.hpp>

High-level interface class for LZ4 compressor.

Note

Any uncompressed data buffer to be compressed MUST be a size that is a multiple of the data type size, else compression may crash or result in invalid output.

Public Functions

LZ4Manager(size_t uncomp_chunk_size, const nvcompBatchedLZ4Opts_t &format_opts = nvcompBatchedLZ4DefaultOpts, cudaStream_t user_stream = 0, ChecksumPolicy checksum_policy = NoComputeNoVerify, BitstreamKind bitstream_kind = BitstreamKind::NVCOMP_NATIVE)

LZ4Manager(size_t uncomp_chunk_size, const nvcompBatchedLZ4Opts_t &format_opts, cudaStream_t user_stream, const int device_id, ChecksumPolicy checksum_policy = NoComputeNoVerify, BitstreamKind bitstream_kind = BitstreamKind::NVCOMP_NATIVE)

~LZ4Manager()

Snappy

namespace nvcomp

struct SnappyFormatSpecHeader

#include <snappy.hpp>

Format specification for Snappy compression.

struct SnappyManager : public nvcomp::PimplManager

#include <snappy.hpp>

High-level interface class for Snappy compressor.

Public Functions

SnappyManager(size_t uncomp_chunk_size, const nvcompBatchedSnappyOpts_t &format_opts, cudaStream_t user_stream = 0, ChecksumPolicy checksum_policy = NoComputeNoVerify, BitstreamKind bitstream_kind = BitstreamKind::NVCOMP_NATIVE)

SnappyManager(size_t uncomp_chunk_size, const nvcompBatchedSnappyOpts_t &format_opts, cudaStream_t user_stream, int device_id, ChecksumPolicy checksum_policy = NoComputeNoVerify, BitstreamKind bitstream_kind = BitstreamKind::NVCOMP_NATIVE)

~SnappyManager()

Deflate

namespace nvcomp

struct DeflateFormatSpecHeader

#include <deflate.hpp>

Format specification for Deflate compression.

Public Members

int algo

Compression algorithm to use. Permitted values are:

• 1: high-throughput, low compression ratio (default)

• 2: medium-througput, medium compression ratio, beat Zlib level 1 on the compression ratio

• 3: placeholder for further compression level support, will fall into MEDIUM_COMPRESSION at this point

• 4: lower-throughput, higher compression ratio, beat Zlib level 6 on the compression ratio

• 5: lowest-throughput, highest compression ratio

struct DeflateManager : public nvcomp::PimplManager

#include <deflate.hpp>

High-level interface class for Deflate compressor.

Public Functions

DeflateManager(size_t uncomp_chunk_size, const nvcompBatchedDeflateOpts_t &format_opts, cudaStream_t user_stream = 0, ChecksumPolicy checksum_policy = NoComputeNoVerify, BitstreamKind bitstream_kind = BitstreamKind::NVCOMP_NATIVE)

DeflateManager(size_t uncomp_chunk_size, const nvcompBatchedDeflateOpts_t &format_opts, cudaStream_t user_stream, const int device_id, ChecksumPolicy checksum_policy = NoComputeNoVerify, BitstreamKind bitstream_kind = BitstreamKind::NVCOMP_NATIVE)

~DeflateManager()

GDeflate

namespace nvcomp

struct GdeflateFormatSpecHeader

#include <gdeflate.hpp>

Format specification for GDeflate compression.

Public Members

int algo

Compression algorithm to use. Permitted values are:

• 0: highest-throughput, entropy-only compression (use for symmetric compression/decompression performance)

• 1: high-throughput, low compression ratio (default)

• 2: medium-througput, medium compression ratio, beat Zlib level 1 on the compression ratio

• 3: placeholder for further compression level support, will fall into MEDIUM_COMPRESSION at this point

• 4: lower-throughput, higher compression ratio, beat Zlib level 6 on the compression ratio

• 5: lowest-throughput, highest compression ratio

struct GdeflateManager : public nvcomp::PimplManager

#include <gdeflate.hpp>

High-level interface class for GDeflate compressor.

Public Functions

GdeflateManager(size_t uncomp_chunk_size, const nvcompBatchedGdeflateOpts_t &format_opts, cudaStream_t user_stream = 0, ChecksumPolicy checksum_policy = NoComputeNoVerify, BitstreamKind bitstream_kind = BitstreamKind::NVCOMP_NATIVE)

GdeflateManager(size_t uncomp_chunk_size, const nvcompBatchedGdeflateOpts_t &format_opts, cudaStream_t user_stream, const int device_id, ChecksumPolicy checksum_policy = NoComputeNoVerify, BitstreamKind bitstream_kind = BitstreamKind::NVCOMP_NATIVE)

~GdeflateManager()

ZSDT

namespace nvcomp

struct ZstdFormatSpecHeader

#include <zstd.hpp>

Format specification for Zstd compression.

struct ZstdManager : public nvcomp::PimplManager

#include <zstd.hpp>

High-level interface class for the Zstd compressor.

uncomp_chunk_size must be <= nvcompZstdCompressionMaxAllowedChunkSize.

Use 64-128 KB for best performance.

Public Functions

ZstdManager(size_t uncomp_chunk_size, const nvcompBatchedZstdOpts_t &format_opts, cudaStream_t user_stream = 0, ChecksumPolicy checksum_policy = NoComputeNoVerify, BitstreamKind bitstream_kind = BitstreamKind::NVCOMP_NATIVE)

ZstdManager(size_t uncomp_chunk_size, const nvcompBatchedZstdOpts_t &format_opts, cudaStream_t user_stream, const int device_id, ChecksumPolicy checksum_policy = NoComputeNoVerify, BitstreamKind bitstream_kind = BitstreamKind::NVCOMP_NATIVE)

~ZstdManager()

ANS

namespace nvcomp

struct ANSFormatSpecHeader

#include <ans.hpp>

Format specification for ANS compression.

struct ANSManager : public nvcomp::PimplManager

#include <ans.hpp>

High-level interface class for ANS compressor.

Public Functions

ANSManager(size_t uncomp_chunk_size, const nvcompBatchedANSOpts_t &format_opts, cudaStream_t user_stream = 0, ChecksumPolicy checksum_policy = NoComputeNoVerify, BitstreamKind bitstream_kind = BitstreamKind::NVCOMP_NATIVE)

ANSManager(size_t uncomp_chunk_size, const nvcompBatchedANSOpts_t &format_opts, cudaStream_t user_stream, const int device_id, ChecksumPolicy checksum_policy = NoComputeNoVerify, BitstreamKind bitstream_kind = BitstreamKind::NVCOMP_NATIVE)

~ANSManager()

Bitcomp

namespace nvcomp

struct BitcompFormatSpecHeader

#include <bitcomp.hpp>

@bried Format specification for Bitcomp compression

Public Members

int algo

Bitcomp algorithm options,.

• 0 : Default algorithm, usually gives the best compression ratios

• 1 : “Sparse” algorithm, works well on sparse data (with lots of zeroes). and is usually a faster than the default algorithm.

nvcompType_t data_type

One of nvcomp’s possible data types.

struct BitcompManager : public nvcomp::PimplManager

#include <bitcomp.hpp>

High-level interface class for Bitcomp compressor.

Note

Any uncompressed data buffer to be compressed MUST be a size that is a multiple of the data type size, else compression may crash or result in invalid output.

Public Functions

BitcompManager(size_t uncomp_chunk_size, const nvcompBatchedBitcompFormatOpts &format_opts, cudaStream_t user_stream = 0, ChecksumPolicy checksum_policy = NoComputeNoVerify, BitstreamKind bitstream_kind = BitstreamKind::NVCOMP_NATIVE)

BitcompManager(size_t uncomp_chunk_size, const nvcompBatchedBitcompFormatOpts &format_opts, cudaStream_t user_stream, const int device_id, ChecksumPolicy checksum_policy = NoComputeNoVerify, BitstreamKind bitstream_kind = BitstreamKind::NVCOMP_NATIVE)

~BitcompManager()

Cascaded

namespace nvcomp

struct CascadedFormatSpecHeader

#include <cascaded.hpp>

Format specification for Cascased compression.

Public Members

nvcompBatchedCascadedOpts_t options

struct CascadedManager : public nvcomp::PimplManager

#include <cascaded.hpp>

High-level interface class for Cascaded compressor.

Note

Any uncompressed data buffer to be compressed MUST be a size that is a multiple of the data type size, else compression may crash or result in invalid output.

Public Functions

CascadedManager(size_t uncomp_chunk_size, const nvcompBatchedCascadedOpts_t &options = nvcompBatchedCascadedDefaultOpts, cudaStream_t user_stream = 0, ChecksumPolicy checksum_policy = NoComputeNoVerify, BitstreamKind bitstream_kind = BitstreamKind::NVCOMP_NATIVE)

CascadedManager(size_t uncomp_chunk_size, const nvcompBatchedCascadedOpts_t &options, cudaStream_t user_stream, int device_id, ChecksumPolicy checksum_policy = NoComputeNoVerify, BitstreamKind bitstream_kind = BitstreamKind::NVCOMP_NATIVE)

virtual ~CascadedManager()