CRC32

This page aims to present the core concepts behind nvCOMP’s CRC32 API. It does not seek to explain the computational principles behind CRC32 calculations. The interested reader is directed to the many excellent resources covering the topic available online.

Terminology

The CRC32 functionality is considered part of nvCOMP’s low-level interface. In analogy with low-level compression and decompression, a contiguous bitstream that gets processed as a whole during a single API call is referred to simply as a chunk.

A closely related concept encountered in discussions of CRC algorithms is that of a message. nvCOMP supports incremental CRC calculation wherein the entire message may not be available at once but only a chunk at a time. In this context, a chunk is referred to as a segment of a message. A message may have a single segment.

Like other low-level-interface functionality, multiple chunks may be processed in parallel. The collection of chunks, segments, or messages processed in parallel is referred as a batch, e.g., a segment batch or a message batch.

The number and sizes of chunks in a segment batch are referred to as the data characteristics of the batch.

In a non-incremental setting, the concepts of chunk and message are interchangeable.

Core Concepts

nvCOMP supports the calculation of generic CRC32 checksums. While “CRC32” colloquially often refers to a particular algorithm, sometimes further qualified as the PKZIP CRC32 algorithm, there is a wide range of CRC32 algorithms which may be parameterized by a CRC32 specification. See nvcompCRC32Spec_t for details of the specification as used in nvCOMP.

Different computational approaches may yield better performance depending on the data characteristics of the processed segment batch. This is tuned through an additional input to the main CRC32 computation function nvcompBatchedCRC32Async(), the kernel configuration.

A heuristic and an exhaustive-search helper function are provided for determining the optimal kernel configuration given the data characteristics of the input segment batch. The heuristic nvcompBatchedCRC32GetHeuristicConf() function is much quicker and should normally be used. The additional processing time of the exhaustive-search nvcompBatchedCRC32SearchConf() function may be worthwhile when a large number of segment bundles with similar data characteristics, such that a single kernel configuration may be used for all of them, is expected.

The kernel configuration, represented by nvcompCRC32KernelConf_t, may be viewed as an opaque type for common workflows. In advanced use cases, the specific parameters comprising the kernel configuration may help explain and optimize a program’s performance characteristics.

Note that a chunk can start at any address and have any size. Unlike with other parts of the low-level-interface, there are no mechanisms for querying alignment requirements.

API Reference

Please refer to the CRC32 section of the C API for detailed API reference documentation.