Compression cutile-python API#

nvmath.tile.compress provides nvCOMPDx-backed compression and decompression for one-dimensional chunks in tiles or global memory Arrays.

The equivalent C++ interface is documented in nvCOMPDx in Tile.

The compressed byte count is known only at runtime, so it cannot determine a tile extent. A tile output has a fixed capacity and a separate meaningful-byte count. Alternatively, pass global memory operands to read and write the chunk and payload directly. See Compression conventions for supported algorithms, element types, and sizing conventions.

Quick start#

This example compresses a 4096-element uint8 tile with ANS and decompresses it in the same kernel. The calls share one scratch buffer, and the host verifies that the restored data matches the input.

import cuda.tile as ct
import cupy as cp
import nvmath.tile.compress as mcomp


@ct.kernel
def compression_round_trip_kernel(source: ct.Array, restored: ct.Array, scratch: ct.Array):
    data = ct.load(source, (0,), shape=(4096,))
    compressed, count = mcomp.compress(data, "ans", scratch=scratch)
    result = mcomp.decompress(compressed, count, "ans", 4096, ct.uint8, scratch)
    ct.store(restored, (0,), result)


def main():
    source = (cp.arange(4096, dtype=cp.uint32) % 23).astype(cp.uint8)
    restored = cp.zeros_like(source)
    # The two calls run one after the other, so they can share one scratch.
    scratch_bytes = max(
        mcomp.compress_scratch_size(4096, "ans", ct.uint8), mcomp.decompress_scratch_size(4096, "ans", ct.uint8)
    )
    scratch = cp.empty((scratch_bytes,), dtype=cp.uint8)
    ct.launch(
        cp.cuda.get_current_stream(),
        (1, 1, 1),
        compression_round_trip_kernel,
        (source, restored, scratch),
    )
    cp.cuda.runtime.deviceSynchronize()
    assert cp.array_equal(source, restored)


if __name__ == "__main__":
    main()

The ans_gmem example processes four float16 chunks. The excerpt below uses comp for nvmath.tile.compress, ct for cuda.tile and np for NumPy. Each block takes contiguous Array slices with aligned bases, writes a payload slot, and stores its compressed byte count. A second kernel loads that count and restores the original number of elements.

ALGORITHM = "ans"
DTYPE, NP_DTYPE = ct.float16, np.float16
CHUNKS, ELEMENTS = 4, 2048
CHUNK_BYTES = ELEMENTS * np.dtype(NP_DTYPE).itemsize
COMPRESS_SCRATCH = comp.compress_scratch_size(CHUNK_BYTES, ALGORITHM, DTYPE)
DECOMPRESS_SCRATCH = comp.decompress_scratch_size(CHUNK_BYTES, ALGORITHM, DTYPE)
# Exact worst case per chunk; slots are padded so each one stays aligned.
CAPACITY = comp.max_compressed_size(CHUNK_BYTES, ALGORITHM)
SLOT = (CAPACITY + comp.buffer_alignment() - 1) // comp.buffer_alignment() * comp.buffer_alignment()


@ct.kernel
def compress_kernel(data: ct.Array, payload: ct.Array, sizes: ct.Array, scratch: ct.Array):
    chunk = ct.bid(0)
    source = data.slice(0, chunk * ELEMENTS, (chunk + 1) * ELEMENTS)
    slot = payload.slice(0, chunk * SLOT, chunk * SLOT + CAPACITY)
    chunk_scratch = scratch.slice(0, chunk * COMPRESS_SCRATCH, (chunk + 1) * COMPRESS_SCRATCH)
    size = comp.compress(source, ALGORITHM, chunk_scratch, out=slot, num_elements=ELEMENTS)
    ct.store(sizes, (chunk,), size)


@ct.kernel
def decompress_kernel(payload: ct.Array, sizes: ct.Array, restored: ct.Array, scratch: ct.Array):
    chunk = ct.bid(0)
    slot = payload.slice(0, chunk * SLOT, chunk * SLOT + CAPACITY)
    target = restored.slice(0, chunk * ELEMENTS, (chunk + 1) * ELEMENTS)
    size = ct.load(sizes, (chunk,), shape=())
    chunk_scratch = scratch.slice(0, chunk * DECOMPRESS_SCRATCH, (chunk + 1) * DECOMPRESS_SCRATCH)
    comp.decompress(slot, size, ALGORITHM, ELEMENTS, DTYPE, chunk_scratch, out=target)


Allocate CHUNKS * SLOT payload bytes, CHUNKS uint64 counts, and CHUNKS slices of each direction’s scratch size. Launch each kernel with (CHUNKS, 1, 1) blocks in the same stream. See Examples for the complete program and the LZ4 variant.

Inputs, outputs and alignment#

Each call’s input and output is independently a tile or a global memory ct.Array:

  • Tile input: pass a one-dimensional tile as x.

  • Array input: pass a one-dimensional Array view of the chunk as x; compression also needs num_elements, a positive compile-time integer.

  • Tile output: compress returns (result, size): a compressed_tile_size()-byte tile, of which only size bytes are meaningful. decompress returns the restored tile.

  • Array output: pass out=, holding max_compressed_size() bytes for compression or uncompressed_size elements for decompression. The call writes it in place and returns a rank-0 uint64 byte count: compressed bytes for compress, restored bytes for decompress. Each call is ordered against the kernel’s other loads and stores, so out can be read back in the same kernel.

Array inputs and outputs must start at a multiple of buffer_alignment() (16 bytes). Tiles have no alignment requirement. When several chunks share one Array, the caller pads every slot so each chunk’s base stays aligned, e.g. a payload stride of max_compressed_size() rounded up to a multiple of buffer_alignment(). The scratch Array must satisfy scratch_alignment(). cudaMalloc bases satisfy both.

Array operands, including scratch, must be contiguous (strides[0] == 1). Only the base pointer reaches nvCOMPDx, so strides are ignored rather than checked: a strided view such as a[::2] is read or written as if it were contiguous, touching elements outside the view.

Behavior is undefined if an Array operand is not contiguous, x holds fewer than num_elements elements for compression or compressed_size bytes for decompression, or out is smaller than stated above. Decompression must use the original algorithm, element type, and element count. Decompressing corrupt input is also undefined behavior.

Scratch memory#

Pass a one-dimensional contiguous uint8 Array as scratch, with at least compress_scratch_size(uncompressed_bytes, algorithm, dtype) or decompress_scratch_size(uncompressed_bytes, algorithm, dtype) bytes for the respective direction. Concurrent calls need disjoint slices, and each base must satisfy scratch_alignment(). Sequential calls can reuse one allocation sized to the larger requirement, as the quickstart does.

API reference#

In-kernel chunk compression and decompression, backed by nvCOMPDx.

Each call handles one one-dimensional chunk. compress() returns a byte tile and its meaningful byte count. The caller supplies private global memory scratch sized by compress_scratch_size() or decompress_scratch_size() and aligned to scratch_alignment(). Decompressing corrupt input is undefined behavior.

The data and the payload can each be a tile or a global memory ct.Array: an Array input is read in place, and an out= Array is written in place instead of returning a tile. Array operands must be contiguous (strides[0] == 1). The API does not check strides. Passing a strided view such as a[::2] is undefined behavior.

The size helpers are ordinary host functions and static CUDA Tile functions, so their results can also be used to construct compile-time shapes.

nvmath.tile.compress.max_compressed_size(uncompressed_bytes, algorithm, dtype=None)#

Return the worst-case compressed size for one chunk.

Parameters:
  • uncompressed_bytes (int) – Uncompressed chunk size in bytes.

  • algorithm (str) – Compression algorithm, either "lz4" or "ans".

  • dtype (cuda.tile.dtype or None) – Optional element dtype used to validate the supported chunk-size range. If omitted, the size must be supported by at least one dtype.

Returns:

Worst-case compressed size in bytes.

Return type:

int

nvmath.tile.compress.compressed_tile_size(uncompressed_bytes, algorithm, dtype=None)#

Return the byte-tile length produced by compress().

This is the power-of-two ceiling of max_compressed_size(). Use it for tile storage; an out= Array needs only max_compressed_size() bytes, with its per-chunk slot stride rounded up to buffer_alignment().

Parameters:
  • uncompressed_bytes (int) – Uncompressed chunk size in bytes.

  • algorithm (str) – Compression algorithm, either "lz4" or "ans".

  • dtype (cuda.tile.dtype or None) – Optional element dtype, with the same validation as max_compressed_size().

Returns:

Compressed tile extent in bytes.

Return type:

int

nvmath.tile.compress.compress_scratch_size(uncompressed_bytes, algorithm, dtype)#

Return the per-call global scratch size required by compression.

Concurrent calls require disjoint slices of at least this size. The result is padded to the selected configuration’s NVCOMPDX_TRAIT_TMP_ALIGNMENT; each slice’s base must also satisfy scratch_alignment().

Parameters:
  • uncompressed_bytes (int) – Uncompressed chunk size in bytes.

  • algorithm (str) – Compression algorithm, either "lz4" or "ans".

  • dtype (cuda.tile.dtype) – Element type of the uncompressed input.

Returns:

Required scratch size in bytes.

Return type:

int

nvmath.tile.compress.decompress_scratch_size(uncompressed_bytes, algorithm, dtype)#

Return the per-call global scratch size required by decompression.

Concurrent calls require disjoint slices of at least this size. The result is padded to the selected configuration’s NVCOMPDX_TRAIT_TMP_ALIGNMENT; each slice’s base must also satisfy scratch_alignment().

Parameters:
  • uncompressed_bytes (int) – Uncompressed chunk size in bytes.

  • algorithm (str) – Compression algorithm, either "lz4" or "ans".

  • dtype (cuda.tile.dtype) – Element type of the restored data.

Returns:

Required scratch size in bytes.

Return type:

int

nvmath.tile.compress.scratch_alignment()#

Return the required base-address alignment for global scratch.

The alignment applies to both directions. Every scratch array must start at an address aligned to this many bytes. This includes slices of a larger allocation.

Returns:

Required scratch alignment in bytes.

Return type:

int

nvmath.tile.compress.buffer_alignment()#

Return the base-address alignment for global memory data and payloads.

The alignment applies to Array inputs and out= Arrays. cudaMalloc bases satisfy it.

Returns:

Required alignment in bytes.

Return type:

int

nvmath.tile.compress.compress(x, algorithm, scratch, out=None, num_elements=None)#

Compress one one-dimensional chunk.

Array operands, including scratch, must be contiguous. Passing a strided view such as a[::2] is undefined behavior.

Parameters:
  • x (cuda.tile.Tile or cuda.tile.Array) – Input tile with shape (N,), or a one-dimensional contiguous global memory Array holding at least num_elements elements (its base must satisfy buffer_alignment()). LZ4 accepts uint8, uint16, and uint32. ANS accepts uint8 and float16.

  • algorithm (str) – Compression algorithm, either "lz4" or "ans".

  • scratch (cuda.tile.Array) – uint8 array spanning this call’s private, contiguous compress_scratch_size(...) bytes of global memory. It must hold at least that many bytes, and its base address must satisfy scratch_alignment().

  • out (cuda.tile.Array or None) – Optional uint8 global memory Array receiving the payload in place. It must be contiguous, hold max_compressed_size(...) bytes and satisfy buffer_alignment(). Pad per-chunk slots of a shared Array to keep every base aligned.

  • num_elements (int or None) – The chunk’s compile-time element count; required when x is an Array.

Returns:

A pair (compressed, compressed_size). compressed is a uint8 tile with shape (compressed_tile_size(...),) and compressed_size is a rank-0 uint64 tile. With out, only compressed_size.

Return type:

tuple[cuda.tile.Tile, cuda.tile.Tile] or cuda.tile.Tile

nvmath.tile.compress.decompress(
x,
compressed_size,
algorithm,
uncompressed_size,
dtype,
scratch,
out=None,
)#

Decompress one chunk into a one-dimensional tile or Array.

Decompressing corrupt input is undefined behavior. Array operands, including scratch, must be contiguous. Passing a strided view such as a[::2] is undefined behavior.

Parameters:
  • x (cuda.tile.Tile or cuda.tile.Array) – Compressed bytes with shape (K,) and uint8 elements, or a one-dimensional contiguous uint8 global memory Array holding them (its base must satisfy buffer_alignment()).

  • compressed_size (cuda.tile.Tile or int) – Number of meaningful bytes in x, either the rank-0 uint64 tile returned by compress() or a scalar value.

  • algorithm (str) – Compression algorithm, either "lz4" or "ans".

  • uncompressed_size (int) – Compile-time number of elements to restore; must equal the element count the payload was compressed from.

  • dtype (cuda.tile.dtype) – Compile-time element type of the restored data.

  • scratch (cuda.tile.Array) – uint8 array spanning this call’s private, contiguous decompress_scratch_size(...) bytes of global memory. It must hold at least that many bytes, and its base address must satisfy scratch_alignment().

  • out (cuda.tile.Array or None) – Optional contiguous dtype global memory Array receiving the uncompressed_size restored elements in place; its base must satisfy buffer_alignment().

Returns:

Restored data with shape (uncompressed_size,) and element type dtype. With out, the rank-0 uint64 restored byte count instead.

Return type:

cuda.tile.Tile