Compression cutile-python API#
nvmath.tile.compress provides nvCOMPDx-backed compression and
decompression for one-dimensional chunks in tiles or global memory Arrays.
The equivalent C++ interface is documented in nvCOMPDx in Tile.
The compressed byte count is known only at runtime, so it cannot determine a tile extent. A tile output has a fixed capacity and a separate meaningful-byte count. Alternatively, pass global memory operands to read and write the chunk and payload directly. See Compression conventions for supported algorithms, element types, and sizing conventions.
Quick start#
This example compresses a 4096-element uint8 tile with ANS and
decompresses it in the same kernel. The calls share one scratch buffer,
and the host verifies that the restored data matches the input.
import cuda.tile as ct
import cupy as cp
import nvmath.tile.compress as mcomp
@ct.kernel
def compression_round_trip_kernel(source: ct.Array, restored: ct.Array, scratch: ct.Array):
data = ct.load(source, (0,), shape=(4096,))
compressed, count = mcomp.compress(data, "ans", scratch=scratch)
result = mcomp.decompress(compressed, count, "ans", 4096, ct.uint8, scratch)
ct.store(restored, (0,), result)
def main():
source = (cp.arange(4096, dtype=cp.uint32) % 23).astype(cp.uint8)
restored = cp.zeros_like(source)
# The two calls run one after the other, so they can share one scratch.
scratch_bytes = max(
mcomp.compress_scratch_size(4096, "ans", ct.uint8), mcomp.decompress_scratch_size(4096, "ans", ct.uint8)
)
scratch = cp.empty((scratch_bytes,), dtype=cp.uint8)
ct.launch(
cp.cuda.get_current_stream(),
(1, 1, 1),
compression_round_trip_kernel,
(source, restored, scratch),
)
cp.cuda.runtime.deviceSynchronize()
assert cp.array_equal(source, restored)
if __name__ == "__main__":
main()
The ans_gmem example processes four float16 chunks. The excerpt below
uses comp for nvmath.tile.compress, ct for cuda.tile and np
for NumPy. Each block takes contiguous Array slices with aligned bases, writes
a payload slot, and stores its compressed byte count. A second kernel loads
that count and restores the original number of elements.
ALGORITHM = "ans"
DTYPE, NP_DTYPE = ct.float16, np.float16
CHUNKS, ELEMENTS = 4, 2048
CHUNK_BYTES = ELEMENTS * np.dtype(NP_DTYPE).itemsize
COMPRESS_SCRATCH = comp.compress_scratch_size(CHUNK_BYTES, ALGORITHM, DTYPE)
DECOMPRESS_SCRATCH = comp.decompress_scratch_size(CHUNK_BYTES, ALGORITHM, DTYPE)
# Exact worst case per chunk; slots are padded so each one stays aligned.
CAPACITY = comp.max_compressed_size(CHUNK_BYTES, ALGORITHM)
SLOT = (CAPACITY + comp.buffer_alignment() - 1) // comp.buffer_alignment() * comp.buffer_alignment()
@ct.kernel
def compress_kernel(data: ct.Array, payload: ct.Array, sizes: ct.Array, scratch: ct.Array):
chunk = ct.bid(0)
source = data.slice(0, chunk * ELEMENTS, (chunk + 1) * ELEMENTS)
slot = payload.slice(0, chunk * SLOT, chunk * SLOT + CAPACITY)
chunk_scratch = scratch.slice(0, chunk * COMPRESS_SCRATCH, (chunk + 1) * COMPRESS_SCRATCH)
size = comp.compress(source, ALGORITHM, chunk_scratch, out=slot, num_elements=ELEMENTS)
ct.store(sizes, (chunk,), size)
@ct.kernel
def decompress_kernel(payload: ct.Array, sizes: ct.Array, restored: ct.Array, scratch: ct.Array):
chunk = ct.bid(0)
slot = payload.slice(0, chunk * SLOT, chunk * SLOT + CAPACITY)
target = restored.slice(0, chunk * ELEMENTS, (chunk + 1) * ELEMENTS)
size = ct.load(sizes, (chunk,), shape=())
chunk_scratch = scratch.slice(0, chunk * DECOMPRESS_SCRATCH, (chunk + 1) * DECOMPRESS_SCRATCH)
comp.decompress(slot, size, ALGORITHM, ELEMENTS, DTYPE, chunk_scratch, out=target)
Allocate CHUNKS * SLOT payload bytes, CHUNKS uint64 counts, and
CHUNKS slices of each direction’s scratch size. Launch each kernel with
(CHUNKS, 1, 1) blocks in the same stream. See Examples for the complete
program and the LZ4 variant.
Inputs, outputs and alignment#
Each call’s input and output is independently a tile or a global memory
ct.Array:
Tile input: pass a one-dimensional tile as
x.Array input: pass a one-dimensional Array view of the chunk as
x; compression also needsnum_elements, a positive compile-time integer.Tile output:
compressreturns(result, size): acompressed_tile_size()-byte tile, of which onlysizebytes are meaningful.decompressreturns the restored tile.Array output: pass
out=, holdingmax_compressed_size()bytes for compression oruncompressed_sizeelements for decompression. The call writes it in place and returns a rank-0uint64byte count: compressed bytes forcompress, restored bytes fordecompress. Each call is ordered against the kernel’s other loads and stores, sooutcan be read back in the same kernel.
Array inputs and outputs must start at a multiple of
buffer_alignment() (16 bytes). Tiles have no alignment requirement. When
several chunks share one Array, the caller pads every slot so each chunk’s
base stays aligned, e.g. a payload stride of max_compressed_size()
rounded up to a multiple of buffer_alignment(). The scratch Array
must satisfy scratch_alignment(). cudaMalloc bases satisfy both.
Array operands, including scratch, must be contiguous (strides[0] ==
1). Only the base pointer reaches nvCOMPDx, so strides are ignored rather
than checked: a strided view such as a[::2] is read or written as if it
were contiguous, touching elements outside the view.
Behavior is undefined if an Array operand is not contiguous, x holds fewer
than num_elements elements for compression or compressed_size bytes
for decompression, or out is smaller than stated above. Decompression must
use the original algorithm, element type, and element count. Decompressing
corrupt input is also undefined behavior.
Scratch memory#
Pass a one-dimensional contiguous uint8 Array as scratch, with at least
compress_scratch_size(uncompressed_bytes, algorithm, dtype) or
decompress_scratch_size(uncompressed_bytes, algorithm, dtype) bytes for the
respective direction. Concurrent calls need disjoint slices, and each base
must satisfy scratch_alignment(). Sequential calls can reuse one allocation
sized to the larger requirement, as the quickstart does.
API reference#
In-kernel chunk compression and decompression, backed by nvCOMPDx.
Each call handles one one-dimensional chunk. compress() returns a byte
tile and its meaningful byte count. The caller supplies private global memory
scratch sized by compress_scratch_size() or
decompress_scratch_size() and aligned to scratch_alignment().
Decompressing corrupt input is undefined behavior.
The data and the payload can each be a tile or a global memory ct.Array:
an Array input is read in place, and an out= Array is
written in place instead of returning a tile. Array operands must be
contiguous (strides[0] == 1). The API does not check strides.
Passing a strided view such as a[::2] is undefined behavior.
The size helpers are ordinary host functions and static CUDA Tile functions, so their results can also be used to construct compile-time shapes.
- nvmath.tile.compress.max_compressed_size(uncompressed_bytes, algorithm, dtype=None)#
Return the worst-case compressed size for one chunk.
- Parameters:
uncompressed_bytes (int) – Uncompressed chunk size in bytes.
algorithm (str) – Compression algorithm, either
"lz4"or"ans".dtype (cuda.tile.dtype or None) – Optional element dtype used to validate the supported chunk-size range. If omitted, the size must be supported by at least one dtype.
- Returns:
Worst-case compressed size in bytes.
- Return type:
int
- nvmath.tile.compress.compressed_tile_size(uncompressed_bytes, algorithm, dtype=None)#
Return the byte-tile length produced by
compress().This is the power-of-two ceiling of
max_compressed_size(). Use it for tile storage; anout=Array needs onlymax_compressed_size()bytes, with its per-chunk slot stride rounded up tobuffer_alignment().- Parameters:
uncompressed_bytes (int) – Uncompressed chunk size in bytes.
algorithm (str) – Compression algorithm, either
"lz4"or"ans".dtype (cuda.tile.dtype or None) – Optional element dtype, with the same validation as
max_compressed_size().
- Returns:
Compressed tile extent in bytes.
- Return type:
int
- nvmath.tile.compress.compress_scratch_size(uncompressed_bytes, algorithm, dtype)#
Return the per-call global scratch size required by compression.
Concurrent calls require disjoint slices of at least this size. The result is padded to the selected configuration’s
NVCOMPDX_TRAIT_TMP_ALIGNMENT; each slice’s base must also satisfyscratch_alignment().- Parameters:
uncompressed_bytes (int) – Uncompressed chunk size in bytes.
algorithm (str) – Compression algorithm, either
"lz4"or"ans".dtype (cuda.tile.dtype) – Element type of the uncompressed input.
- Returns:
Required scratch size in bytes.
- Return type:
int
- nvmath.tile.compress.decompress_scratch_size(uncompressed_bytes, algorithm, dtype)#
Return the per-call global scratch size required by decompression.
Concurrent calls require disjoint slices of at least this size. The result is padded to the selected configuration’s
NVCOMPDX_TRAIT_TMP_ALIGNMENT; each slice’s base must also satisfyscratch_alignment().- Parameters:
uncompressed_bytes (int) – Uncompressed chunk size in bytes.
algorithm (str) – Compression algorithm, either
"lz4"or"ans".dtype (cuda.tile.dtype) – Element type of the restored data.
- Returns:
Required scratch size in bytes.
- Return type:
int
- nvmath.tile.compress.scratch_alignment()#
Return the required base-address alignment for global scratch.
The alignment applies to both directions. Every scratch array must start at an address aligned to this many bytes. This includes slices of a larger allocation.
- Returns:
Required scratch alignment in bytes.
- Return type:
int
- nvmath.tile.compress.buffer_alignment()#
Return the base-address alignment for global memory data and payloads.
The alignment applies to Array inputs and
out=Arrays. cudaMalloc bases satisfy it.- Returns:
Required alignment in bytes.
- Return type:
int
- nvmath.tile.compress.compress(x, algorithm, scratch, out=None, num_elements=None)#
Compress one one-dimensional chunk.
Array operands, including
scratch, must be contiguous. Passing a strided view such asa[::2]is undefined behavior.- Parameters:
x (cuda.tile.Tile or cuda.tile.Array) – Input tile with shape
(N,), or a one-dimensional contiguous global memory Array holding at leastnum_elementselements (its base must satisfybuffer_alignment()). LZ4 acceptsuint8,uint16, anduint32. ANS acceptsuint8andfloat16.algorithm (str) – Compression algorithm, either
"lz4"or"ans".scratch (cuda.tile.Array) –
uint8array spanning this call’s private, contiguouscompress_scratch_size(...)bytes of global memory. It must hold at least that many bytes, and its base address must satisfyscratch_alignment().out (cuda.tile.Array or None) – Optional
uint8global memory Array receiving the payload in place. It must be contiguous, holdmax_compressed_size(...)bytes and satisfybuffer_alignment(). Pad per-chunk slots of a shared Array to keep every base aligned.num_elements (int or None) – The chunk’s compile-time element count; required when
xis an Array.
- Returns:
A pair
(compressed, compressed_size).compressedis auint8tile with shape(compressed_tile_size(...),)andcompressed_sizeis a rank-0uint64tile. Without, onlycompressed_size.- Return type:
tuple[cuda.tile.Tile, cuda.tile.Tile] or cuda.tile.Tile
- nvmath.tile.compress.decompress(
- x,
- compressed_size,
- algorithm,
- uncompressed_size,
- dtype,
- scratch,
- out=None,
Decompress one chunk into a one-dimensional tile or Array.
Decompressing corrupt input is undefined behavior. Array operands, including
scratch, must be contiguous. Passing a strided view such asa[::2]is undefined behavior.- Parameters:
x (cuda.tile.Tile or cuda.tile.Array) – Compressed bytes with shape
(K,)anduint8elements, or a one-dimensional contiguousuint8global memory Array holding them (its base must satisfybuffer_alignment()).compressed_size (cuda.tile.Tile or int) – Number of meaningful bytes in
x, either the rank-0uint64tile returned bycompress()or a scalar value.algorithm (str) – Compression algorithm, either
"lz4"or"ans".uncompressed_size (int) – Compile-time number of elements to restore; must equal the element count the payload was compressed from.
dtype (cuda.tile.dtype) – Compile-time element type of the restored data.
scratch (cuda.tile.Array) –
uint8array spanning this call’s private, contiguousdecompress_scratch_size(...)bytes of global memory. It must hold at least that many bytes, and its base address must satisfyscratch_alignment().out (cuda.tile.Array or None) – Optional contiguous
dtypeglobal memory Array receiving theuncompressed_sizerestored elements in place; its base must satisfybuffer_alignment().
- Returns:
Restored data with shape
(uncompressed_size,)and element typedtype. Without, the rank-0uint64restored byte count instead.- Return type:
cuda.tile.Tile