Python API#
This is the Python API reference for the NVIDIA® nvCOMP library.
Free-Threaded Python Support#
nvCOMP supports free-threaded Python
(Python 3.14t+, built with --disable-gil). The extension module
declares py::mod_gil_not_used(), so it operates correctly without
the GIL. All compression algorithms (LZ4, Snappy, Zstd, Cascaded,
Deflate, GDeflate, ANS, Bitcomp) can be used concurrently from multiple
threads.
Thread-safety guarantees#
Type |
Thread-safety level |
Notes |
|---|---|---|
|
Thread-local |
Each thread must create and use its own |
|
Thread-local |
Each thread must build and use its own config. |
|
Externally synchronized |
An |
|
Externally synchronized |
Same rules as |
Allocators ( |
Internally synchronized |
Allocator registration and allocation calls are protected by an internal mutex. Allocator functions can be set and used from any thread. |
Recommended pattern#
Create a separate Codec instance per thread. Each Codec creates
its own internal CUDA stream by default, so independent instances can
operate fully in parallel without any synchronization:
import threading
import numpy as np
from nvidia import nvcomp
def compress_worker(thread_id):
codec = nvcomp.Codec(algorithm="LZ4")
data = np.random.default_rng(seed=thread_id).integers(0, 256, 4096, dtype=np.uint8)
arr = nvcomp.as_array(data).cuda()
compressed = codec.encode(arr)
decompressed = codec.decode(compressed)
assert bytes(decompressed.cpu()) == data.tobytes()
threads = [threading.Thread(target=compress_worker, args=(i,)) for i in range(4)]
for t in threads:
t.start()
for t in threads:
t.join()
Data Type Association#
nvCOMP type |
Python array-protocol type string |
Type description |
|---|---|---|
|
|
Bit |
|
|
8-bit signed character |
|
|
8-bit unsigned character |
|
|
Little-endian 2-byte signed integer |
|
|
Little-endian 2-byte unsigned integer |
|
|
Little-endian 4-byte signed integer |
|
|
Little-endian 4-byte unsigned integer |
|
|
Little-endian 8-byte signed integer |
|
|
Little-endian 8-byte unsigned integer |
|
|
Little-endian 2-byte float |
|
|
Little-endian 4-byte float |
|
|
Little-endian 8-byte float |
|
|
8-bit float (E4M3). Buffer-protocol export uses the PEP 3118
tag |
BitstreamKind#
- class nvidia.nvcomp.BitstreamKind#
Defines how buffer will be compressed in nvcomp
Members:
NVCOMP_NATIVE : Each input buffer is chunked according to manager setting and compressed in parallel. Allows computation of checksums. Adds custom header with nvCOMP metadata at the beginning of the compressed data.
RAW : Compresses input data as is, just using underlying compression algorithm. Does not add header with nvCOMP metadata.
WITH_UNCOMPRESSED_SIZE : Similar to RAW, but stores the uncompressed size at the start of the buffer. The compressed payload follows at an offset rounded up to 16B alignment.
CudaStream#
- class nvidia.nvcomp.CudaStream#
Wrapper around a CUDA stream. Provides either shared-ownership or view semantics, depending on whether it was constructed through
borrowormake_new, respectively.CudaStreamis the type of stream parameters passed to allocation functions that can be used withset_*_allocator. If the deallocation of such memory needs to access the stream passed to the allocation function, the allocation function should return anExternalMemoryinstance wrapping the newly constructed memory object and theCudaStreamargument. The memory object should, from then on, only be accessed through theExternalMemorywrapper. This ensures that the stream is still alive when the memory is deallocated.It is not envisioned that
CudaStreamwill be used outside allocation functions. Nevertheless,borrowandmake_neware provided for completeness.- static borrow(
- cuda_stream: int,
- device_idx: int = -1,
Create a stream view.
The device index is primarily intended for special CUDA streams (i.e., the default, legacy, and per-thread streams) whose device cannot be inferred from the stream value itself. By default, it is equal to -1, a special value whose meaning depends on whether
streamis special or not. Ifstreamis special, the default value associates the shared stream with the current device. Otherwise, theCudaStreamwill always be associated with the stream’s actual device. In this case, passing adevice_idxthat is neither the default value nor the stream’s actual device will raise an exception.- Parameters:
cuda_stream – The
cudaStream_tto wrap, represented as a Python integer.device_idx – Optional index of the device with which to associate the borrowed stream. See function description for details. Equal to -1 by default.
- property device#
The device index associated with the stream.
- property is_special#
Whether the underlying stream is one of the special streams (default, legacy, or per-thread).
Note that passing a special stream to any CUDA API call will actually pass the current device’s corresponding special stream. It must therefore be ensured that the stream’s associated device, as given by
device, is selected before using the stream. This is currently entirely the user’s responsibility.
- static make_new(
- device_idx: int = -1,
Create a new stream with shared ownership.
- Parameters:
device_idx – Optional index of the device with which to associate the newly created stream. By default equal to -1, a special value that represents the current device.
- property ptr#
The underlying
cudaStream_trepresented as a Python integer.The property name follows the convention of
cupy.Streamand reflects the fact that acudaStream_tis internally a pointer.
Codec#
- class nvidia.nvcomp.Codec#
- __init__(
- self: nvidia.nvcomp.nvcomp_impl.Codec,
- **kwargs,
Initialize codec.
- Parameters:
algorithm – An optional name of the compression algorithm to use. By default it is empty, and the algorithm can be deduced during decoding.
device_id – An optional device id to execute decoding/encoding on. If not specified, the default (current) device will be used.
cuda_stream – An optional cudaStream_t represented as a Python integer. By default an internal CUDA stream is created for the given device id.
uncomp_chunk_size – An optional uncompressed data chunk size. By default it is 65536 bytes (64 KiB). For NVCOMP_NATIVE, this must be a multiple of the format’s required buffer alignment
bitstream_kind – Determines the format of the bitstream that nvCOMP will work on. By default
BitstreamKind::NVCOMP_NATIVEis assumed.execution_policy – Determines whether nvCOMP uses the implementation optimized for latency (
ExecutionPolicy.LATENCY) or for concurrency with other device work (ExecutionPolicy.CONCURRENCY). By defaultExecutionPolicy.CONCURRENCYis assumed.checksum_policy – Defines the strategy for computing and verification of the checksum. By default
NO_COMPUTE_NO_VERIFYis assumed.decompress_backend – Defines the decompression strategy (HW/CUDA-based Decompression). By default
NVCOMP_DECOMPRESS_BACKEND_DEFAULTis assumed, which lets nvCOMP decide the best decompression backend. Used for LZ4, Snappy, Deflate, and Gzip algorithms.use_de_sort –
Determines whether to sort chunks before hardware decompression for better performance. Only used when the backend is the hardware decompression engine (
NVCOMP_DECOMPRESS_BACKEND_HARDWARE).- LZ4 algorithm specific options:
data_type: An optional array-protocol type string for default data type to use. bitshuffle_mode: An optional bitshuffle mode to use. By default
nvcomp.BitshuffleMode.NVCOMP_BITSHUFFLE_NONEis assumed. Permitted values are:nvcomp.BitshuffleMode.NVCOMP_BITSHUFFLE_NONE
nvcomp.BitshuffleMode.NVCOMP_BITSHUFFLE_MSB_FIRST
nvcomp.BitshuffleMode.NVCOMP_BITSHUFFLE_LSB_FIRST
- GDeflate algorithm specific options:
- algorithm_type: Compression algorithm type to use. Permitted values are:
0 : highest-throughput, entropy-only compression (use for symmetric compression/decompression performance)
1 : high-throughput, low compression ratio (default)
2 : medium-throughput, medium compression ratio, beat Zlib level 1 on the compression ratio
3 : placeholder for further compression level support, will fall into
MEDIUM_COMPRESSIONat this point4 : lower-throughput, higher compression ratio, beat Zlib level 6 on the compression ratio
5 : lowest-throughput, highest compression ratio
- Deflate algorithm specific options:
- algorithm_type: Compression algorithm type to use. Permitted values are:
0 : highest-throughput, entropy-only compression (use for symmetric compression/decompression performance)
1 : high-throughput, low compression ratio (default)
2 : medium-throughput, medium compression ratio, beat Zlib level 1 on the compression ratio
3 : placeholder for further compression level support, will fall into
MEDIUM_COMPRESSIONat this point4 : lower-throughput, higher compression ratio, beat Zlib level 6 on the compression ratio
5 : lowest-throughput, highest compression ratio
- Gzip algorithm specific options:
- algorithm_type: Compression algorithm type to use. Permitted values are:
0 : highest-throughput, entropy-only compression (use for symmetric compression/decompression performance)
1 : high-throughput, low compression ratio (default)
2 : medium-throughput, medium compression ratio, beat Zlib level 1 on the compression ratio
3 : placeholder for further compression level support, will fall into
MEDIUM_COMPRESSIONat this point4 : lower-throughput, higher compression ratio, beat Zlib level 6 on the compression ratio
5 : lowest-throughput, highest compression ratio
- Bitcomp algorithm specific options:
- algorithm_type: The type of Bitcomp algorithm used.
0 : Default algorithm, usually gives the best compression ratios
1 : “Sparse” algorithm, works well on sparse data (with lots of zeroes). and is usually a faster than the default algorithm.
- data_type: An optional array-protocol type string for default data type to use.
Lossless compression supports integer types and
<f2,<f4, and<f8. Lossy compression supports<f2,<f4, and<f8.- mode: An optional Bitcomp compression mode. By default
nvcomp.BitcompMode.BITCOMP_LOSSLESSis assumed. Permitted values are: * nvcomp.BitcompMode.BITCOMP_LOSSLESS * nvcomp.BitcompMode.BITCOMP_LOSSY_FP_TO_SIGNED * nvcomp.BitcompMode.BITCOMP_LOSSY_FP_TO_UNSIGNED- delta: The scalar quantization delta required by lossy compression. The maximum
reconstruction error is at most
delta / 2. Bitcomp internally rounds the value down to the nearest power of two. It must remain finite, positive, and normal after conversion to the selected data type. By default it is 0.0 and is ignored in lossless mode.
- ANS algorithm specific options:
- data_type: An optional array-protocol type string for default data type to use. Permitted values are:
|u1: For unsigned 8-bit integer|i1: For signed 8-bit integer. Coded identically to|u1.|f1: For 8-bit FP8 (E4M3).<f2: For 16-bit little-endian float<f4: For 32-bit little-endian float
- states_per_lane: Number of interleaved rANS states per lane. 0 (default) is auto:
two streams for every type. 1 is single stream. 2 is two interleaved streams. The low-level C decompress API treats 0 as “read the bitstream” instead.
- histogram_reduction_log2: 0 (default) builds the frequency model from the whole
chunk. N keeps 1/2^N of each warp’s slice (1 = 1/2, 3 = 1/8, max 8). Useful to speed up compression when full chunks are known to be sampled from a common distribution. Applies to every data type. Chunks below 4096 ANS symbols are always histogrammed exactly, whatever the requested reduction.
- max_sub_chunk_count: Compress: 0 (default) uses 8 sub-chunks. Nonzero must be a
power of two in [4, 64]. Decompress: 0 uses whatever compression used; a nonzero value smaller than the stream’s count fails decompression.
- skip_validate: If true, skip per-chunk header/request checks. Trusted
input only; a malformed chunk then has undefined decode behavior.
- Cascaded algorithm specific options:
- data_type: An optional array-protocol type string for the logical input type.
Asymmetric mode supports all integer
nvcompType_tvalues. Symmetric mode supports signed and unsigned 32- and 64-bit integers.- mode: Cascaded compression mode:
nvcomp.CascadedMode.ASYMMETRICselects useful optional stages independently for each internal chunk, prioritizing compression ratio.nvcomp.CascadedMode.SYMMETRICuses the terminal implementation for more balanced compression and decompression throughput.
The default is
nvcomp.CascadedMode.ASYMMETRIC.- compression_level: Controls how many optional encoding stages asymmetric mode
considers. Values above the current stage capacity select the highest supported level. Symmetric mode currently ignores this option. The default is 3.
- terminal_codecs: Terminal codec mask. Values from
nvcomp.CascadedTerminalCodecmay be OR-combined. OnlyCASCADED_BITPACKis currently supported; it is required and enabled by default.- fine_grained_encodings: Encoding mask considered per internal chunk when
CASCADED_BITPACKis used as the terminal codec. Values fromnvcomp.CascadedFineGrainedEncodingmay be OR-combined (DELTA,RLE, and, in symmetric mode,FOR- frame of reference). When RLE is enabled, only an RLE-before-transform stage is considered. The default isDELTA | RLE.- coarse_grained_encodings: Coarse-grained encoding mask. Only
nvcomp.CascadedCoarseGrainedEncoding.NONEis currently supported.
When Cascaded decompression common options are specified, their mode must match compression. Native nvCOMP bitstreams store the common options in their format metadata so a reconstructed manager can restore them.
- compression_config(*args, **kwargs)#
Overloaded function.
compression_config(self: nvidia.nvcomp.nvcomp_impl.Codec, uncompressed_size: int) -> nvidia.nvcomp.nvcomp_impl.CompressConfig
Build a compression config from the uncompressed buffer size in bytes.
Unlike decompression_config it does not need to synchronize codec’s stream.
- Args:
- uncompressed_size: Uncompressed buffer size in bytes. Must be
greater than zero.
- Returns:
nvcomp.CompressConfig
compression_config(self: nvidia.nvcomp.nvcomp_impl.Codec, uncompressed_sizes: list[int]) -> nvidia.nvcomp.nvcomp_impl.CompressConfig
Build a batch compression config from per-element uncompressed sizes.
Unlike decompression_config it does not need to synchronize codec’s stream.
- Args:
- uncompressed_sizes: List of uncompressed buffer sizes in bytes.
Must be non-empty; each entry must be greater than zero.
- Returns:
nvcomp.CompressConfig
- decode(*args, **kwargs)#
Overloaded function.
decode(self: nvidia.nvcomp.nvcomp_impl.Codec, src: nvidia.nvcomp.nvcomp_impl.Array, data_type: str = ‘’, out: object = None, decompression_config: nvidia.nvcomp.nvcomp_impl.DecompressConfig = None) -> object
Decode (decompress) a single Array.
- Args:
src: Decode source object.
data_type: An optional array-protocol type string for output data type. By default it is equal to
|u1.- out: An optional writable buffer to store decoded data.
If it is a native nvcomp.Array, it will be resized to fit the decompressed output. If it is an externally-allocated buffer (e.g. cupy/numba array), its size is fixed and a ValueError is raised when it is too small to hold the decompressed data.
- decompression_config: An optional config from
codec.decompression_config. There are three cases:Not provided — decode internally calls
configure_decompressiononsrc, forcing a stream synchronization.Provided, built from a buffer via
codec.decompression_config(compressed): the synchronization already happened when the config was built; decode itself does not synchronize.Provided, built from a CompressionConfig via
codec.decompression_config(comp_cfg): no header parse ever happened; this path is fully sync-free at decode time and is the recommended pattern when you compress and decompress in the same process.
The same DecompressConfig can be reused across multiple decode calls operating on different compressed buffers of the same uncompressed shape.
- Returns:
nvcomp.Array
decode(self: nvidia.nvcomp.nvcomp_impl.Codec, srcs: nvidia.nvcomp.nvcomp_impl.BatchArray, data_type: str = ‘’, out: object = None, decompression_config: nvidia.nvcomp.nvcomp_impl.DecompressConfig = None) -> object
Decode (decompress) a batch of Arrays.
- Args:
srcs: nvcomp.BatchArray of compressed buffers.
- data_type: An optional array-protocol type string for output data
type. It only applies when
outis omitted, in which case it is used for every entry of the newly created batch and is what the views returned by indexing it report. A batch passed asoutkeeps the data type it was created with and this argument is ignored.- out: An optional nvcomp.BatchArray to store decoded data. Its
buffers keep their fixed capacity and a ValueError is raised when any of them is too small to hold the decompressed data. When omitted, a new BatchArray will be created to store the output.
- decompression_config: See the single-array overload. The same
three cases apply.
- Returns:
Decoded nvcomp.BatchArray.
- decompression_config(*args, **kwargs)#
Overloaded function.
decompression_config(self: nvidia.nvcomp.nvcomp_impl.Codec, src: nvidia.nvcomp.nvcomp_impl.Array) -> nvidia.nvcomp.nvcomp_impl.DecompressConfig
Build a decompression config by parsing the compressed buffer header.
Warning
This call synchronizes the codec’s CUDA stream to read the compressed buffer’s metadata back to the host.
The returned DecompressConfig can be reused across multiple
decodecalls, as long as the compressed buffers have the same uncompressed shape.- Args:
src: Encoded object
- Returns:
nvcomp.DecompressConfig
decompression_config(self: nvidia.nvcomp.nvcomp_impl.Codec, srcs: nvidia.nvcomp.nvcomp_impl.BatchArray) -> nvidia.nvcomp.nvcomp_impl.DecompressConfig
Build a batch decompression config by parsing compressed buffer headers.
Warning
This call synchronizes the codec’s CUDA stream to read each compressed buffer’s metadata back to the host.
The returned DecompressConfig can be reused across multiple
decodecalls, as long as the compressed buffers have the same uncompressed per-element shape.- Args:
srcs: nvcomp.BatchArray of compressed buffers.
- Returns:
nvcomp.DecompressConfig
decompression_config(self: nvidia.nvcomp.nvcomp_impl.Codec, compression_config: nvidia.nvcomp.nvcomp_impl.CompressConfig) -> nvidia.nvcomp.nvcomp_impl.DecompressConfig
Build a decompression config directly from a compression config.
The resulting DecompressConfig is reusable across any compressed buffer produced by encodes that used the same CompressionConfig on the same Codec object (i.e., same uncompressed shape).
This is the recommended path when you compress and decompress in the same process — combine it with a CompressionConfig built from a size (
codec.compression_config(uncompressed_size)) to get a fully sync-free preconfigured round trip.- Args:
- compression_config: An nvcomp.CompressConfig previously obtained
from codec.compression_config(…).
- Returns:
nvcomp.DecompressConfig
- encode(*args, **kwargs)#
Overloaded function.
encode(self: nvidia.nvcomp.nvcomp_impl.Codec, array: nvidia.nvcomp.nvcomp_impl.Array, out: object = None, compression_config: nvidia.nvcomp.nvcomp_impl.CompressConfig = None) -> object
Encode (compress) a single Array.
- Args:
array: Array to encode.
- out: An optional writable buffer to store encoded data.
If it is a native nvcomp.Array, it will be resized to fit the compressed output. If it is an externally-allocated buffer (e.g. cupy/numba array), its size is fixed and a ValueError is raised when it is too small to hold the compressed data.
- compression_config: An optional config from
codec.compression_config. When provided, encode skips the internalconfigure_compressionstep.
- Returns:
Encoded nvcomp.Array.
encode(self: nvidia.nvcomp.nvcomp_impl.Codec, srcs: nvidia.nvcomp.nvcomp_impl.BatchArray, out: object = None, compression_config: nvidia.nvcomp.nvcomp_impl.CompressConfig = None) -> object
Encode (compress) a batch of Arrays.
As in the single-array overload,
configure_compressiondoes not synchronize the stream, so this call is fully asynchronous regardless of whethercompression_configis provided.- Args:
srcs: nvcomp.BatchArray of buffers to compress.
- out: An optional nvcomp.BatchArray to store encoded data. Its
buffers keep their fixed capacity and a ValueError is raised when any of them is too small to hold the compressed data. When omitted, a new BatchArray will be created to store the output.
- compression_config: An optional config from
codec.compression_config. When provided, encode skips the internal configure step.
- Returns:
Encoded nvcomp.BatchArray.
- get_max_comp_buffer_size(
- self: nvidia.nvcomp.nvcomp_impl.Codec,
- source: nvidia.nvcomp.nvcomp_impl.Array,
Retrieves the maximum compressed buffer size for an uncompressed array.
Returns an upper bound on the number of bytes the codec may emit when compressing an input of the given size. Use this to pre-allocate an output buffer that is guaranteed to fit the compressed result. The exact compressed size is typically smaller and is only known after encoding completes.
- Parameters:
source – Input Array object.
- Returns:
Upper bound in bytes for the compressed output.
- get_required_compression_alignment( ) int#
Retrieves the selected format’s required compression alignment.
The returned value is the most restrictive minimum alignment, in bytes, across compression input and output buffers. Use it when laying out buffers that will be passed to
encode.- Returns:
Required compression alignment in bytes.
If this Codec was created without an algorithm, returns the conservative maximum alignment of 16 bytes.
- get_required_decompression_alignment( ) int#
Retrieves the selected format’s required decompression alignment.
The returned value is the most restrictive minimum alignment, in bytes, across decompression input and output buffers. Use it when laying out buffers that will be passed to
decode.- Returns:
Required decompression alignment in bytes.
If this Codec was created without an algorithm, returns the conservative maximum alignment of 16 bytes.
- get_uncomp_buffer_size(
- self: nvidia.nvcomp.nvcomp_impl.Codec,
- source: nvidia.nvcomp.nvcomp_impl.Array,
Retrieves the uncompressed buffer size from a compressed nvCOMP array.
The size is read from metadata embedded in the compressed bitstream and depends on the codec’s bitstream kind:
NVCOMP_NATIVE: read from the nvCOMP common header.WITH_UNCOMPRESSED_SIZE: read from the size prefix at the start of the buffer.RAW: derived by calling each format’s native size-query path.Bitcomp, Zstd, Snappy, ANS and Cascaded recover the size from a dedicated field in the bitstream header, effectively a constant-time lookup.
LZ4, Deflate and GDeflate carry no explicit decompressed-size field, so the function walks the compressed stream (a “decompress-without-output” pass) to compute it. This is the most expensive of the three bitstream-kind paths.
Both host and device buffers are accepted; for device buffers this call synchronizes the codec’s CUDA stream before returning.
- Parameters:
source – Input Array object. It must contain nvCOMP compressed data produced with a matching algorithm and bitstream kind.
- Returns:
Size in bytes of the uncompressed payload.
ArrayBufferKind#
- class nvidia.nvcomp.ArrayBufferKind#
Defines buffer kind in which array data is stored.
Members:
STRIDED_DEVICE : GPU-accessible in pitch-linear layout.
STRIDED_HOST : Host-accessible in pitch-linear layout.
Array#
- class nvidia.nvcomp.Array#
Class which wraps array. It can be decoded data or data to encode.
- property __cuda_array_interface__#
The CUDA array interchange interface compatible with Numba v0.39.0 or later (see CUDA Array Interface for details)
- __dlpack__(
- self: nvidia.nvcomp.nvcomp_impl.Array,
- stream: object = None,
Export the array as a DLPack tensor
- __dlpack_device__( ) tuple#
Get the device associated with the buffer
- property buffer_kind#
Buffer kind in which array data is stored.
- property buffer_size#
The total number of bytes to store the array.
- property capacity#
Amount of memory allocated for array.
- cpu(self: nvidia.nvcomp.nvcomp_impl.Array) object#
Returns a copy of this array in CPU memory. If this array is already in CPU memory, than no copy is performed and the original object is returned.
- Returns:
Array object with content in CPU memory or None if copy could not be done.
- cuda(
- self: nvidia.nvcomp.nvcomp_impl.Array,
- synchronize: bool = True,
- cuda_stream: int = 0,
Returns a copy of this array in device memory. If this array is already in device memory, than no copy is performed and the original object is returned.
- Parameters:
synchronize – If True (by default) it blocks and waits for copy from host to device to be finished, else not synchronization is executed and further synchronization needs to be done using cuda stream provided by e.g. __cuda_array_interface__.
cuda_stream – An optional cudaStream_t represented as a Python integer to copy host buffer to.
- Returns:
Array object with content in device memory or None if copy could not be done.
- property dtype#
- property item_size#
Size of each element in bytes.
- property ndim#
- property precision#
Maximum number of significant bits in data type. Value 0 means that precision is equal to data type bit depth
- property shape#
- property size#
Number of elements this array holds.
- property strides#
Strides of axes in bytes
- to_dlpack(
- self: nvidia.nvcomp.nvcomp_impl.Array,
- cuda_stream: object = None,
Export the array with zero-copy conversion to a DLPack tensor.
- Parameters:
cuda_stream – An optional cudaStream_t represented as a Python integer, upon which synchronization must take place in created Array.
- Returns:
DLPack tensor which is encapsulated in a PyCapsule object.
BatchArray#
- class nvidia.nvcomp.BatchArray#
A batch of buffers held as a structure of arrays, on host and/or device.
Supports
len(), indexing and iteration.- __getitem__(
- self: nvidia.nvcomp.nvcomp_impl.BatchArray,
- index: int,
Return the array at the given index as a view.
Waits for the work that produced this batch, so the first index of an encode output is where compression is awaited and the compressed sizes are read back from the device.
The returned nvcomp.Array does not own its memory, it points into the batch, so it must not outlive it.
The view is always one-dimensional, with the element type the entry was created with: the source’s dtype for
nvcomp.as_arrays, thedata_typepassed todecode, and bytes for compressed output.- Parameters:
index – Index of the array, negative values count from the end.
- Returns:
nvcomp.Array viewing the batch’s memory.
- __len__(self: nvidia.nvcomp.nvcomp_impl.BatchArray) int#
Number of arrays in the batch.
- property buffer_sizes#
Sizes in bytes of every entry, as a list.
Waits for the work that produced this batch, exactly like indexing it.
- cpu(self: nvidia.nvcomp.nvcomp_impl.BatchArray) object#
Returns a copy of this batch with every entry in CPU memory. If every entry is already in CPU memory, no copy is performed and the original object is returned.
Waits for the work that produced this batch, exactly like indexing it.
- Returns:
BatchArray with content in CPU memory.
- cuda(
- self: nvidia.nvcomp.nvcomp_impl.BatchArray,
- synchronize: bool = True,
- cuda_stream: int = 0,
Returns a copy of this batch with every entry in device memory. If every entry is already in device memory, no copy is performed and the original object is returned.
- Parameters:
synchronize – If True (by default) it blocks and waits for the host-to-device copy to finish, else no synchronization is done and further synchronization needs to be done using the given cuda_stream.
cuda_stream – An optional cudaStream_t represented as a Python integer, to copy host buffers to.
- Returns:
BatchArray with content in device memory.
as_array#
- nvidia.nvcomp.as_array(
- source: object,
- cuda_stream: int = 0,
Wraps an external buffer as an array and keeps the source alive as long as the array exists.
- Parameters:
source – Input DLPack tensor which is encapsulated in a PyCapsule object or other object with __cuda_array_interface__, __array_interface__ or __dlpack__ and __dlpack_device__ methods.
cuda_stream – An optional cudaStream_t represented as a Python integer, upon which synchronization must take place in the created Array.
- Returns:
nvcomp.Array
as_arrays#
- nvidia.nvcomp.as_arrays(*args, **kwargs)#
Overloaded function.
as_arrays(buffer: object, offsets: list[int], sizes: list[int], cuda_stream: int = 0) -> nvidia.nvcomp.nvcomp_impl.BatchArray
Wraps one contiguous buffer as a batch, carved by an offset table.
Offsets and sizes are independent, so entries may sit at a fixed stride while each carries its own used length. Entries take their element type from the buffer itself.
Use
Codec.get_required_compression_alignmentorCodec.get_required_decompression_alignmentto ensure each resulting buffer address (the base address plus its offset) has the appropriate alignment. If in doubt, use 16B alignment.- Args:
- buffer: A single object exposing __cuda_array_interface__,
__array_interface__ or __dlpack__.
offsets: Byte offset of each entry into buffer.
sizes: Byte size of each entry.
- cuda_stream: An optional cudaStream_t represented as a Python integer, upon which
synchronization must take place in the created array, exactly as in as_array.
- Returns:
Instance of nvcomp.BatchArray
as_arrays(sources: list[object], cuda_stream: int = 0) -> nvidia.nvcomp.nvcomp_impl.BatchArray
Wraps external buffers as a batch and keeps the sources alive as long as the batch exists.
- Args:
- sources: List of input DLPack tensors which is encapsulated in a PyCapsule objects or other objects
with __cuda_array_interface__, __array_interface__ or __dlpack__ and __dlpack_device__ methods.
- cuda_stream: An optional cudaStream_t represented as a Python integer, upon which synchronization
must take place in the created arrays, exactly as in as_array.
- Returns:
Instance of nvcomp.BatchArray
from_dlpack#
- nvidia.nvcomp.from_dlpack(
- source: object,
- cuda_stream: int = 0,
Zero-copy conversion from a DLPack tensor to a array.
- Parameters:
source – Input DLPack tensor which is encapsulated in a PyCapsule object or other (array) object with __dlpack__ and __dlpack_device__ methods.
cuda_stream – An optional cudaStream_t represented as a Python integer, upon which synchronization must take place in created Array.
- Returns:
nvcomp.Array
set_device_allocator#
- nvidia.nvcomp.set_device_allocator(allocator: object = None) None#
Sets a new allocator to be used for future device allocations.
The signature of the allocator should be like in the following example:
def my_allocator(nbytes: int, stream: nvcomp.Stream) -> PtrProtocol: return MyBuffer(nbytes, stream)
PtrProtocoldenotes any object that has aptrattribute of integral type. This should be the pointer to the allocated buffer (represented as an integer).In the signature,
nbytesis the number of bytes in the requested buffer.streamis the CUDA stream on which to perform the allocation and/or deallocation if the allocator is stream-ordered. Non-stream-ordered allocators may ignorestreamor may synchronize with it before deallocation, depending on the desired behavior. A separate allocation and deallocation stream are currently not supported.The returned object should be such that, when it is deleted, either on account of there being no more valid Python references to it or because it was garbage collected, the memory gets deallocated. In a custom Python class, this may be achieved through the
__del__method. This is considered an advanced usage pattern, so the recommended approach is to compose pre-existing solutions from other libraries, such as cupy’sMemoryclasses and rmm’sDeviceBuffer.It is generally allowed to set a new allocator while one or more buffers allocated by the previous allocator are still active. Individual allocator implementations may, however, choose to prohibit this.
If the deallocation requires accessing
stream, the allocator should return anExternalMemoryinstance wrapping the newly constructed memory object and theCudaStreamargument. The memory object should, from then on, only be accessed through theExternalMemorywrapper. This ensures that the stream is still alive when the memory is deallocated.The allocated memory must be device-accessible.
- Parameters:
allocator – Callable satisfying the conditions above.
set_pinned_allocator#
- nvidia.nvcomp.set_pinned_allocator(allocator: object = None) None#
Sets a new allocator to be used for future pinned host allocations.
Note that his should allocate pinned host memory. For non-pinned host memory, use
set_host_allocator. It is not an error to allocate non-pinned host memory with this allocator but may lead to performance degradation.The allocator must allocate host accessible memory. Other than that, the conditions on
allocatorare the same as inset_device_allocator, including stream semantics.- Parameters:
allocator – Callable satisfying the conditions above.
set_host_allocator#
- nvidia.nvcomp.set_host_allocator(allocator: object = None) None#
Sets a new allocator to be used for future non-pinned host allocations.
This is primarily intended for potentially large allocations, such as those backing CPU Array instances. Moderately-sized internal host allocations may still use System Allocated Memory.
This should allocate non-pinned host memory. For pinned host memory, use
set_pinned_allocator. It is not an error to allocate pinned host memory with this allocator but may lead to performance degradation.The allocator must allocate host accessible memory. Other than that, the conditions on
allocatorare the same as inset_device_allocator, including stream semantics.- Parameters:
allocator – Callable satisfying the conditions above.