aistore.sdk.obj.content_iterator.buffer

Module Contents

Classes

Name	Description
`ParallelBuffer`	Result of a parallel object download, backed by shared memory.
`RingBuffer`	Shared-memory ring buffer with per-slot ready Events.

API

class aistore.sdk.obj.content_iterator.buffer.ParallelBuffer(
    shm: multiprocessing.shared_memory.SharedMemory,
    size: int
)

Result of a parallel object download, backed by shared memory.

Parallel download splits a single object into byte ranges and fetches them concurrently using multiple workers. Workers write directly into a SharedMemory segment; the caller receives a memoryview with no copy.

The caller must call close() (or use as a context manager) when done.

_buf

memoryview = shm.buf[:size]

buf

memoryview

Memoryview of the downloaded content (no extra copy on return).

name

str

Name of the underlying shared memory segment.

aistore.sdk.obj.content_iterator.buffer.ParallelBuffer.__enter__() -> aistore.sdk.obj.content_iterator.buffer.ParallelBufferaistore.sdk.obj.content_iterator.buffer.ParallelBuffer.__enter__() -> aistore.sdk.obj.content_iterator.buffer.ParallelBuffer

aistore.sdk.obj.content_iterator.buffer.ParallelBuffer.__exit__(
    _: aistore.sdk.obj.object = ()
) -> None

aistore.sdk.obj.content_iterator.buffer.ParallelBuffer.__len__() -> int

aistore.sdk.obj.content_iterator.buffer.ParallelBuffer.close() -> None

Release and unlink the underlying shared memory segment.

Safe to call multiple times; the second call is a no-op.

aistore.sdk.obj.content_iterator.buffer.ParallelBuffer.tobytes() -> bytes

Copy content into a new bytes object (extra copy).

class aistore.sdk.obj.content_iterator.buffer.RingBuffer(
    num_slots: int,
    slot_size: int
)

Bases: ParallelBuffer

Shared-memory ring buffer with per-slot ready Events.

Extends ParallelBuffer: inherits shm allocation, buf, name, and close(). Adds slot indexing and Event-based synchronization for the streaming (create_iter) path.

Allocates num_slots x slot_size bytes in /dev/shm. Each slot has a dedicated mp.Event that a worker sets when its data is ready.

slot_ready

List[Event] = [(mp.Event()) for _ in (range(num_slots))]

aistore.sdk.obj.content_iterator.buffer.RingBuffer.read_slot(
    slot: int,
    data_len: int
) -> bytes

Copy data_len bytes out of the slot into a new bytes object.

aistore.sdk.obj.content_iterator.buffer.RingBuffer.wait_slot(
    slot: int
) -> None

Block until the slot is ready, then reset it for reuse.