aistore.sdk.batch.multipart.stateful_streaming_parser

Module Contents

Classes

Name	Description
`StatefulStreamingParser`	A stateful streaming parser for efficiently processing multipart HTTP responses.

Data

logger

API

class aistore.sdk.batch.multipart.stateful_streaming_parser.StatefulStreamingParser(
    content_iter: typing.Iterator[bytes],
    boundary: bytes,
    max_buffer_size: int
)

A stateful streaming parser for efficiently processing multipart HTTP responses.

The parser maintains state across multiple parts of a multipart response, automatically detecting boundaries, extracting headers, and providing streaming access to body content.

Parameters:

content_iter

Iterator[bytes]

Iterator yielding chunks of the HTTP response

boundary

bytes

The multipart boundary marker (including leading dashes)

max_buffer_size

int

Maximum size of the internal sliding window buffer

_stream_buffer

aistore.sdk.batch.multipart.stateful_streaming_parser.StatefulStreamingParser._extract_headers() -> typing.Optional[bytes]

Extract headers using the optimized content reader abstraction.

Returns: Optional[bytes]

Optional[bytes]: Headers as bytes if found, None if stream exhausted

aistore.sdk.batch.multipart.stateful_streaming_parser.StatefulStreamingParser._locate_first_boundary() -> bool

Find the first boundary marker using the optimized buffer reader.

Returns: bool

True if boundary found, False if exhausted without finding boundary

aistore.sdk.batch.multipart.stateful_streaming_parser.StatefulStreamingParser.get_next_part() -> typing.Optional[typing.Tuple[bytes, aistore.sdk.batch.multipart.body_stream_reader.BodyStreamReader]]

Get the next part without loading it completely into memory.

Returns: Optional[Tuple[bytes, BodyStreamReader]]

Optional[Tuple[bytes, BodyStreamReader]]: If part exists, returns tuple of headers and body stream

aistore.sdk.batch.multipart.stateful_streaming_parser.logger = get_logger(__name__)