Python SDKPythonPythonAistoreSdkBatchMultipart

aistore.sdk.batch.multipart.multipart_decoder

View as Markdown

Module Contents

Classes

NameDescription
MultipartDecoderA multipart decoder for parsing multipart HTTP responses.

Data

logger

API

class aistore.sdk.batch.multipart.multipart_decoder.MultipartDecoder(
    encoding: str = UTF_ENCODING,
    parse_as_stream: bool = False,
    chunk_size: int = DEFAULT_CHUNK_SIZE,
    max_buffer_size: int = DEFAULT_MAX_BUFFER_SIZE
)

A multipart decoder for parsing multipart HTTP responses.

Handles boundary extraction, content parsing, and data extraction from individual parts. The decoder supports both Windows (\r\n) and Unix (\n) line endings and gracefully handles malformed content by skipping invalid parts.

Parameters:

encoding
strDefaults to UTF_ENCODING

Character encoding to use. Defaults to “utf-8”.

parse_as_stream
boolDefaults to False

If True, yields part body content as an iterator of chunks that can be directly read from. If False, yields entire content in memory as bytes. Defaults to False. Note that this functionality is in development, not fully tested, and subject to change.

chunk_size
intDefaults to DEFAULT_CHUNK_SIZE

Size of chunks when streaming. Defaults to 8192.

max_buffer_size
intDefaults to DEFAULT_MAX_BUFFER_SIZE

Maximum buffer size for sliding window. Defaults to 64KB.

aistore.sdk.batch.multipart.multipart_decoder.MultipartDecoder._parse_content(
    content: bytes,
    boundary: str
) -> typing.Iterator[typing.Tuple[bytes, bytes]]

Parse multipart content and extract data from each body part (non-streaming).

Parameters:

boundary
str

Boundary identifier used to separate body parts

content
bytes

Raw multipart content to parse

Returns: Iterator[Tuple[bytes, bytes]]

Iterator[Tuple[bytes, bytes]]: Iterator over extracted body part header and data

aistore.sdk.batch.multipart.multipart_decoder.MultipartDecoder._parse_content_streaming(
    response: requests.models.Response,
    boundary: str
) -> typing.Iterator[typing.Tuple[bytes, aistore.sdk.batch.multipart.body_stream_reader.BodyStreamReader]]

Use a streaming parser to start decoding incoming multipart stream. Only extracts headers in memory and streams the body content per part.

aistore.sdk.batch.multipart.multipart_decoder.MultipartDecoder._parse_part(
    part: bytes
) -> typing.Optional[typing.Tuple[bytes, bytes]]

Parse individual multipart body part and extract headers and body.

Parameters:

part
bytes

Raw body part content including headers and body

Returns: Optional[Tuple[bytes, bytes]]

Optional[Tuple[bytes, bytes]]: Extracted headers and body, or None if parsing fails

aistore.sdk.batch.multipart.multipart_decoder.MultipartDecoder.decode(
    response: requests.models.Response
) -> typing.Iterator[typing.Tuple[bytes, typing.Union[bytes, aistore.sdk.batch.multipart.body_stream_reader.BodyStreamReader]]]

Decode contents of a multipart HTTP response.

Parameters:

response
Response

HTTP response object containing multipart data

aistore.sdk.batch.multipart.multipart_decoder.logger = get_logger(__name__)