aistore.sdk.batch.extractor.tar_stream_extractor

Module Contents

Classes

Name	Description
`TarStreamExtractor`	Class for extracting batch .tar, .tar.gz, and .tgz streams from AIStore.

Data

logger

API

class aistore.sdk.batch.extractor.tar_stream_extractor.TarStreamExtractor()

Bases: ArchiveStreamExtractor

Class for extracting batch .tar, .tar.gz, and .tgz streams from AIStore.

Integrates with Batch API (MossReq/MossResp) to provide proper metadata mapping.

_supported_fmts

= (EXT_TARGZ, EXT_TGZ, EXT_TAR)

aistore.sdk.batch.extractor.tar_stream_extractor.TarStreamExtractor.extract(
    response: requests.Response,
    data_stream: typing.Union[io.BytesIO, typing.Any],
    moss_req: aistore.sdk.batch.types.MossReq,
    moss_resp: typing.Optional[aistore.sdk.batch.types.MossResp] = None
) -> typing.Generator[typing.Tuple[aistore.sdk.batch.types.MossOut, bytes], None, None]

Extract from tar archive stream.

Parameters:

response

Response

HTTP response

data_stream

Union[BytesIO, Any]

Archive data

moss_req

MossReq

Original request

moss_resp

Optional[MossResp]Defaults to None

Response metadata (None for streaming)

aistore.sdk.batch.extractor.tar_stream_extractor.logger = get_logger(__name__)