Python SDKPythonPythonAistoreSdkBatch

aistore.sdk.batch.batch

View as Markdown

Module Contents

Classes

NameDescription
BatchBatch (Get-Batch) API - Direct mapping to Go’s MOSS (Multi-Object Streaming Service).

Data

BatchResult

_BUCKET_REQUIRED_MSG

logger

API

class aistore.sdk.batch.batch.Batch(
    request_client: aistore.sdk.request_client.RequestClient,
    objects: typing.Optional[typing.Union[typing.List[aistore.sdk.obj.object.Object], aistore.sdk.obj.object.Object, str, typing.List[str]]] = None,    objects: typing.Optional[typing.Union[typing.List[aistore.sdk.obj.object.Object], aistore.sdk.obj.object.Object, str, typing.List[str]]] = None,
    bucket: typing.Optional[aistore.sdk.bucket.Bucket] = None,
    output_format: str = EXT_TAR,
    cont_on_err: bool = True,
    only_obj_name: bool = False,
    streaming_get: bool = True,
    colocation: aistore.sdk.enums.Colocation = Colocation.NONE
)

Batch (Get-Batch) API - Direct mapping to Go’s MOSS (Multi-Object Streaming Service).

Builds and executes Get-Batch requests to retrieve multiple objects, archived files, or byte ranges in a single efficient operation.

extractor
= ExtractorManager().get_extractor(output_format)
request
requests_list
List[MossIn]

Return the list of MossIn requests in the batch.

aistore.sdk.batch.batch.Batch.__len__() -> int

Number of objects in batch.

Returns: int

Number of objects in the batch

aistore.sdk.batch.batch.Batch.__repr__() -> str

String representation of the Batch object.

Returns: str

String representation showing number of objects and format

aistore.sdk.batch.batch.Batch._add_objects(
    objects: typing.Union[typing.List[aistore.sdk.obj.object.Object], aistore.sdk.obj.object.Object, str, typing.List[str]]    objects: typing.Union[typing.List[aistore.sdk.obj.object.Object], aistore.sdk.obj.object.Object, str, typing.List[str]]
)

Internal helper to add objects in bulk. Supports strings, Object instances, or lists of either.

aistore.sdk.batch.batch.Batch._extract_multipart(
    response,
    decode_as_stream: bool,
    request_snapshot: aistore.sdk.batch.types.MossReq
) -> aistore.sdk.batch.batch.BatchResult

Extract from multipart response (with metadata). Returns actual MossOut with size, errors, etc.

Parameters:

response
Response

HTTP response object

decode_as_stream
bool

Whether to decode multipart as stream

request_snapshot
MossReq

Snapshot of the request for this batch

Returns: BatchResult

Generator yielding (MossOut, content) tuples

aistore.sdk.batch.batch.Batch._extract_streaming(
    response,
    request_snapshot: aistore.sdk.batch.types.MossReq
) -> aistore.sdk.batch.batch.BatchResult

Extract from streaming response (no metadata). Infer MossOut from request data.

Parameters:

response
Response

HTTP response object

request_snapshot
MossReq

Snapshot of the request for this batch

Returns: BatchResult

Generator yielding (MossOut, content) tuples

aistore.sdk.batch.batch.Batch.add(
    obj: typing.Union[aistore.sdk.obj.object.Object, str],
    opaque: typing.Optional[bytes] = None,
    archpath: typing.Optional[str] = None,
    start: typing.Optional[int] = None,
    length: typing.Optional[int] = None
) -> aistore.sdk.batch.batch.Batch

Add object with advanced parameters (archpath, byte ranges, opaque data).

For simple objects, prefer passing them to init instead.

Note: if objects are specified as raw names (str), default bucket must be provided in init

Parameters:

obj
Union[Object, str]

Object or object name string

opaque
Optional[bytes]Defaults to None

User-provided binary identifier (returned unchanged)

archpath
Optional[str]Defaults to None

Extract file from archive (e.g., “images/photo.jpg”)

start
Optional[int]Defaults to None

Byte range start offset

length
Optional[int]Defaults to None

Byte range length

Returns: Batch

Self for method chaining

aistore.sdk.batch.batch.Batch.clear() -> aistore.sdk.batch.batch.Batchaistore.sdk.batch.batch.Batch.clear() -> aistore.sdk.batch.batch.Batch

Clear all objects from the batch request.

This allows reusing the same batch instance for multiple requests without creating a new Batch object.

Returns: Batch

Self for method chaining

aistore.sdk.batch.batch.Batch.get(
    raw: bool = False,
    decode_as_stream: bool = False,
    clear_batch: bool = True
) -> typing.Union[aistore.sdk.batch.batch.BatchResult, urllib3.response.HTTPResponse]

Execute the Get-Batch request.

After execution, the batch is automatically cleared to allow reuse. You can add new objects and call get() again on the same instance.

Parameters:

raw
boolDefaults to False

Return raw HTTP response stream. User must close the stream

decode_as_stream
boolDefaults to False

Stream multipart decoding (memory efficient)

clear_batch
boolDefaults to True

Whether to clear the batch after execution.

  • True (default): Automatically clears all objects from the batch after running get(), enabling reuse with new objects.
  • False: Retains the batch objects, allowing repeated get() calls or adding more objects before execution.

Returns: Union[BatchResult, HTTPResponse]

Union[BatchResult, HTTPResponse]:

  • If raw=True: HTTPResponse object (caller must close)
  • If raw=False: Generator yielding (MossOut, file_content) tuples

Raises:

  • ValueError: If no objects added to batch
aistore.sdk.batch.batch.BatchResult = Generator[Tuple[MossOut, bytes], None, None]
 
aistore.sdk.batch.batch._BUCKET_REQUIRED_MSG = 'Bucket must be provided when objects are specified as raw names (str or list of...
 
aistore.sdk.batch.batch.logger = get_logger(__name__)