Python SDKPythonPythonAistoreSdk

aistore.sdk.types

View as MarkdownOpen in Claude

Module Contents

Classes

NameDescription
ActionMsgRepresents the action message passed by the client via json
AggregatedJobSnapRepresents job snapshots grouped by target ID.
ArchiveMultiObjAPI message structure for multi-object archive requests
BlobMsgRepresents the set of args the sdk will pass to AIStore when making a blob-download request
BsummCtrlMsgRepresents the bucket summary control message
BucketEntryRepresents a single entry in a bucket — an object
BucketListRepresents the response when getting a list of bucket items, containing a list of BucketEntry objects
BucketModelRepresents the response from the API containing bucket info
CopyBckMsgAPI message structure for copying a bucket
CreateNBIMsgAPI message structure for creating a native bucket inventory.
ETLDetailsRepresents the API response of queries on single ETL details
ETLInfoRepresents the API response when querying an ETL
ETLInitMsgRepresents the API message structure for initializing an ETL
ETLNodeLogsRepresents decoded logs from a single ETL pod on one target node.
ETLObjErrorRepresents an error encountered during ETL processing of an object
ETLRuntimeSpecRepresents the ETL runtime message structure
ETLSpecMsgSimplified version for Init Spec ETL args
EnvVarRepresents an environment variable
InitETLArgsRepresents the args shared by ETL initialization with code or spec
InitSpecETLArgsRepresents the set of args the sdk will pass to AIStore when making a request to initialize an ETL with a spec
JobArgsRepresents the set of args to pass when making a job-related request
JobQueryStructure to send the API when querying the cluster for multiple jobs
JobSnapRepresents a snapshot of a job on a target node.
JobStatsRepresents performance statistics for a job.
JobStatusRepresents the response of an API query to fetch job status
ListObjectsMsgAPI message structure for listing objects in a bucket
MptCompletedPartRepresents a part of a multipart upload
NBIInfoInformation about a single native bucket inventory.
NetInfoRepresents a set of network-related info
PrefetchMsgAPI message structure for prefetching objects from remote buckets.
PromoteAPIArgsRepresents the set of args the sdk will pass to AIStore when making a promote request and
SmapRepresents a system map
SnodeRepresents a system node
TCBckMsgAPI message structure for transforming or copying between buckets.
TCMultiObjAPI message structure for transforming or copying multiple objects between buckets
TransformBckMsgAPI message structure for requesting an etl transform on a bucket

API

class aistore.sdk.types.ActionMsg()

Bases: BaseModel

Represents the action message passed by the client via json

action
str
name
str = ''
class aistore.sdk.types.AggregatedJobSnap()

Bases: RootModel[Dict[str, List[JobSnap]]]

Represents job snapshots grouped by target ID.

aistore.sdk.types.AggregatedJobSnap.all_idle() -> bool

Check if all snapshots are idle, aborted, or have an abort error.

aistore.sdk.types.AggregatedJobSnap.any_finished() -> bool

Check if any snapshot is finished (aborted, errored, or has valid end_time).

aistore.sdk.types.AggregatedJobSnap.get_num_workers() -> int

Return the nworkers from any one snapshot (as all are the same).

aistore.sdk.types.AggregatedJobSnap.list_snapshots() -> typing.List[aistore.sdk.types.JobSnap]

Returns a flat list of all job snapshots across all target nodes.

class aistore.sdk.types.ArchiveMultiObj()

Bases: BaseModel

API message structure for multi-object archive requests

allow_append
bool = False
archive_name
str
continue_on_err
bool = False
include_source_name
bool = False
mime
Optional[str] = None
object_selection
Dict
to_bck
BucketModel
aistore.sdk.types.ArchiveMultiObj.as_dict()
class aistore.sdk.types.BlobMsg()

Bases: BaseModel

Represents the set of args the sdk will pass to AIStore when making a blob-download request and provides conversion to the expected json format

chunk_size
Optional[int] = None
latest
bool
num_workers
Optional[int] = None
aistore.sdk.types.BlobMsg.as_dict()
class aistore.sdk.types.BsummCtrlMsg()

Bases: BaseModel

Represents the bucket summary control message

cached
bool
prefix
str
present
bool
uuid
str
class aistore.sdk.types.BucketEntry()

Bases: Struct

Represents a single entry in a bucket — an object See cmn/objlist.go/LsoEntry

a
str = ''
c
int = 0
cs
str = ''
f
int = 0
n
str
s
int = 0
t
str = ''
v
str = ''
aistore.sdk.types.BucketEntry.generate_object_props() -> aistore.sdk.obj.object_props.ObjectProps

Convert bucket entry data into Object Props.

NOTE: Bucket entry data and object props are not a one-to-one mapping.

Returns: ObjectProps

ObjectProps with object data

aistore.sdk.types.BucketEntry.is_cached()
aistore.sdk.types.BucketEntry.is_ok()
class aistore.sdk.types.BucketList()

Bases: Struct

Represents the response when getting a list of bucket items, containing a list of BucketEntry objects

ContinuationToken
str
Entries
Optional[List[BucketEntry]] = None
Flags
int
UUID
str
class aistore.sdk.types.BucketModel()

Bases: BaseModel

Represents the response from the API containing bucket info

name
str
namespace
Optional[Namespace] = None
provider
str
aistore.sdk.types.BucketModel.as_dict()
aistore.sdk.types.BucketModel.get_path() -> str

Get the path representation of this bucket

class aistore.sdk.types.CopyBckMsg()

Bases: BaseModel

API message structure for copying a bucket

dry_run
bool
force
bool
latest
bool
prefix
str = ''
prepend
str
sync
bool
aistore.sdk.types.CopyBckMsg.as_dict()
class aistore.sdk.types.CreateNBIMsg()

Bases: BaseModel

API message structure for creating a native bucket inventory. See api/apc/nbi.go CreateNBIMsg.

flags
str = '0'
force
bool = False
name
str = ''
names_per_chunk
int = 0
prefix
str = ''
props
str = ''
aistore.sdk.types.CreateNBIMsg.as_dict()
class aistore.sdk.types.ETLDetails()

Bases: BaseModel

Represents the API response of queries on single ETL details

init_msg
ETLInitMsg
obj_errors
Optional[List[ETLObjError]] = None
class aistore.sdk.types.ETLInfo()

Bases: BaseModel

Represents the API response when querying an ETL

id
str = ''
in_bytes
int = 0
obj_count
int = 0
out_bytes
int = 0
stage
str = ''
xaction_id
str = ''
class aistore.sdk.types.ETLInitMsg()

Bases: BaseModel

Represents the API message structure for initializing an ETL

chunk_size
int = 0
code
Optional[bytes] = None
communication
str
dependencies
Optional[bytes] = None
init_timeout
Optional[str] = None
name
str
obj_timeout
Optional[str] = None
runtime
Optional[Union[str, ETLRuntimeSpec]] = None
spec
Optional[bytes] = None
aistore.sdk.types.ETLInitMsg.set_code(
    code
)
aistore.sdk.types.ETLInitMsg.set_spec(
    spec
)
class aistore.sdk.types.ETLNodeLogs()

Bases: BaseModel

Represents decoded logs from a single ETL pod on one target node.

logs
str = ''
target_id
str = ''
class aistore.sdk.types.ETLObjError()

Bases: BaseModel

Represents an error encountered during ETL processing of an object

ecode
int
msg
str
obj_name
str
class aistore.sdk.types.ETLRuntimeSpec()

Bases: BaseModel

Represents the ETL runtime message structure

command
Optional[List[str]] = None
env
Optional[List[EnvVar]] = None
image
str
aistore.sdk.types.ETLRuntimeSpec.as_dict() -> typing.Dict[str, typing.Any]
class aistore.sdk.types.ETLSpecMsg()

Bases: InitETLArgs

Simplified version for Init Spec ETL args

runtime
ETLRuntimeSpec
aistore.sdk.types.ETLSpecMsg.as_dict()
class aistore.sdk.types.EnvVar()

Bases: BaseModel

Represents an environment variable

name
str
value
str
aistore.sdk.types.EnvVar.as_dict()
aistore.sdk.types.EnvVar.convert_to_str(
    v
)
class aistore.sdk.types.InitETLArgs()

Bases: BaseModel

Represents the args shared by ETL initialization with code or spec

comm_type
str
direct_put
bool = False
init_timeout
Optional[str]
name
str
obj_timeout
Optional[str]
class aistore.sdk.types.InitSpecETLArgs()

Bases: InitETLArgs

Represents the set of args the sdk will pass to AIStore when making a request to initialize an ETL with a spec

spec
str
aistore.sdk.types.InitSpecETLArgs.as_dict()
class aistore.sdk.types.JobArgs()

Bases: BaseModel

Represents the set of args to pass when making a job-related request

bucket
Optional[BucketModel] = None
buckets
Optional[List[BucketModel]] = None
daemon_id
str = ''
id
str = ''
kind
str = ''
aistore.sdk.types.JobArgs.as_dict()
class aistore.sdk.types.JobQuery()

Bases: BaseModel

Structure to send the API when querying the cluster for multiple jobs

active
bool = False
kind
str = ''
target
str = ''
aistore.sdk.types.JobQuery.as_dict()
class aistore.sdk.types.JobSnap()

Bases: BaseModel

Represents a snapshot of a job on a target node.

abort_err
str = Field(default='', alias='abort-err')
aborted
bool = Field(default=False)
bucket
Optional[BucketModel] = Field(default=None, alias='bck')
destination_bck
Optional[BucketModel] = Field(default=None, alias='dst-bck')
end_time
str = Field(default='', alias='end-time')
id
str = Field(default='')
is_idle
bool = Field(default=False)
kind
str = Field(default='')
model_config
= {'populate_by_name': True}
packed
int = Field(default=0, alias='glob.id')
source_bck
Optional[BucketModel] = Field(default=None, alias='src-bck')
start_time
str = Field(default='', alias='start-time')
stats
Optional[JobStats] = Field(default=None)
aistore.sdk.types.JobSnap.unpack_glob_id() -> tuple[int, int, int]

Unpack the glob_id into (njoggers, nworkers, chan_full). The packed value is a bitwise representation of the number of joggers, workers, and channel full status.

class aistore.sdk.types.JobStats()

Bases: BaseModel

Represents performance statistics for a job.

bytes
int = Field(default=0, alias='loc-bytes')
in_bytes
int = Field(default=0, alias='in-bytes')
in_objects
int = Field(default=0, alias='in-objs')
model_config
= {'populate_by_name': True}
objects
int = Field(default=0, alias='loc-objs')
out_bytes
int = Field(default=0, alias='out-bytes')
out_objects
int = Field(default=0, alias='out-objs')
class aistore.sdk.types.JobStatus()

Bases: BaseModel

Represents the response of an API query to fetch job status

aborted
bool = False
end_time
int = 0
err
str = ''
uuid
str = ''
class aistore.sdk.types.ListObjectsMsg()

Bases: BaseModel

API message structure for listing objects in a bucket

continuation_token
str
flags
List[ListObjectFlag]
page_size
int
prefix
str
props
str
target
str
uuid
str
aistore.sdk.types.ListObjectsMsg.as_dict()
class aistore.sdk.types.MptCompletedPart()

Bases: BaseModel

Represents a part of a multipart upload

etag
str = ''
part_number
int
aistore.sdk.types.MptCompletedPart.as_dict()
class aistore.sdk.types.NBIInfo()

Bases: BaseModel

Information about a single native bucket inventory. See api/apc/nbi.go NBIInfo.

bucket
str = ''
chunks
int = 0
finished
int = 0
name
str = ''
nat
int = 0
ntotal
int = 0
obj_name
str = ''
prefix
str = ''
size
int = 0
smap_ver
int = 0
started
int = 0
class aistore.sdk.types.NetInfo()

Bases: BaseModel

Represents a set of network-related info

daemon_port
str = ''
direct_url
str = ''
node_hostname
str = ''
class aistore.sdk.types.PrefetchMsg()

Bases: BaseModel

API message structure for prefetching objects from remote buckets.

blob_threshold
Optional[int] = None
continue_on_err
bool
latest
bool
num_workers
Optional[int] = None
object_selection
Dict
aistore.sdk.types.PrefetchMsg.as_dict()
class aistore.sdk.types.PromoteAPIArgs()

Bases: BaseModel

Represents the set of args the sdk will pass to AIStore when making a promote request and provides conversion to the expected json format

delete_source
bool = False
object_name
str = ''
overwrite_dest
bool = False
recursive
bool = False
source_path
str = ''
src_not_file_share
bool = False
target_id
str = ''
aistore.sdk.types.PromoteAPIArgs.as_dict()
class aistore.sdk.types.Smap()

Bases: BaseModel

Represents a system map

creation_time
str = ''
pmap
Mapping[str, Snode]
proxy_si
Snode
tmap
Mapping[str, Snode]
uuid
str = ''
version
int = 0
aistore.sdk.types.Smap.get_target_for_object(
    uname: str
) -> aistore.sdk.types.Snode

Determine the target node responsible for an object based on its bucket path and name.

Parameters:

uname
str

Fully qualified (namespaced) object name (e.g., f”{bck.get_path()}{obj.name}”).

Returns: Snode

The assigned target node.

Raises:

  • NoTargetError: If no target node is found.
class aistore.sdk.types.Snode()

Bases: BaseModel

Represents a system node

daemon_id
str
daemon_type
str
flags
int = 0
id_digest
int = 0
intra_control_net
Optional[NetInfo] = None
intra_data_net
Optional[NetInfo] = None
public_net
Optional[NetInfo] = None
aistore.sdk.types.Snode.in_maint_or_decomm() -> bool
class aistore.sdk.types.TCBckMsg()

Bases: BaseModel

API message structure for transforming or copying between buckets. Can be used on its own for an entire bucket or encapsulated in TCMultiObj to apply only to a selection of objects

cont_on_err
bool = False
copy_msg
Optional[CopyBckMsg] = None
ext
Optional[Dict[str, str]] = None
num_workers
Optional[int] = None
transform_msg
Optional[TransformBckMsg] = None
aistore.sdk.types.TCBckMsg.as_dict()
class aistore.sdk.types.TCMultiObj()

Bases: BaseModel

API message structure for transforming or copying multiple objects between buckets

continue_on_err
bool
num_workers
Optional[int] = None
object_selection
Dict
tc_msg
Optional[TCBckMsg] = None
to_bck
BucketModel
aistore.sdk.types.TCMultiObj.as_dict()
class aistore.sdk.types.TransformBckMsg()

Bases: BaseModel

API message structure for requesting an etl transform on a bucket

etl_name
str
etl_pipeline
Optional[List[str]] = None
timeout
str
aistore.sdk.types.TransformBckMsg.as_dict()