aistore.sdk.utils | NVIDIA AIStore

Module Contents

Classes

Name	Description
`HttpError`	Represents an error returned by the API.

Functions

Name	Description
`_check_path_exists`	-
`convert_to_seconds`	Converts a time value (e.g., ‘5s’, ‘10m’, ‘2h’, ‘3d’, or 10) to seconds.
`decode_response`	Parse response content from the cluster into a Python class,
`expand_braces`	Given a string template, apply bash-style brace expansion to return a list of strings
`extract_and_parse_url`	Extract provider, bucket, and whether an object is present from raw string.
`get_digest`	Get the xxhash digest of a given string.
`get_file_size`	Get the size of a file and return it in human-readable format
`get_logger`	Create or retrieve a logger with the specified configuration.
`get_provider_from_request`	Given either a Request or PreparedRequest, return an AIS bucket provider.
`is_read_timeout`	Check if a given ConnectionError was caused by an underlying ReadTimeoutError
`parse_url`	Parse AIS URLs for bucket and object names.
`probing_frequency`	Given a timeout, return an interval to wait between retries
`read_file_bytes`	Given a filepath, read the content as bytes
`validate_directory`	Validate that a directory exists and is a directory
`validate_file`	Validate that a file exists and is a file
`xoshiro256_hash`	Xoshiro256-inspired hash function with 64-bit overflow behavior.

Data

MAX_BUCKET_PART_LEN

ROTATION_BITS

T

URL_PROVIDERS

API

class aistore.sdk.utils.HttpError()

Bases: BaseModel

Represents an error returned by the API.

caller

str = ''

message

str = ''

method

str = ''

node

str = ''

remote_addr

str = ''

status_code

int

url_path

str = ''

aistore.sdk.utils._check_path_exists(
    path: pathlib.Path
) -> None

aistore.sdk.utils.convert_to_seconds(
    time_val: typing.Union[str, int]
) -> int

Converts a time value (e.g., ‘5s’, ‘10m’, ‘2h’, ‘3d’, or 10) to seconds. If no unit is provided (e.g., ‘10’ or 10), seconds are assumed.

Parameters:

time_val

Union[str, int]

The time value to convert.

Returns: int

The equivalent time in seconds.

Raises:

ValueError: If the format or unit is invalid.

aistore.sdk.utils.decode_response(
    res_model: typing.Type[aistore.sdk.utils.T],
    resp: requests.Response
) -> aistore.sdk.utils.T

Parse response content from the cluster into a Python class, decoding with msgpack depending on content type in header

Parameters:

res_model

Type[T]

Resulting type to which the response should be deserialized

resp

Response

Response from the AIS cluster

aistore.sdk.utils.expand_braces(
    template: str
) -> typing.Iterator[str]

Given a string template, apply bash-style brace expansion to return a list of strings Args: template: Valid brace expansion input, e.g. prefix-{0..10..2}-gap-{11..15}-suffix

Returns: Iterator[str]

Iterator of brace expansion output

aistore.sdk.utils.extract_and_parse_url(
    msg: str
) -> typing.Optional[typing.Tuple[str, str, bool]]

Extract provider, bucket, and whether an object is present from raw string.

Parameters:

msg

str

Any string that may contain an AIS FQN.

Returns: Optional[Tuple[str, str, bool]]

Optional[Tuple[str, str, bool]]: (prov, bck, has_obj) if a FQN is found, otherwise None.

aistore.sdk.utils.get_digest(
    name: str
) -> int

Get the xxhash digest of a given string.

aistore.sdk.utils.get_file_size(
    file: pathlib.Path
) -> str

Get the size of a file and return it in human-readable format Args: file: File to read

Returns: str

Size of file as human-readable string

aistore.sdk.utils.get_logger(
    name: str,
    log_format: str = DEFAULT_LOG_FORMAT
)

Create or retrieve a logger with the specified configuration.

Parameters:

name

str

The name of the logger.

log_format

strDefaults to DEFAULT_LOG_FORMAT

Logging format.

Returns:

logging.Logger: Configured logger instance.

aistore.sdk.utils.get_provider_from_request(
    req: typing.Union[requests.Request, requests.PreparedRequest]
) -> aistore.sdk.provider.Provider

Given either a Request or PreparedRequest, return an AIS bucket provider. The request property of a requests.RequestException can be either of these types, so this can be used to find the bucket provider involved in the initial request.

Parameters:

req

Union[requests.Request, requests.PreparedRequest]

Any request or prepared request.

Returns: Provider

Parsed AIS bucket Provider Enum.

aistore.sdk.utils.is_read_timeout(
    exc: requests.ConnectionError
) -> bool

Check if a given ConnectionError was caused by an underlying ReadTimeoutError Args: exc: Any requests.ConnectionError.

Returns: If ReadTimeoutError cause the exception.

aistore.sdk.utils.parse_url(
    url: str
) -> typing.Tuple[str, str, str]

Parse AIS URLs for bucket and object names.

Parameters:

url

str

Complete URL of the object (e.g., “ais://bucket1/file.txt”)

Returns: Tuple[str, str, str]

Tuple[str, str, str]: Provider, bucket name, and object name

aistore.sdk.utils.probing_frequency(
    dur: int
) -> float

Given a timeout, return an interval to wait between retries

Parameters:

dur

int

Duration of timeout

Returns: float

Frequency to probe

aistore.sdk.utils.read_file_bytes(
    filepath: str
) -> bytes

Given a filepath, read the content as bytes Args: filepath: Existing local filepath

Returns: Raw bytes

aistore.sdk.utils.validate_directory(
    path: typing.Union[str, pathlib.Path]
) -> None

Validate that a directory exists and is a directory Args: path (str or Path): Path to validate Raises: ValueError: If path does not exist or is not a directory

aistore.sdk.utils.validate_file(
    path: typing.Union[str, pathlib.Path]
) -> None

Validate that a file exists and is a file Args: path (str or Path): Path to validate Raises: ValueError: If path does not exist or is not a file

aistore.sdk.utils.xoshiro256_hash(
    seed: int
) -> int

Xoshiro256-inspired hash function with 64-bit overflow behavior.

aistore.sdk.utils.BUCKET_CHARS = '[A-Za-z0-9@#._-]'

aistore.sdk.utils.CONST1 = 13787848793156543929

aistore.sdk.utils.CONST2 = 10723151780598845931

aistore.sdk.utils.GOLDEN_RATIO = 11400714819323198485

aistore.sdk.utils.MASK = 18446744073709551615

aistore.sdk.utils.MAX_BUCKET_PART_LEN = 132

aistore.sdk.utils.ROTATION_BITS = 7

aistore.sdk.utils.T = TypeVar('T')

aistore.sdk.utils.URL_PROVIDERS = '|'.join([(p.value) for p in Provider] + list(provider_aliases))