aistore.pytorch.utils | NVIDIA AIStore

Utils for AIS PyTorch Plugin

Module Contents

Functions

Name	Description
`convert_bytes_to_mb`	Converts byes to megabytes.
`convert_mb_to_bytes`	Converts megabytes to bytes and truncates any extra bytes (floor).
`get_basename`	Get the basename of the object name by stripping any directory information and suffix.
`get_extension`	Get the file extension of the object by stripping any basename or prefix.
`parse_url`	Wrapper of sdk/utils.py parse_url. Parse AIS URLs for bucket and object names.
`unparse_url`	Generate URL based on provider, bucket name, and object name.

Data

MB_TO_B

API

aistore.pytorch.utils.convert_bytes_to_mb(
    bytes: int
) -> float

Converts byes to megabytes.

Parameters:

bytes

int

number of bytes to convert to megabytes

Returns: float

number of megabytes after conversion

aistore.pytorch.utils.convert_mb_to_bytes(
    megabytes: float
) -> int

Converts megabytes to bytes and truncates any extra bytes (floor).

Parameters:

megabytes

float

number of megabytes to convert

Returns: int

number of bytes after conversion (floor of actual byte value)

aistore.pytorch.utils.get_basename(
    name: str
) -> str

Get the basename of the object name by stripping any directory information and suffix.

Parameters:

name

str

Complete object name

Returns: str

Basename of the object

aistore.pytorch.utils.get_extension(
    name: str
) -> str

Get the file extension of the object by stripping any basename or prefix.

Parameters:

name

str

Complete object name

Returns: str

File extension of the object

aistore.pytorch.utils.parse_url(
    url: str
) -> typing.Tuple[str, str, str]

Wrapper of sdk/utils.py parse_url. Parse AIS URLs for bucket and object names. TODO: This can be removed once the upstream torch package for aiso is updated.

Parameters:

url

str

Complete URL of the object (e.g., “ais://bucket1/file.txt”)

Returns: Tuple[str, str, str]

Tuple[str, str, str]: Provider, bucket name, and object name

aistore.pytorch.utils.unparse_url(
    provider: str,
    bck_name: str,
    obj_name: str
) -> str

Generate URL based on provider, bucket name, and object name.

Parameters:

provider

str

Provider name (‘ais’, ‘gcp’, etc.)

bck_name

str

Bucket name

obj_name

str

Object name with extension

Returns: str

Complete URL

aistore.pytorch.utils.MB_TO_B = 1000000