aistore.sdk.dsort.framework

Module Contents

Classes

Name	Description
`DsortAlgorithm`	Represents the algorithm used in a Dsort job
`DsortFramework`	Represents the framework for a dSort job, including input and output shard configurations.
`DsortShardsGroup`	Represents the configuration for the input or output of a shard group in a dSort job

API

class aistore.sdk.dsort.framework.DsortAlgorithm()

Bases: BaseModel

Represents the algorithm used in a Dsort job

content_key_type

Optional[Literal['int', 'float', 'string']] = None

decreasing

bool = False

extension

Optional[str] = None

kind

Literal['alphanumeric', 'shuffle', 'content'] = 'alphanumeric'

seed

Optional[str] = ''

aistore.sdk.dsort.framework.DsortAlgorithm.as_dict()

Converts the DsortAlgorithm instance to a dictionary representation

aistore.sdk.dsort.framework.DsortAlgorithm.validate_content_fields()

Validates required key fields

class aistore.sdk.dsort.framework.DsortFramework(
    input_shards: aistore.sdk.dsort.framework.DsortShardsGroup,
    output_shards: aistore.sdk.dsort.framework.DsortShardsGroup,
    algorithm: typing.Optional[aistore.sdk.dsort.framework.DsortAlgorithm] = None,
    description = None,
    output_shard_size = None
)

Represents the framework for a dSort job, including input and output shard configurations.

aistore.sdk.dsort.framework.DsortFramework.from_file(
    spec
)

classmethod

Class method to create a DsortFramework instance from a JSON or YAML file

Parameters:

spec

str or Path

The path to the JSON or YAML file containing the specification

Returns:

An instance of the DsortFramework class

Raises:

ValueError: If the file extension is not .json, .yml, or .yaml

aistore.sdk.dsort.framework.DsortFramework.to_spec()

Converts the DsortFramework instance to a dictionary representation for use in dSort job specification

Returns:

Dictionary representation of dSort specification

class aistore.sdk.dsort.framework.DsortShardsGroup(
    bck: aistore.sdk.types.BucketModel,
    role: typing.Literal['input', 'output'],
    format: typing.Union[aistore.sdk.multiobj.ObjectNames, aistore.sdk.multiobj.ObjectRange, aistore.sdk.dsort.ekm.ExternalKeyMap],    format: typing.Union[aistore.sdk.multiobj.ObjectNames, aistore.sdk.multiobj.ObjectRange, aistore.sdk.dsort.ekm.ExternalKeyMap],
    extension: str
)

Represents the configuration for the input or output of a shard group in a dSort job

aistore.sdk.dsort.framework.DsortShardsGroup.as_dict()

Converts the DsortShardsGroup instance to a dictionary representation