aiq.data_models.dataset_handler

Attributes

EvalDatasetConfig

Classes

EvalS3Config
EvalFilterEntryConfig
EvalFilterConfig
EvalDatasetStructureConfig
EvalDatasetBaseConfig	Subclass of Pydantic BaseModel that allows for specifying the object type. Use in Pydantic discriminated unions.
EvalDatasetJsonConfig	Subclass of Pydantic BaseModel that allows for specifying the object type. Use in Pydantic discriminated unions.
EvalDatasetJsonlConfig	Subclass of Pydantic BaseModel that allows for specifying the object type. Use in Pydantic discriminated unions.
EvalDatasetCsvConfig	Subclass of Pydantic BaseModel that allows for specifying the object type. Use in Pydantic discriminated unions.
EvalDatasetParquetConfig	Subclass of Pydantic BaseModel that allows for specifying the object type. Use in Pydantic discriminated unions.
EvalDatasetXlsConfig	Subclass of Pydantic BaseModel that allows for specifying the object type. Use in Pydantic discriminated unions.

Functions

read_jsonl(file_path, **kwargs)

Module Contents

class EvalS3Config(/, **data: Any)

Bases: pydantic.BaseModel

endpoint_url: str

bucket: str

access_key: str

secret_key: str

class EvalFilterEntryConfig(/, **data: Any)

Bases: pydantic.BaseModel

field: dict[str, list[str | int | float]]

class EvalFilterConfig(/, **data: Any)

Bases: pydantic.BaseModel

allowlist: EvalFilterEntryConfig | None = None

denylist: EvalFilterEntryConfig | None = None

class EvalDatasetStructureConfig(/, **data: Any)

Bases: pydantic.BaseModel

disable: bool = False

question_key: str = 'question'

answer_key: str = 'answer'

generated_answer_key: str = 'generated_answer'

trajectory_key: str = 'intermediate_steps'

expected_trajectory_key: str = 'expected_intermediate_steps'

class EvalDatasetBaseConfig(/, **data: Any)

Bases: aiq.data_models.common.TypedBaseModel, aiq.data_models.common.BaseModelRegistryTag

Subclass of Pydantic BaseModel that allows for specifying the object type. Use in Pydantic discriminated unions.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

id_key: str = 'id'

structure: EvalDatasetStructureConfig

filter: EvalFilterConfig | None

s3: EvalS3Config | None = None

remote_file_path: str | None = None

file_path: pathlib.Path | str

class EvalDatasetJsonConfig(/, **data: Any)

Bases: EvalDatasetBaseConfig

Subclass of Pydantic BaseModel that allows for specifying the object type. Use in Pydantic discriminated unions.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

static parser() → tuple[collections.abc.Callable, dict]

read_jsonl(file_path: pydantic.FilePath, **kwargs)

class EvalDatasetJsonlConfig(/, **data: Any)

Bases: EvalDatasetBaseConfig

Subclass of Pydantic BaseModel that allows for specifying the object type. Use in Pydantic discriminated unions.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

static parser() → tuple[collections.abc.Callable, dict]

class EvalDatasetCsvConfig(/, **data: Any)

Bases: EvalDatasetBaseConfig

Subclass of Pydantic BaseModel that allows for specifying the object type. Use in Pydantic discriminated unions.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

static parser() → tuple[collections.abc.Callable, dict]

class EvalDatasetParquetConfig(/, **data: Any)

Bases: EvalDatasetBaseConfig

Subclass of Pydantic BaseModel that allows for specifying the object type. Use in Pydantic discriminated unions.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

static parser() → tuple[collections.abc.Callable, dict]

class EvalDatasetXlsConfig(/, **data: Any)

Bases: EvalDatasetBaseConfig

Subclass of Pydantic BaseModel that allows for specifying the object type. Use in Pydantic discriminated unions.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

static parser() → tuple[collections.abc.Callable, dict]

EvalDatasetConfig