aiq.eval.rag_evaluator.register

Attributes

logger

Classes

RagasMetricConfig	RAGAS metrics configuration
RagasEvaluatorConfig	Evaluation using RAGAS metrics.

Functions

register_ragas_evaluator(config, builder)

Module Contents

logger

class RagasMetricConfig(/, **data: Any)

Bases: pydantic.BaseModel

RAGAS metrics configuration skip: Allows the metric config to be present but not used kwargs: Additional arguments to pass to the metric’s callable

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

skip: bool = False

kwargs: dict | None = None

class RagasEvaluatorConfig(/, **data: Any)

Bases: aiq.data_models.evaluator.EvaluatorBaseConfig

Evaluation using RAGAS metrics.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

llm_name: str = None

metric: str | dict[str, RagasMetricConfig] = None

classmethod validate_metric(values)

Ensures metric is either a string or a single-item dictionary.

property metric_name: str

Returns the single metric name.

property metric_config: RagasMetricConfig

Returns the metric configuration (or a default if only a string is provided).

async register_ragas_evaluator(

config: RagasEvaluatorConfig,

builder: aiq.builder.builder.EvalBuilder,

)