data_designer.config.processors

Bases: str, enum.Enum

Enumeration of available processor types.

Attributes:

Initialization:

Initialize self. See help(type(self)) for accurate signature.

1
DROP_COLUMNS = drop_columns

1
SCHEMA_TRANSFORM = schema_transform

Create a processor configuration from a processor type and keyword arguments.

Parameters:

Returns:

Bases: data_designer.config.base.ProcessorConfig

Drop columns from the output dataset (prefer drop=True in the column config).

This processor removes specified columns from the generated dataset. The dropped columns are saved separately in the dropped-columns-parquet-files directory for reference. When this processor is added via the config builder, the corresponding column configs are automatically marked with drop = True.

Parameters:

Inherited Attributes: name (required): Name of the processor. Attributes:

Initialization:

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

1
column_names: list[str] = Field(...)

1
processor_type: typing.Literal[data_designer.config.processors.ProcessorType]

Bases: data_designer.config.base.ProcessorConfig

Configuration for transforming the dataset schema using Jinja2 templates.

This processor creates a new dataset with a transformed schema. Each key in the template becomes a column in the output, and values are Jinja2 templates that can reference any column in the batch. The transformed dataset is written to a processors-files/{processor_name}/ directory alongside the main dataset.

Parameters:

Inherited Attributes: name (required): Name of the processor. Attributes:

Initialization:

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

1
template: dict[str, typing.Any] = Field(...)

1
processor_type: typing.Literal[data_designer.config.processors.ProcessorType]

1
validate_template(v: dict[str, typing.Any]) -> dict[str, typing.Any]