Example Plugin: Column Generator

Data Designer supports three plugin types: column generators, seed readers, and processors. This page walks through a complete column generator example. For filesystem-backed seed reader plugins, see FileSystemSeedReader Plugins.

A Data Designer plugin is implemented as a Python package with three main components:

1. Configuration Class: Defines the parameters users can configure
2. Implementation Class: Contains the core logic of the plugin
3. Plugin Object: Connects the config and implementation classes to make the plugin discoverable

We recommend separating these into individual files (config.py, impl.py, plugin.py) within a plugin subdirectory. This keeps the code organized, makes it easy to test each component independently, and guards against circular dependencies — since the config module can be imported without pulling in the engine-level implementation classes, and the plugin object can be discovered without importing either.

Column Generator Plugin: Index Multiplier

In this section, we will build a simple column generator plugin that generates values by multiplying the row index by a user-specified multiplier.

Step 1: Create a Python package

We recommend the following structure for column generator plugins:

data-designer-index-multiplier/
├── pyproject.toml
└── src/
    └── data_designer_index_multiplier/
        ├── __init__.py
        ├── config.py
        ├── impl.py
        └── plugin.py

Step 2: Create the config class

The configuration class defines what parameters users can set when using your plugin. For column generator plugins, it must inherit from SingleColumnConfig and include a discriminator field.

Create src/data_designer_index_multiplier/config.py:

1
from typing import Literal
2
3
from data_designer.config.base import SingleColumnConfig
4
5
6
class IndexMultiplierColumnConfig(SingleColumnConfig):
7
    """Configuration for the index multiplier column generator."""
8
9
    # Required: discriminator field with a unique Literal type
10
    # This value identifies your plugin and becomes its column_type
11
    column_type: Literal["index-multiplier"] = "index-multiplier"
12
13
    # Configurable parameter for this plugin
14
    multiplier: int = 2
15
16
    @staticmethod
17
    def get_column_emoji() -> str:
18
        return "✖️"
19
20
    @property
21
    def required_columns(self) -> list[str]:
22
        """Columns that must exist before this generator runs."""
23
        return []
24
25
    @property
26
    def side_effect_columns(self) -> list[str]:
27
        """Additional columns produced beyond the primary column."""
28
        return []

Key points:

• The column_type field must be a Literal type with a string default
• This value uniquely identifies your plugin (use kebab-case)
• Add any custom parameters your plugin needs (here: multiplier)
• SingleColumnConfig is a Pydantic model, so you can leverage all of Pydantic’s validation features
• get_column_emoji() returns the emoji displayed in logs for this column type
• required_columns lists any columns this generator depends on (empty if none)
• side_effect_columns lists any additional columns this generator produces beyond the primary column (empty if none)

If your plugin can expand or retract the number of rows (1:N or N:1): set allow_resize=True in the config class so the pipeline updates batch bookkeeping correctly. For example:

1
class MyColumnConfig(SingleColumnConfig):
2
    column_type: Literal["my-plugin"] = "my-plugin"
3
    allow_resize: bool = True  # required when output row count can differ from input
4
    # ...

The default is False; only set it to True when your generate method can return more or fewer rows than it receives.

Step 3: Create the implementation class

The implementation class defines the actual business logic of the plugin. For column generator plugins, inherit from ColumnGeneratorFullColumn or ColumnGeneratorCellByCell and implement the generate method.

Create src/data_designer_index_multiplier/impl.py:

1
import logging
2
3
import pandas as pd
4
from data_designer.engine.column_generators.generators.base import ColumnGeneratorFullColumn
5
6
from data_designer_index_multiplier.config import IndexMultiplierColumnConfig
7
8
logger = logging.getLogger(__name__)
9
10
11
class IndexMultiplierColumnGenerator(ColumnGeneratorFullColumn[IndexMultiplierColumnConfig]):
12
13
    def generate(self, data: pd.DataFrame) -> pd.DataFrame:
14
        """Generate the column data.
15
16
        Args:
17
            data: The current DataFrame being built
18
19
        Returns:
20
            The DataFrame with the new column added
21
        """
22
        logger.info(
23
            f"Generating column {self.config.name} "
24
            f"with multiplier {self.config.multiplier}"
25
        )
26
27
        data[self.config.name] = data.index * self.config.multiplier
28
29
        return data

Key points:

• Generic type ColumnGeneratorFullColumn[IndexMultiplierColumnConfig] connects the implementation to its config
• You have access to the configuration parameters via self.config

Step 4: Create the plugin object

Create a Plugin object that makes the plugin discoverable and connects the implementation and config classes.

Create src/data_designer_index_multiplier/plugin.py:

1
from data_designer.plugins import Plugin, PluginType
2
3
plugin = Plugin(
4
    config_qualified_name="data_designer_index_multiplier.config.IndexMultiplierColumnConfig",
5
    impl_qualified_name="data_designer_index_multiplier.impl.IndexMultiplierColumnGenerator",
6
    plugin_type=PluginType.COLUMN_GENERATOR,
7
)

Step 5: Package your plugin

Create a pyproject.toml file to define your package and register the entry point:

1
[project]
2
name = "data-designer-index-multiplier"
3
version = "1.0.0"
4
description = "Data Designer index multiplier plugin"
5
requires-python = ">=3.10"
6
dependencies = [
7
    "data-designer",
8
]
9
10
# Register this plugin via entry points
11
[project.entry-points."data_designer.plugins"]
12
index-multiplier = "data_designer_index_multiplier.plugin:plugin"
13
14
[build-system]
15
requires = ["hatchling"]
16
build-backend = "hatchling.build"
17
18
[tool.hatch.build.targets.wheel]
19
packages = ["src/data_designer_index_multiplier"]

1
[project.entry-points."data_designer.plugins"]
2
<entry-point-name> = "<module.path>:<plugin-instance-name>"

Step 6: Install and use your plugin locally

Install your plugin in editable mode — this is all you need to start using it. No PyPI publishing required:

$
# From the plugin directory
$
uv pip install -e .

That’s it. The editable install registers the entry point so Data Designer discovers your plugin automatically. Any changes you make to the plugin source code are picked up immediately without reinstalling.

Once installed, your plugin works just like built-in column types:

1
import data_designer.config as dd
2
from data_designer.interface import DataDesigner
3
4
from data_designer_index_multiplier.config import IndexMultiplierColumnConfig
5
6
data_designer = DataDesigner()
7
builder = dd.DataDesignerConfigBuilder()
8
9
# Add a regular column
10
builder.add_column(
11
    dd.SamplerColumnConfig(
12
        name="category",
13
        sampler_type="category",
14
        params=dd.CategorySamplerParams(values=["A", "B", "C"]),
15
    )
16
)
17
18
# Add your custom plugin column
19
builder.add_column(
20
    IndexMultiplierColumnConfig(
21
        name="scaled_index",
22
        multiplier=5,
23
    )
24
)
25
26
# Generate data
27
results = data_designer.create(builder, num_records=10)
28
print(results.load_dataset())

Output:

  category  scaled_index
0        B             0
1        A             5
2        C            10
3        A            15
4        B            20
...

Validating Your Plugin

Data Designer provides a testing utility to validate that your plugin is structured correctly. Use assert_valid_plugin to check that your config and implementation classes are properly defined:

1
from data_designer.engine.testing.utils import assert_valid_plugin
2
from data_designer_index_multiplier.plugin import plugin
3
4
# Raises AssertionError with a descriptive message if anything is wrong with the general plugin structure
5
assert_valid_plugin(plugin)

This validates that:

• The config class is a subclass of ConfigBase
• For column generator plugins: the implementation class is a subclass of ConfigurableTask
• For seed reader plugins: the implementation class is a subclass of SeedReader

Multiple Plugins in One Package

A single Python package can register multiple plugins. Simply define multiple Plugin instances and register each one as a separate entry point:

1
[project.entry-points."data_designer.plugins"]
2
my-column-generator = "my_package.plugins.column_generator.plugin:column_generator_plugin"
3
my-seed-reader = "my_package.plugins.seed_reader.plugin:seed_reader_plugin"

For an example of this pattern, see the end-to-end test plugins in the tests_e2e/ directory.

That’s it! You now know how to create a Data Designer plugin. A local editable install (uv pip install -e .) is all you need to develop, test, and use your plugin. If you want to make it available for others to install via pip install, publish it to PyPI or your organization’s package index.