aiq.eval.dataset_handler.dataset_filter#
Classes#
Apply allowlist and denylist filters to the DataFrame based on specified column filters. |
Module Contents#
- class DatasetFilter(
- filter_config: aiq.data_models.dataset_handler.EvalFilterConfig,
- Apply allowlist and denylist filters to the DataFrame based on specified column filters.
If a allowlist is provided, only keep rows matching the filter values.
If a denylist is provided, remove rows matching the filter values.
If the filter column does not exist in the DataFrame, the filtering is skipped for that column.
This is a utility class that is dataset agnostic and can be used to filter any DataFrame based on the provided filter configuration.
- filter_config#
- apply_filters(df) pandas.DataFrame#