nemo_curator.stages.interleaved.utils.validation_utils

Module Contents

Functions

Name	Description
`require_source_id_field`	-
`resolve_storage_options`	-
`validate_and_project_source_fields`	Validate requested source `fields` and normalize selected values for tabular output.

API

nemo_curator.stages.interleaved.utils.validation_utils.require_source_id_field(
    source_id_field: str
) -> str

nemo_curator.stages.interleaved.utils.validation_utils.resolve_storage_options(
    task: nemo_curator.tasks.Task[typing.Any] | None = None,
    io_kwargs: dict[str, object] | None = None
) -> dict[str, object]

nemo_curator.stages.interleaved.utils.validation_utils.validate_and_project_source_fields(
    sample: dict[str, typing.Any],
    fields: tuple[str, ...] | None,
    excluded_fields: set[str]
) -> dict[str, typing.Any]

Validate requested source fields and normalize selected values for tabular output.