nemo_curator.config.run

Module Contents

Functions

Name	Description
`_instantiate_stage`	Instantiate a single stage from its Hydra config.
`create_pipeline_from_yaml`	-
`create_ray_client_from_yaml`	-
`main`	-

API

nemo_curator.config.run._instantiate_stage(
    stage_cfg: omegaconf.DictConfig
) -> typing.Any

Instantiate a single stage from its Hydra config.

Extracts resources before calling hydra.utils.instantiate (it is applied via .with_(), not as a constructor argument) and re-applies it after construction. batch_size is left in the config dict so that stages declaring it as a dataclass field receive it during construction.

nemo_curator.config.run.create_pipeline_from_yaml(
    cfg: omegaconf.DictConfig
) -> nemo_curator.pipeline.Pipeline | typing.Any

nemo_curator.config.run.create_ray_client_from_yaml(
    cfg: omegaconf.DictConfig
) -> nemo_curator.core.client.RayClient

nemo_curator.config.run.main(
    cfg: omegaconf.DictConfig
) -> None