`nemo_rl.data.energon.config`#

Module Contents#

Classes#

`EnergonSourceConfig`	One prepared Energon dataset split.
`EnergonPackingOptions`	Options for task-encoder-owned sequence packing.
`EnergonPackingConfig`	One task-encoder-owned packing implementation.
`_EnergonComponentConfig`	Shared selection fields for built-in and file-backed Energon plugins.
`EnergonTaskEncoderConfig`	One built-in or file-backed task encoder and optional packing.
`EnergonCookerConfig`	One built-in or file-backed source cooker.
`EnergonLoaderConfig`	Shared Energon settings for driver- and worker-owned SFT loaders.

API#

class nemo_rl.data.energon.config.EnergonSourceConfig#

Bases: pydantic.BaseModel

One prepared Energon dataset split.

path: str#: None

split: str#: None

virtual_epoch_length: Annotated[int, Field(ge=0)]#: 0

limit: Annotated[int, Field(ge=1)] | None#: None

class nemo_rl.data.energon.config.EnergonPackingOptions#

Bases: pydantic.BaseModel

Options for task-encoder-owned sequence packing.

max_sequences_per_bin: Annotated[int, Field(ge=1)] | None#: None

balanced_knapsack_delta: Annotated[int, Field(ge=0)] | None#: None

class nemo_rl.data.energon.config.EnergonPackingConfig#

Bases: pydantic.BaseModel

One task-encoder-owned packing implementation.

name: str#: None

buffer_size: Annotated[int, Field(ge=1)]#: None

options: nemo_rl.data.energon.config.EnergonPackingOptions#: None

class nemo_rl.data.energon.config._EnergonComponentConfig#

Bases: pydantic.BaseModel

Shared selection fields for built-in and file-backed Energon plugins.

_label: ClassVar[str]#: None

name: str | None#: None

python_file: str | None#: None

object_name: str | None#: None

options: dict[str, Any]#: ‘Field(…)’

classmethod _from_registry_key(value: Any) → Any#

_validate_component_reference() → Self#

class nemo_rl.data.energon.config.EnergonTaskEncoderConfig#

Bases: nemo_rl.data.energon.config._EnergonComponentConfig

One built-in or file-backed task encoder and optional packing.

_label: ClassVar[str]#: ‘Task encoder’

name: str | None#: ‘generic_sft’

packing: nemo_rl.data.energon.config.EnergonPackingConfig | None#: None

class nemo_rl.data.energon.config.EnergonCookerConfig#

Bases: nemo_rl.data.energon.config._EnergonComponentConfig

One built-in or file-backed source cooker.

_label: ClassVar[str]#: ‘Cooker’

name: str | None#: ‘generic_conversation’

has_subflavors: dict[str, str | int | float | bool | None] | None#: None

class nemo_rl.data.energon.config.EnergonLoaderConfig#

Bases: pydantic.BaseModel

Shared Energon settings for driver- and worker-owned SFT loaders.

model_family: nemo_rl.data.energon.multimodal.model_families.ModelFamily#: ‘Field(…)’

num_workers: Annotated[int, Field(ge=0)]#: 8

shuffle_buffer_size: Annotated[int, Field(ge=0)]#: 1000

max_samples_per_sequence: Annotated[int, Field(ge=1, description='Maximum sequential sample run used when sharding a dataset.')] | None#: None

packing_buffer_size: None#: None

batch_grouping: Literal[auto]#: ‘auto’

processor_adapter: Literal[hf_multimodal]#: ‘hf_multimodal’

topology_mapper: Literal[default]#: ‘default’

task_encoder: nemo_rl.data.energon.config.EnergonTaskEncoderConfig#: ‘Field(…)’

cookers: list[nemo_rl.data.energon.config.EnergonCookerConfig]#: ‘Field(…)’

seed_offset: int#: 0

prefetch_factor: Annotated[int, Field(ge=1)]#: 2

checkpoint_every_sec: Annotated[float, Field(gt=0)]#: 60.0

watchdog_timeout_seconds: Annotated[float, Field(gt=0)] | None#: 60.0

nvdataset_cache_dir: str | None#: ‘Field(…)’

cache_pool_max_gbytes: Annotated[float, Field(gt=0)] | None#: ‘Field(…)’

cache_pool_num_workers: Annotated[int, Field(ge=1)] | None#: ‘Field(…)’

gc_collect_every_n_steps: Annotated[int, Field(ge=0)] | None#: ‘Field(…)’

nemo_rl.data.energon.config#

Module Contents#

Classes#

API#

`nemo_rl.data.energon.config`#