nemo_gym.dataset_metrics

Dependency-light, environment-owned dataset metric hooks.

Module Contents

Classes

Name	Description
`DatasetMetricsHookError`	An environment’s dataset metrics hook does not satisfy the protocol.

Functions

Name	Description
`load_dataset_metrics_hook`	Load an owning server’s optional dataset-metrics hook once.

Data

DATASET_METRICS_EXPORT_NAME

DATASET_METRICS_MODULE_NAME

DatasetMetricHook

DatasetMetricScalar

DatasetMetricValue

API

class nemo_gym.dataset_metrics.DatasetMetricsHookError()

Exception

Bases: Exception

An environment’s dataset metrics hook does not satisfy the protocol.

nemo_gym.dataset_metrics.load_dataset_metrics_hook(
    server_dir: pathlib.Path
) -> nemo_gym.dataset_metrics.DatasetMetricHook | None

Load an owning server’s optional dataset-metrics hook once.

The server may export compute_task_metrics(task_input) from dataset_metrics.py. It receives one materialized row’s task_input mapping and returns a mapping from display names to scalar values, lists of scalar values, or None. Numeric values produce numeric summaries; strings produce bounded categorical summaries.

nemo_gym.dataset_metrics.DATASET_METRICS_EXPORT_NAME = 'compute_task_metrics'

nemo_gym.dataset_metrics.DATASET_METRICS_MODULE_NAME = 'dataset_metrics'

nemo_gym.dataset_metrics.DatasetMetricHook: TypeAlias = Callable[[Mapping[str, object]], Mapping[str, DatasetMetricValue]]

nemo_gym.dataset_metrics.DatasetMetricScalar: TypeAlias = bool | int | float | str

nemo_gym.dataset_metrics.DatasetMetricValue: TypeAlias = DatasetMetricScalar | list[DatasetMetricScalar] | None