nemo_gym.artifacts.exporters.evalrdl

Module Contents

Classes

Name	Description
`EvalRDLExportConfig`	Configuration for `gym eval export evalrdl`.
`EvalRDLExporter`	Upload a finished run to EvalRDL.

Functions

Name	Description
`_without_resume_identity`	The run directory to upload: as it is, or, when it holds a resume identity anywhere below it, a mirror without

Data

RESUME_IDENTITY_SUFFIX

UPLOAD_ALLOWANCE_S

API

class nemo_gym.artifacts.exporters.evalrdl.EvalRDLExportConfig()

Bases: RunExportConfig

Configuration for gym eval export evalrdl.

Credentials are not settings here: the EvalRDL SDK reads its token from $EVALRDL_TOKEN, $GITLAB_TOKEN or $GITLAB_PRIVATE_TOKEN, then from glab.

label

Optional[str]

tags

list[str]

timeout

float

url

Optional[str]

wait

bool

nemo_gym.artifacts.exporters.evalrdl.EvalRDLExportConfig.plain_containers(
    value: typing.Any
) -> typing.Any

classmethod

class nemo_gym.artifacts.exporters.evalrdl.EvalRDLExporter()

Bases: IsolatedExporter

Upload a finished run to EvalRDL.

The evalrdl SDK lives in the exporter’s own environment. Install it from the index your organization provides by configuring uv’s package index settings for the export command; this exporter does not know one. The upload is deduplicated by EvalRDL, so exporting the same run again reports skipped.

config

EvalRDLExportConfig

config_model

type[RunExportConfig] = EvalRDLExportConfig

name

str = 'evalrdl'

worker_fpath

Path = Path(__file__).with_name('_evalrdl_worker.py')

nemo_gym.artifacts.exporters.evalrdl.EvalRDLExporter.build_payload(
    artifacts: nemo_gym.artifacts.run_artifacts.RunArtifacts,
    workdir: pathlib.Path
) -> dict[str, typing.Any]

Describe the upload for the worker: the directory the rollouts are in, uploaded as it is.

Everything in that directory is part of the run (the rollouts and their sidecars, health reports, model-call captures, benchmark-specific files), so nothing is copied; only the resume identity beside the run manifest is left out, through a mirror of links (see _without_resume_identity). The saved config and the aggregate metrics are still required first, as Gym’s own rule for a complete run.

nemo_gym.artifacts.exporters.evalrdl.EvalRDLExporter.required_packages() -> tuple[str, ...]

nemo_gym.artifacts.exporters.evalrdl.EvalRDLExporter.worker_timeout_s(
    payload: dict[str, typing.Any]
) -> float

The wait for the verdict (when waiting) plus an allowance for the upload itself, which timeout excludes.

nemo_gym.artifacts.exporters.evalrdl._without_resume_identity(
    run_dir: pathlib.Path,
    workdir: pathlib.Path
) -> pathlib.Path

The run directory to upload: as it is, or, when it holds a resume identity anywhere below it, a mirror without any such file.

The mirror, under workdir and named like the run directory, has the run’s directories and a link to each of its files, so nothing is copied and the run directory is left as it is.

nemo_gym.artifacts.exporters.evalrdl.RESUME_IDENTITY_SUFFIX = 'manifest.identity.json'

nemo_gym.artifacts.exporters.evalrdl.UPLOAD_ALLOWANCE_S = 3600