nemo_gym.orchestration.executors.base

Module Contents

Classes

Name	Description
`BaseExecutor`	-

API

class nemo_gym.orchestration.executors.base.BaseExecutor()

Abstract

nemo_gym.orchestration.executors.base.BaseExecutor.persist(
    record: nemo_gym.orchestration.jobs.SubmissionRecord,
    config: nemo_gym.orchestration.api.SubmitConfig,
    write_manifest: collections.abc.Callable[[Path, str], None]
) -> None

Store the record and the resolved config, in the order that survives a partial failure.

Shared by every executor because the ordering and the failure handling are policy rather than transport. The machine-local index goes first precisely because it cannot fail the submit, so if the manifest write does fail there is still a parseable record for the by-hand recovery the error asks for.

The resolved config is written here, not by each executor’s staging step, so that every executor — today Slurm, tomorrow k8s or a local runner — gets it for free from the one place all of them already call.

Only the files’ transport differs per executor, so it arrives as write_manifest — Connection.write_text already has this signature — rather than this class owning a connection it cannot know how to open. Call it while that transport is still open: reopening one here would pay a second connection on every submit.

The jobs are queued by the time this runs, so a failure has to name them or they are stranded with no record anywhere.

nemo_gym.orchestration.executors.base.BaseExecutor.run(
    config: nemo_gym.orchestration.api.SubmitConfig,
    dry_run: bool = False
) -> nemo_gym.orchestration.jobs.SubmissionRecord | None

abstract

Submit config and return the record describing it.

Returns None on a dry run, which renders the scripts and stops before anything is submitted; every other path either returns a record or raises.