nemo_gym.tasks.harbor.cli

gym eval run <folder | harbor:dataset> --agent <name>: prepare a taskset and run it.

Preparation writes, under results/harbor/<taskset>/:

tasks.jsonl: one materialized row per loadable task;
skipped.json: omitted task IDs and reasons (empty when all tasks load);
run_config.yaml: the resources server, agent and environment server blocks.

Then the ordinary end-to-end rollout collection runs with that config plus the selected agent, environment server and model configs.

Module Contents

Classes

Name	Description
`AgentSelection`	-
`PreparedTaskset`	-

Functions

Name	Description
`_config_path`	-
`_has_override`	-
`build_run`	Write `run_config.yaml` and return it with the Hydra overrides for collection.
`prepare_target`	Fetch (for hub references) and load the tasks, then write the rows file.
`resolve_agent`	Map `--agent NAME[/FLAVOR]` to `responses_api_agents/<NAME>/configs/<FLAVOR>.yaml`.
`resolve_sandbox_config`	-
`run_target`	Entry point for `gym eval run <target> ...`.

Data

AGENT_OVERRIDES

DEFAULT_OUTPUT_ROOT

ENVIRONMENT_SERVER_CONFIG

SANDBOX_PROVIDER_CONFIG

logger

API

class nemo_gym.tasks.harbor.cli.AgentSelection(
    config_path: pathlib.Path,
    instance_name: str,
    impl_name: str
)

Dataclass

config_path

Path

impl_name

str

instance_name

str

class nemo_gym.tasks.harbor.cli.PreparedTaskset(
    taskset: str,
    folder: pathlib.Path,
    tasks: list[nemo_gym.tasks.harbor.task.HarborTask],
    rows_path: pathlib.Path,
    output_dir: pathlib.Path,
    skipped: dict[str, str] = dict()
)

Dataclass

folder

Path

output_dir

Path

rows_path

Path

skipped

dict[str, str] = field(default_factory=dict)

tasks

list[HarborTask]

taskset

str

nemo_gym.tasks.harbor.cli._config_path(
    relative: str
) -> pathlib.Path

nemo_gym.tasks.harbor.cli._has_override(
    overrides: list[str],
    key: str
) -> bool

nemo_gym.tasks.harbor.cli.build_run(
    prepared: nemo_gym.tasks.harbor.cli.PreparedTaskset,
    agent: nemo_gym.tasks.harbor.cli.AgentSelection,
    sandbox: str | None,
    overrides: list[str]
) -> tuple[pathlib.Path, list[str]]

Write run_config.yaml and return it with the Hydra overrides for collection.

nemo_gym.tasks.harbor.cli.prepare_target(
    target: str,
    output_root: pathlib.Path = DEFAULT_OUTPUT_ROOT,
    refresh_registry: bool = False
) -> nemo_gym.tasks.harbor.cli.PreparedTaskset

Fetch (for hub references) and load the tasks, then write the rows file.

A task folder that does not load is skipped with a warning (and listed in PreparedTaskset.skipped) so the rest of the dataset still prepares.

nemo_gym.tasks.harbor.cli.resolve_agent(
    agent: str
) -> nemo_gym.tasks.harbor.cli.AgentSelection

Map --agent NAME[/FLAVOR] to responses_api_agents/<NAME>/configs/<FLAVOR>.yaml.

NAME may omit a trailing _agent (hermes selects hermes_agent).

nemo_gym.tasks.harbor.cli.resolve_sandbox_config(
    provider: str
) -> pathlib.Path

nemo_gym.tasks.harbor.cli.run_target(
    args: argparse.Namespace,
    overrides: list[str]
) -> None

Entry point for gym eval run <target> ....

nemo_gym.tasks.harbor.cli.AGENT_OVERRIDES: dict[str, dict[str, Any]] = {'hermes_agent': {'sandbox_provider': None, 'enabled_toolsets': ['terminal'], 'c...

nemo_gym.tasks.harbor.cli.DEFAULT_OUTPUT_ROOT = Path('results/harbor')

nemo_gym.tasks.harbor.cli.ENVIRONMENT_SERVER_CONFIG = 'environment_servers/single_agent_turn/configs/single_agent_turn.yaml'

nemo_gym.tasks.harbor.cli.SANDBOX_PROVIDER_CONFIG = 'nemo_gym/sandbox/providers/{provider}/configs/{provider}.yaml'

nemo_gym.tasks.harbor.cli.logger = logging.getLogger(__name__)