nemo_gym.rollout_compare.join

How rows on the two sides are matched to the same task.

Key detection by vote across rows, the taskset and task id behind a Harbor row’s task id, grouping rows by task with anonymous sidecar rows kept apart, stripping a namespace prefix one side carries and the other does not, and matching plain ids to keyed labels while reporting the ambiguous ones.

Module Contents

Functions

Name	Description
`_common_prefix`	A `<prefix>/` every id shares (e.g. `terminal-bench/`), or None.
`_has_any_key`	-
`_join_sides`	Strip a `<prefix>/` namespace from whichever side carries one when that is what keeps the ids apart:
`_lookup`	-
`_match_plain_to_keyed`	Join a side of plain task ids to one keyed `taskset/task_id`. A keyed group is relabelled with its plain
`_names_a_task`	Whether a row has a say in key detection: a sidecar failure that names no task at all does not.
`_strip_prefix`	-
`_task_identity`	(taskset, task id) named by `key`; the taskset is only known for a dict `_ng_task_id`.
`_unkeyed_sidecar`	-
`detect_task_key`	The first candidate field every row carries. Sidecar failure rows that name no task at all are left out
`group_by_task`	(rows grouped by task label, anonymous rows). An anonymous row is a sidecar failure that names no task:
`has_keyed_rows`	Whether a side names any task at all. A side that is empty, or holds only anonymous sidecar failures, has
`task_id_of`	The label a row is joined and listed under: the plain task id, or `taskset/task_id` with `with_taskset`
`tasksets_of`	-

Data

Groups

API

nemo_gym.rollout_compare.join._common_prefix(
    ids: list[str]
) -> typing.Optional[str]

A <prefix>/ every id shares (e.g. terminal-bench/), or None.

nemo_gym.rollout_compare.join._has_any_key(
    row: dict[str, typing.Any]
) -> bool

nemo_gym.rollout_compare.join._join_sides(
    old: dict[str, nemo_gym.rollout_compare.schema._T],
    new: dict[str, nemo_gym.rollout_compare.schema._T]
) -> tuple[dict[str, nemo_gym.rollout_compare.schema._T], dict[str, nemo_gym.rollout_compare.schema._T], typing.Optional[str], typing.Optional[str]]

Strip a <prefix>/ namespace from whichever side carries one when that is what keeps the ids apart: legacy servers often namespace the task (terminal-bench/x) where the Harbor path does not.

nemo_gym.rollout_compare.join._lookup(
    row: dict[str, typing.Any],
    dotted: str
) -> typing.Any

nemo_gym.rollout_compare.join._match_plain_to_keyed(
    plain: nemo_gym.rollout_compare.join.Groups,
    plain_key: str,
    keyed: nemo_gym.rollout_compare.join.Groups,
    keyed_key: str
) -> tuple[nemo_gym.rollout_compare.join.Groups, nemo_gym.rollout_compare.join.Groups, typing.Optional[str], typing.Optional[str], dict[str, list[str]]]

Join a side of plain task ids to one keyed taskset/task_id. A keyed group is relabelled with its plain task id when that id lives in exactly one taskset (or in the plain side’s own taskset); an id the plain side carries that recurs across tasksets is ambiguous: its groups keep their taskset labels and are reported. Returns the two sides, the prefix stripped from each, and the ambiguous ids with their taskset labels.

nemo_gym.rollout_compare.join._names_a_task(
    row: dict[str, typing.Any]
) -> bool

Whether a row has a say in key detection: a sidecar failure that names no task at all does not.

nemo_gym.rollout_compare.join._strip_prefix(
    grouped: dict[str, nemo_gym.rollout_compare.schema._T],
    prefix: str
) -> dict[str, nemo_gym.rollout_compare.schema._T]

nemo_gym.rollout_compare.join._task_identity(
    row: dict[str, typing.Any],
    key: str
) -> tuple[typing.Optional[str], str]

(taskset, task id) named by key; the taskset is only known for a dict _ng_task_id.

nemo_gym.rollout_compare.join._unkeyed_sidecar(
    row: dict[str, typing.Any],
    key: str
) -> bool

nemo_gym.rollout_compare.detect_task_key(
    rows: list[dict[str, typing.Any]]
) -> str

The first candidate field every row carries. Sidecar failure rows that name no task at all are left out of the vote; they cannot be joined and are listed as masked under an unknown (<side>) #<n> label.

nemo_gym.rollout_compare.group_by_task(
    rows: list[dict[str, typing.Any]],
    key: str,
    with_taskset: bool = False
) -> tuple[dict[str, list[dict[str, typing.Any]]], list[dict[str, typing.Any]]]

(rows grouped by task label, anonymous rows). An anonymous row is a sidecar failure that names no task: it is kept out of the join entirely and listed as masked on its own side.

nemo_gym.rollout_compare.join.has_keyed_rows(
    rows: list[dict[str, typing.Any]]
) -> bool

Whether a side names any task at all. A side that is empty, or holds only anonymous sidecar failures, has no key of its own to detect.

nemo_gym.rollout_compare.task_id_of(
    row: dict[str, typing.Any],
    key: str,
    with_taskset: bool = False
) -> str

The label a row is joined and listed under: the plain task id, or taskset/task_id with with_taskset (used when a side spans several tasksets, where the same task id may recur).

nemo_gym.rollout_compare.tasksets_of(
    rows: list[dict[str, typing.Any]],
    key: str
) -> set[str]

nemo_gym.rollout_compare.join.Groups = dict[str, list[dict[str, Any]]]