Skip to navigation

nemo_gym.models.token_capture.lineage_scope

View as MarkdownOpen in Claude

Scope an episode’s captured calls by agent instance and model server, and attribute failures to them.

A lineage scope is the calls of one agent instance on one model server. Parent search, retry grouping and terminal selection never cross a lineage scope, so a judge whose prompt embeds the assistant’s transcript never continues the assistant’s chain, and two agent instances with identical first prompts are never retries of each other. The calls of a single-agent episode form one lineage scope.

Outcomes are per agent instance: a failure or a pending call belongs to the agent instance that made the call. A failure is episode-wide when nothing about its identity is known, or when the episode names its agent instances and the failure names none. Delivery and the manifest both use these helpers, so they agree on every attribution.

Module Contents

Classes

NameDescription
OutcomeAttributionWhich agent instances an episode’s failures and pending calls belong to.

Functions

NameDescription
by_agent_instancePartition items by key(item)[0], the agent instance of their lineage scope, in first-appearance order.
entries_by_lineage_scopePartition entries by lineage scope, in first-appearance order.
entry_lineage_scope-
inherited_identity_columnsReturn each ledger row’s identity columns, inheriting them from its call’s intent row when it has none.
is_episode_wideWhether a failed or lost call applies to the whole episode rather than to one agent instance.
lineage_scope_keyThe lineage scope of a call made with identity.
primary_agent_instanceThe agent instance whose result is the episode’s top-level result.
row_lineage_scopeThe lineage scope of a ledger row, or of a wire record such as CallRecord or ManifestFailure.
same_agent_other_serverWhether other is the same agent instance served by a different model server.

Data

LineageScopeKey

PRIMARY_AGENT_INSTANCE

UNNAMED_AGENT_INSTANCE

_T

API

class nemo_gym.models.token_capture.lineage_scope.OutcomeAttribution()

Which agent instances an episode’s failures and pending calls belong to.

agent_instances holds every agent instance with its own failure or pending call; the unnamed agent is None. episode_wide is set by a failure with no known identity, or with no agent instance in an episode that names them: it masks every agent instance.

agent_instances
set[str | None] = set()
nemo_gym.models.token_capture.lineage_scope.OutcomeAttribution.add(
identity_columns: collections.abc.Mapping[str, typing.Any],
named_episode: bool
) -> None

Attribute one failed or pending call.

In an episode that names its agent instances, a call without a name cannot be placed and is episode-wide. In a single-agent episode it belongs to the unnamed agent.

nemo_gym.models.token_capture.lineage_scope.by_agent_instance(
items: collections.abc.Iterable[nemo_gym.models.token_capture.lineage_scope._T],

Partition items by key(item)[0], the agent instance of their lineage scope, in first-appearance order.

nemo_gym.models.token_capture.lineage_scope.entries_by_lineage_scope(
entries: collections.abc.Iterable[nemo_gym.models.token_capture.records.TokenEntry]

Partition entries by lineage scope, in first-appearance order.

nemo_gym.models.token_capture.lineage_scope.entry_lineage_scope(
nemo_gym.models.token_capture.lineage_scope.inherited_identity_columns(
rows: list[collections.abc.Mapping[str, typing.Any]]
) -> list[dict[str, typing.Any]]

Return each ledger row’s identity columns, inheriting them from its call’s intent row when it has none.

A row whose call has no named agent instance, on the row or on its intent, keeps whatever identity it has. is_episode_wide decides from those columns whether the row applies to the whole episode: it does when they are empty, or when the episode names its agent instances and they name none.

nemo_gym.models.token_capture.lineage_scope.is_episode_wide(
identity_columns: collections.abc.Mapping[str, typing.Any],
named_episode: bool
) -> bool

Whether a failed or lost call applies to the whole episode rather than to one agent instance.

It does when nothing about its identity is known, or when the episode names its agent instances but this call’s agent instance is unknown. A mark that names no model call, which the model server writes for a request it could not classify, belongs to the agent instance that made the request when that is known.

nemo_gym.models.token_capture.lineage_scope.lineage_scope_key(

The lineage scope of a call made with identity.

nemo_gym.models.token_capture.lineage_scope.primary_agent_instance(
agent_instances: collections.abc.Iterable[str | None]
) -> str | None

The agent instance whose result is the episode’s top-level result.

assistant when the episode has one; otherwise the unnamed agent; otherwise the only agent instance. assistant comes first, so an unnamed caller beside it, such as a helper or a judge that does not name itself, never takes the scored response. Raise LookupError when none of these exists.

nemo_gym.models.token_capture.lineage_scope.row_lineage_scope(
row: collections.abc.Mapping[str, typing.Any] | object

The lineage scope of a ledger row, or of a wire record such as CallRecord or ManifestFailure.

nemo_gym.models.token_capture.lineage_scope.same_agent_other_server(
) -> bool

Whether other is the same agent instance served by a different model server.

nemo_gym.models.token_capture.lineage_scope.LineageScopeKey = tuple[str | None, str | None]
nemo_gym.models.token_capture.lineage_scope.PRIMARY_AGENT_INSTANCE = 'assistant'
nemo_gym.models.token_capture.lineage_scope.UNNAMED_AGENT_INSTANCE: str | None = None
nemo_gym.models.token_capture.lineage_scope._T = TypeVar('_T')