nemo_gym.token_id_capture.terminal

View as Markdown

Attribute a finished rollout’s verified terminal model call.

Every agent’s /run result carries response: the object the verifier scored. Capture, by contrast, records every call the model server served, including auxiliary calls, abandoned retries, and sub-agent branches. Terminal attribution joins the verified response to exactly one captured call, so the builder can deliver the chain that earned the reward instead of guessing among chains or masking a healthy rollout.

The join is independent witnesses with corroboration, not a trust hierarchy. The fact “which call was kept” is emitted in different places by different agent classes, so up to three witnesses testify:

explicit — the caller names the kept call directly (a gate seal or an agent-declared terminal id). response_id — the /run response’s id equals the served id recorded on exactly one entry. Possession of the id proves which response the client actually received. content — the fingerprint of the response’s model-authored items matches one entry. Three readings are pooled: the entry’s cumulative continuation_fingerprint when a lineage-aware writer recorded one (a full-transcript response), the fingerprint of the entry’s own output (a final-turn-only response), and the transcript’s trailing model-authored block (a merged multi-turn transcript).

Each witness abstains rather than guesses (ambiguity inside a witness is an abstention, not a vote). The verdict then follows the stack’s rule that claims are verified, never ranked: witnesses that agree — or that name calls with identical full token sequences — attribute; witnesses that contradict each other attribute nothing and persist the disagreement, because a contradiction is evidence of a real defect (a stale seal mapping, backend id reuse, a transcript-synthesis bug) that outranking would silently bury. A rollout with no witness, or with disagreeing witnesses, falls back to the builder’s strict single-chain policy.

The content witness deliberately uses assistant_fingerprint alone, without a request-context digest. Attribution selects a chain whose tokens the builder verifies independently; it never reuses tokens across the matched boundary. Requiring context verification would spuriously refuse synthesized transcripts that reformat tool output, without adding safety.

Module Contents

Classes

NameDescription
TerminalAttributionThe joined terminal call, or the reasons no witness could name one.

Functions

NameDescription
_collapse_identicalReduce candidates that carry the same full token sequence to one.
_content_witnessThe content witness: fingerprint the response’s model-authored items.
_sequence_identityIdentify an entry by its full token sequence.
resolve_terminalJoin the verified /run response to one captured model call.

API

class nemo_gym.token_id_capture.terminal.TerminalAttribution(
model_call_id: str | None,
method: str = '',
reason: str = ''
)
Dataclass

The joined terminal call, or the reasons no witness could name one.

attributed
bool
method
str = ''
model_call_id
str | None
reason
str = ''
nemo_gym.token_id_capture.terminal._collapse_identical(
candidates: list[nemo_gym.token_id_capture.records.TokenEntry]
) -> nemo_gym.token_id_capture.records.TokenEntry | None

Reduce candidates that carry the same full token sequence to one.

Identical retries produce entries whose sequences match; any of them yields the same delivered chain, so the smallest call id wins deterministically. Candidates with different sequences are genuinely ambiguous and collapse to None.

nemo_gym.token_id_capture.terminal._content_witness(
entries: list[nemo_gym.token_id_capture.records.TokenEntry],
response: dict,
reasons: list[str]
) -> tuple[str, nemo_gym.token_id_capture.records.TokenEntry] | None

The content witness: fingerprint the response’s model-authored items.

Three readings of one response are possible and must compete, not race. A full transcript matches an entry’s cumulative fingerprint (the model-authored spine of request context + that call’s output); a final-turn-only response matches the fingerprint of one entry’s own output; a merged transcript’s trailing model-authored block matches the terminal call’s own output. The readings can name different calls — a first call’s cumulative fingerprint IS its own-output fingerprint, because non-model turns never contribute — so candidates from all keys pool before the ambiguity decision.

nemo_gym.token_id_capture.terminal._sequence_identity(
entry: nemo_gym.token_id_capture.records.TokenEntry
) -> tuple

Identify an entry by its full token sequence.

A lineage-aware writer records a cumulative digest and length. Records without one compare their token arrays directly. Both identify the delivered sequence, which is what training consumes.

nemo_gym.token_id_capture.terminal.resolve_terminal(
entries: list[nemo_gym.token_id_capture.records.TokenEntry],
response: dict | None,
explicit_call_id: str | None = None
) -> nemo_gym.token_id_capture.terminal.TerminalAttribution

Join the verified /run response to one captured model call.

entries is the frozen snapshot. response is the result’s scored response object (or None when the record carries none). This function never raises: malformed content is an abstention, not an error.