Skip to navigation
API ReferenceFull Library ReferenceNemo GymNemo GymTrajectory

nemo_gym.trajectory.trajectory_record

View as Markdown

Shared contracts for rollout observations and trajectories.

Module Contents

Classes

NameDescription
AgentEpisodeAn Agent response and the observations available at its execution boundary.
AgentInvocationOne agent activation or subagent invocation observed by a harness.
AgentObservationBundleNormalized observations returned by one Agent Server for one rollout.
ContextCompactionObservationAn explicit context-compaction event reported by the Agent harness.
GradingOperationObservationA source-reviewed grading calculation actually executed inside a verifier invocation.
ModelCallRefStable identifiers an Agent integration can observe for one model call.
ObservationGapA fact that the selected integration could not observe or join exactly.
ObservationModel-
OutputBudgetRefitA model call the server sent again with its output budget lowered, because the budget it requested exceeded
SandboxObservationOutcome and lifetime resource usage reported by a sandbox-owning harness.
ToolCallObservationRaw tool evidence; span and parent IDs are local to its producer until imported.
TrajectoryModelCall-
TrajectoryRecord-
TrajectoryResponseMetadata-
TrajectoryTokenStats-
TrajectoryToolCallCanonical tool attempt with qualified span IDs and model-visible output.
TrajectoryTurn-
VerifierGradingRecordThe grade a verifier returned without asking any model, recorded by the shared verify path.
VerifierInvocationOne verify() execution, its outcome, and all model attempts made while grading.

Functions

NameDescription
exclude_verifier_model_callsExclude explicitly verifier-owned attempts from agent and step joins.
failure_observationsKeep the existing typed observation slots across an unsuccessful HTTP reply.
grading_result_bindingBind the actual graded response and numeric reward/components, without exporting their payload.
join_agent_model_callsJoin captured calls to agent invocations and their compactions without guessing ownership.
join_assistant_message_callsJoin persisted messages to HTTP attempts using session, message and model-server IDs.
tool_parent_cyclesReturn only cycle members in the same-instance tool-parent relation.

Data

AgentObservationRecord

NG_AGENT_OBSERVATIONS_KEY

NG_AGENT_SANDBOX_OBSERVATION_KEY

NG_TRAJECTORY_KEY

NG_VERIFIER_INVOCATIONS_KEY

API

class nemo_gym.trajectory.trajectory_record.AgentEpisode(
    response: nemo_gym.core.responses_api_types.NeMoGymResponse,
    observations: nemo_gym.trajectory.trajectory_record.AgentObservationBundle
)
Dataclass

An Agent response and the observations available at its execution boundary.

observations
AgentObservationBundle
response
NeMoGymResponse
class nemo_gym.trajectory.trajectory_record.AgentInvocation()

Bases: ObservationModel

One agent activation or subagent invocation observed by a harness.

A harness’s saved session may span several invocations; harness_session_id retains that context while invocation_id owns each activation’s evidence.

Harness adapters should report an iteration-limit event through an on_iteration_limit_reached(*, invocation_id: str) -> None observer hook. The hook records stop_reason="max_iterations" once on that invocation, leaving parent invocations, budgets, and summary generation alone. Harnesses with native stop events can call it directly; compatibility shims may call it at the point where the harness detects its limit.

activation_id
Optional[int]
agent_instance
Optional[str] = Field(default=None, min_length=1)
conversation
list[NeMoGymResponseInputItem]
duration_ms
Optional[float] = Field(default=None, ge=0)
error_type
Optional[str] = None
harness_session_id
Optional[str]
in_session
bool | None = Field(default=None, strict=True)
invocation_id
str
kind
Literal['agent_invocation'] = 'agent_invocation'
model_calls
list[ModelCallRef] = Field(default_factory=list)
parent_invocation_id
Optional[str] = None
spawned_by_tool_call_id
Optional[str] = None
status
Literal['completed', 'failed', 'incomplete', 'unknown']
stop_reason
Optional[str]
verification_skipped
bool | None = Field(default=None, strict=True)
nemo_gym.trajectory.trajectory_record.AgentInvocation._serialize_optional_metadata(
    handler: pydantic.SerializerFunctionWrapHandler
)
class nemo_gym.trajectory.trajectory_record.AgentObservationBundle()

Bases: ObservationModel

Normalized observations returned by one Agent Server for one rollout.

gaps
list[ObservationGap] = Field(default_factory=list)
records
list[AgentObservationRecord]
schema_version
Literal['1.0', '2.0'] = '1.0'
source
str
nemo_gym.trajectory.trajectory_record.AgentObservationBundle.decode_version(
    value: typing.Any
) -> typing.Any
classmethod
nemo_gym.trajectory.trajectory_record.AgentObservationBundle.serialize_version(
    handler: pydantic.SerializerFunctionWrapHandler
)
nemo_gym.trajectory.trajectory_record.AgentObservationBundle.validate_identity() -> 'AgentObservationBundle'

Require unique invocation IDs and reject cycles in the observed parent graph.

class nemo_gym.trajectory.trajectory_record.ContextCompactionObservation()

Bases: ObservationModel

An explicit context-compaction event reported by the Agent harness.

after_model_call
Optional[ModelCallRef]
before_model_call
Optional[ModelCallRef]
first_kept_item_id
Optional[str] = None
invocation_id
str
kind
Literal['context_compaction'] = 'context_compaction'
model_calls
list[ModelCallRef]
observed_at
Optional[float] = None
outcome
Literal['completed', 'failed', 'aborted', 'unknown'] = 'unknown'
source_message_ids
list[str]
source_model_ref
Optional[ModelServerRef] = None
summary
Optional[str] = None
tokens_after
Optional[int]
tokens_before
Optional[int]
trigger
Optional[str] = None
class nemo_gym.trajectory.trajectory_record.GradingOperationObservation()

Bases: ObservationModel

A source-reviewed grading calculation actually executed inside a verifier invocation.

llm_judge grades from judge calls made through call_judge; its dependencies are those calls’ model and response ids, and it completes only when the reviewed call site marked an explicitly parsed verdict.

dependencies
list[ModelCallRef] | None = None
kind
Literal['programmatic_calculation', 'llm_judge'] = 'programmatic_calculation'
operation_id
str = Field(min_length=1)
outcome
Literal['completed', 'failed', 'unknown'] = 'unknown'
result_binding_sha256
str | None = Field(default=None, pattern='^[0-9a-f]{64}$')
class nemo_gym.trajectory.trajectory_record.ModelCallRef()

Bases: ObservationModel

Stable identifiers an Agent integration can observe for one model call.

model_call_id
Optional[str] = None
model_ref
Optional[ModelServerRef] = None
response_id
Optional[str] = None
nemo_gym.trajectory.trajectory_record.ModelCallRef.validate_join_key() -> 'ModelCallRef'
class nemo_gym.trajectory.trajectory_record.ObservationGap()

Bases: ObservationModel

A fact that the selected integration could not observe or join exactly.

agent_instance
Optional[str] = None
code
str
detail
Optional[str] = None
invocation_id
Optional[str] = None
span_id
Optional[str] = None
nemo_gym.trajectory.trajectory_record.ObservationGap.matches_tool(
    tool: nemo_gym.trajectory.trajectory_record.ToolCallObservation
) -> bool

A supplied owner constrains the span; missing ownership never selects between matches.

class nemo_gym.trajectory.trajectory_record.ObservationModel()

Bases: BaseModel

model_config
class nemo_gym.trajectory.trajectory_record.OutputBudgetRefit()

Bases: ObservationModel

A model call the server sent again with its output budget lowered, because the budget it requested exceeded the engine’s whole context window.

applied_max_tokens
int = Field(ge=1)
requested_max_tokens
int = Field(ge=1)
window
int = Field(ge=1)
class nemo_gym.trajectory.trajectory_record.SandboxObservation()

Bases: ObservationModel

Outcome and lifetime resource usage reported by a sandbox-owning harness.

cleanup_errors
list[str]
cpu_time_s
Optional[float]
error_message
Optional[str] = None
error_type
Optional[str] = None
exit_code
Optional[int] = None
failure_id
Optional[str]
kind
Literal['sandbox'] = 'sandbox'
observed_client_interval_s
Optional[float]
outcome
Literal['completed', 'failed', 'timeout', 'oom', 'sandbox_error', 'cancelled', 'unknown'] = 'unknown'
peak_memory_mib
Optional[float]
provider
Optional[str] = None
resource_usage_source
Optional[str] = None
role
Literal['agent', 'verifier', 'environment']
sandbox_id
Optional[str] = None
stage
FailureStage | None
teardown_confirmed
Optional[bool]
wall_time_s
Optional[float] = Field(default=None, ge=0)
class nemo_gym.trajectory.trajectory_record.ToolCallObservation()

Bases: ObservationModel

Raw tool evidence; span and parent IDs are local to its producer until imported.

agent_instance
Optional[str] = Field(default=None, min_length=1)
completed_at
Optional[float] = None
duration_ms
Optional[float] = Field(default=None, ge=0)
error_type
Optional[str] = None
failure_id
Optional[str] = None
invocation_id
str
kind
Literal['tool_call'] = 'tool_call'
model_call_id
Optional[str] = None
model_call_ref
Optional[ModelCallRef] = None
native_tool_call_id
Optional[str] = None
parent_span_id
Optional[str] = None
producer_id
Optional[str] = Field(default=None, min_length=1)
sandbox_id
Optional[str]
source_message_id
Optional[str] = None
source_model_ref
Optional[ModelServerRef] = None
span_id
Optional[str] = Field(default=None, min_length=1)
started_at
Optional[float] = None
status
Literal['completed', 'failed', 'timeout', 'cancelled', 'incomplete', 'unknown'] = 'unknown'
timing_source
Optional[Literal['executor', 'artifact', 'harness']] = None
timings
list[OperationTiming] = Field(default_factory=list)
tool_call_id
str
tool_name
Optional[str] = None
nemo_gym.trajectory.trajectory_record.ToolCallObservation.validate_timing() -> 'ToolCallObservation'
class nemo_gym.trajectory.trajectory_record.TrajectoryModelCall()

Bases: ObservationModel

agent_instance
Optional[str] = Field(default=None, min_length=1)
client_assistant_message_id
Optional[str] = None
client_session_id
Optional[str] = None
clock_domain
Optional[str] = Field(default=None, min_length=1)
completed_at
Optional[float] = None
duration_ms
Optional[float] = Field(default=None, ge=0)
model_call_id
Optional[str] = None
request
Optional[Any] = None
response
Optional[Any] = None
response_metadata
TrajectoryResponseMetadata = Field(default_factory=TrajectoryResponseMetadata)
started_at
Optional[float] = None
token_stats
TrajectoryTokenStats = Field(default_factory=TrajectoryTokenStats)
class nemo_gym.trajectory.trajectory_record.TrajectoryRecord()

Bases: ObservationModel

capture_key
str

The attempt-qualified episode identity (called rollout_id in v1).

failure_events
dict[str, dict[str, Any]] = Field(default_factory=dict)
gaps
list[ObservationGap] = Field(default_factory=list)
invocations
list[AgentInvocation] = Field(default_factory=list)
model_calls
list[TrajectoryModelCall] = Field(default_factory=list)
rollout_id
str
schema_version
Literal['1.0', '2.0'] = '1.0'
task_id
str
tool_calls
list[TrajectoryToolCall] = Field(default_factory=list)
turns
list[TrajectoryTurn] = Field(default_factory=list)
verifier_invocations
list[VerifierInvocation] = Field(default_factory=list)
nemo_gym.trajectory.trajectory_record.TrajectoryRecord.decode_identity(
    value: typing.Any
) -> typing.Any
classmethod
nemo_gym.trajectory.trajectory_record.TrajectoryRecord.serialize_version(
    handler: pydantic.SerializerFunctionWrapHandler,
    info: pydantic.SerializationInfo
)
nemo_gym.trajectory.trajectory_record.TrajectoryRecord.validate_identity() -> 'TrajectoryRecord'
class nemo_gym.trajectory.trajectory_record.TrajectoryResponseMetadata()

Bases: ObservationModel

applied_max_tokens
Optional[int] = Field(default=None, ge=1)
dialect
Optional[str] = None
error_category
Optional[str] = None
failure_report
FailureReport | None = None
finish_reason
Optional[str] = None
latency_ttft_ms
Optional[float] = Field(default=None, ge=0)
local_response_reason
Optional[str] = None
model
Optional[str] = None
model_operations
list[ModelOperationObservation] | None = None
model_ref
Optional[ModelServerRef] = None
output_budget_refit
Optional[OutputBudgetRefit] = None
requested_max_tokens
Optional[int] = Field(default=None, ge=1)
response_id
Optional[str] = None
response_source
Optional[Literal['upstream', 'local']] = None
response_status
Optional[str] = None
status_code
Optional[int] = None
upstream_attempted
Optional[bool] = None
upstream_status_code
Optional[int] = None
class nemo_gym.trajectory.trajectory_record.TrajectoryTokenStats()

Bases: ObservationModel

cached_tokens
Optional[int] = Field(default=None, ge=0)
completion_tokens
Optional[int] = Field(default=None, ge=0)
prompt_tokens
Optional[int] = Field(default=None, ge=0)
reasoning_tokens
Optional[int] = Field(default=None, ge=0)
total_tokens
Optional[int] = Field(default=None, ge=0)
class nemo_gym.trajectory.trajectory_record.TrajectoryToolCall()

Bases: ToolCallObservation

Canonical tool attempt with qualified span IDs and model-visible output.

Unlike raw ToolCallObservation inputs, these IDs have already crossed the import boundary and must remain unchanged on replay.

output
Optional[Any] = None
class nemo_gym.trajectory.trajectory_record.TrajectoryTurn()

Bases: ObservationModel

activation_id
Optional[int]
agent_instance
Optional[str] = Field(default=None, min_length=1)
answer
Optional[Any] = None
invocation_id
str
model_calls
list[ModelCallRef] = Field(default_factory=list)
question
Optional[Any] = None
reasoning_content
Optional[Any] = None
resolved
Optional[bool] = None
rollout_id
str
source_message_id
Optional[str]
source_model_ref
Optional[ModelServerRef]
step_count
int
task_id
str
timestamp
float
turn_no
int
nemo_gym.trajectory.trajectory_record.TrajectoryTurn._serialize_optional_metadata(
    handler: pydantic.SerializerFunctionWrapHandler
)
class nemo_gym.trajectory.trajectory_record.VerifierGradingRecord()

Bases: ObservationModel

The grade a verifier returned without asking any model, recorded by the shared verify path.

It names the verifier (its resources server), the score and the verifier’s verdict on it (masked when the verifier marked the score as no valid measurement), and binds the graded response and score like a grading operation. A verify execution that sent a model request records none: its grading is observed only through its captured calls.

result_binding_sha256
str = Field(pattern='^[0-9a-f]{64}$')
score
float
verdict
Literal['scored', 'masked']
verifier
str = Field(min_length=1)
class nemo_gym.trajectory.trajectory_record.VerifierInvocation()

Bases: ObservationModel

One verify() execution, its outcome, and all model attempts made while grading.

Captured calls establish ownership. The shared verify boundary records the execution outcome, including for programmatic verifiers with no model calls.

failure_kind
Optional[str] = None
grading_operations
list[GradingOperationObservation] | None
grading_record
VerifierGradingRecord | None
invocation_id
str
kind
Literal['verifier_invocation'] = 'verifier_invocation'
model_calls
list[ModelCallRef] = Field(default_factory=list)
sandbox_observations
list[SandboxObservation] | None
status
Literal['completed', 'failed', 'incomplete', 'unknown'] = 'unknown'
nemo_gym.trajectory.trajectory_record.VerifierInvocation.validate_failure_kind(
    value: typing.Optional[str]
) -> typing.Optional[str]
classmethod
nemo_gym.trajectory.trajectory_record.exclude_verifier_model_calls(
    calls: collections.abc.Iterable[nemo_gym.trajectory.model_call_record.ModelCallRecord],
    verifier_invocations: collections.abc.Iterable[nemo_gym.trajectory.trajectory_record.VerifierInvocation]
) -> list[nemo_gym.trajectory.model_call_record.ModelCallRecord]

Exclude explicitly verifier-owned attempts from agent and step joins.

nemo_gym.trajectory.trajectory_record.failure_observations(
    value: typing.Any
) -> dict[str, typing.Any]

Keep the existing typed observation slots across an unsuccessful HTTP reply.

Error strings are not searched recursively: only producer-owned top-level fields are accepted, and malformed observations remain unavailable.

nemo_gym.trajectory.trajectory_record.grading_result_binding(
    result: object
) -> str | None

Bind the actual graded response and numeric reward/components, without exporting their payload.

Join captured calls to agent invocations and their compactions without guessing ownership.

Join persisted messages to HTTP attempts using session, message and model-server IDs.

Run after invocation ownership resolution. Summary attempts belong to compactions; calls without a saved message (for example title generation) stay invocation-owned. Producers declare source_message_id(s), source_model_ref and invocation IDs. Source labels are not dispatch keys. No content or timing fallback is used.

nemo_gym.trajectory.trajectory_record.tool_parent_cycles(
    tools: collections.abc.Iterable[nemo_gym.trajectory.trajectory_record.ToolCallObservation]
) -> set[tuple[str | None, str]]

Return only cycle members in the same-instance tool-parent relation.

nemo_gym.trajectory.trajectory_record.AgentObservationRecord = Annotated[AgentInvocation | ToolCallObservation | ContextCompactionObservation |...
 
nemo_gym.trajectory.trajectory_record.NG_AGENT_OBSERVATIONS_KEY = 'ng_agent_observations'
 
nemo_gym.trajectory.trajectory_record.NG_AGENT_SANDBOX_OBSERVATION_KEY = 'ng_agent_sandbox_observation'
 
nemo_gym.trajectory.trajectory_record.NG_TRAJECTORY_KEY = 'ng_trajectory'
 
nemo_gym.trajectory.trajectory_record.NG_VERIFIER_INVOCATIONS_KEY = 'ng_verifier_invocations'