Skip to navigation

nemo_gym.core.failure_records

View as Markdown

What a failure records: the events Gym servers observe, the report that forwards them, and the episode failure.

Failure kinds live in nemo_gym.core.failure_kinds; nemo_gym.core.failure_transport carries reports over HTTP.

Module Contents

Classes

NameDescription
EpisodeFailureDescribe a failure using the same fields on the wire and in saved records.
FailureBoundaryThe places a failure is observed, as the failure-boundaries RFC names them.
FailureEventOne observed failure: who observed it, what failed, and which failure caused it.
FailureReportThe failures one reply forwards: its own failure (failure_id, absent on success) and every failure
ModelFailureDetailsObserved model-request facts, independent of retry or scoring policy.
SandboxFailureDetailsObserved sandbox facts; missing evidence stays absent, with no inferred cause or policy.
SdkErrorDetailsDiagnostics an adapter selected from a third-party SDK error before it raised its own failure.

Functions

NameDescription
validate_failure_event_mapCheck that events is keyed by event id and that every cause_id resolves in it without a cycle.

Data

FailureDetails

API

class nemo_gym.core.failure_records.EpisodeFailure()

Bases: BaseModel

Describe a failure using the same fields on the wire and in saved records.

An environment server returns this envelope for one failed episode attempt. Collection uses its existing terminal flag and retry budget; failure_kind describes the failure without deciding terminality. Kind and stage are optional because a collector observing a lost reply may not know them.

The attempt’s FailureReport travels beside this envelope (BaseEpisodeResponse.failure_report), so records that nest EpisodeFailure keep their saved format.

failure_kind
str | None = None
failure_reason
str = Field(max_length=2000)
model_config
= ConfigDict(extra='forbid')
stage
FailureStage | None = None
terminal
bool
nemo_gym.core.failure_records.EpisodeFailure._serialize_failure(
handler: pydantic.SerializerFunctionWrapHandler
)
nemo_gym.core.failure_records.EpisodeFailure._validate_failure_kind(
value: str | None
) -> str | None
classmethod
class nemo_gym.core.failure_records.FailureBoundary

Bases: enum.Enum

The places a failure is observed, as the failure-boundaries RFC names them.

Each one reports what it observed; none of them chooses retries, terminality or scores.

AGENT
= 'agent'
COLLECTION
= 'collection'
ENVIRONMENT
= 'environment'
HARNESS
= 'harness'
MODEL
= 'model'
RESOURCES_SERVER
= 'resources_server'
SANDBOX
= 'sandbox'
class nemo_gym.core.failure_records.FailureEvent()

Bases: BaseModel

One observed failure: who observed it, what failed, and which failure caused it.

This is the validated wire projection of a NgFailureEvent, whose fields and origin identity live on the exception itself. cause_id names another Gym failure in the same map. Foreign SDK facts live in foreign_error; they are not boundary-less Gym events. Canonical projections validate kind and boundary on both failed and recovered reports, independently of the legacy outcome envelope. details uses the schema selected by boundary; common facts stay on this event, without a second copy in details.

Readers count each failure once by event_id. Outcome policy stays with EpisodeFailure.terminal and the collector’s existing rules.

attempt_id
str | None = Field(default=None, max_length=128)
boundary
FailureBoundary
cause_id
str | None = Field(default=None, max_length=128)
details
FailureDetails | None = None
event_id
str
exception_type
str | None = Field(default=None, max_length=256)
failure_kind
str
failure_reason
str = Field(default='', max_length=2000)
foreign_error
SdkErrorDetails | None = None
http_status
int | None = Field(default=None, ge=100, le=599)
model_config
= ConfigDict(extra='forbid', frozen=True)
operation
str | None = Field(default=None, max_length=256)
operation_id
str | None = Field(default=None, max_length=128)
stage
FailureStage | None = None
nemo_gym.core.failure_records.FailureEvent._serialize_event(
handler: pydantic.SerializerFunctionWrapHandler
)
nemo_gym.core.failure_records.FailureEvent._validate_details(
value: object,
info: pydantic.ValidationInfo
classmethod
nemo_gym.core.failure_records.FailureEvent._validate_failure_kind(
value: str
) -> str
classmethod
class nemo_gym.core.failure_records.FailureReport()

Bases: BaseModel

The failures one reply forwards: its own failure (failure_id, absent on success) and every failure observed while serving the request, each with its explicit causes.

omitted says that a report on the way here was too large to send, so events are missing. nemo_gym.core.failure_transport carries it in one HTTP header; Collection saves it beside the outcome.

events
dict[str, FailureEvent] = Field(default_factory=dict)
failure_id
str | None = Field(default=None, max_length=128)
model_config
= ConfigDict(extra='forbid')
omitted
bool = False
nemo_gym.core.failure_records.FailureReport._failure_id_is_an_event() -> typing_extensions.Self
nemo_gym.core.failure_records.FailureReport._restore_event_ids(
value
)
classmethod
nemo_gym.core.failure_records.FailureReport._validate_events(
classmethod

This report with other’s events added; failure_id stays this report’s.

An event id names one immutable record, so the first copy of an id is kept.

class nemo_gym.core.failure_records.ModelFailureDetails()

Bases: BaseModel

Observed model-request facts, independent of retry or scoring policy.

model_config
provider_request_id
str | None = Field(default=None, max_length=256)
request_upload_complete
bool | None = None
class nemo_gym.core.failure_records.SandboxFailureDetails()

Bases: BaseModel

Observed sandbox facts; missing evidence stays absent, with no inferred cause or policy.

A last execution ID does not establish causation. Preserve provider reason codes as reported; use only reported command exit codes, not synthesized fallback codes.

execution_id
str | None = Field(default=None, max_length=256)
exit_code
int | None = None
model_config
provider
str | None = Field(default=None, max_length=256)
provider_status_reason
str | None = Field(default=None, max_length=2000)
sandbox_id
str | None = Field(default=None, max_length=256)
class nemo_gym.core.failure_records.SdkErrorDetails()

Bases: BaseModel

Diagnostics an adapter selected from a third-party SDK error before it raised its own failure.

The adapter chooses the message (for example the SDK’s message field), because str() of an SDK error can contain request data. The SDK object, its request and its response body are never stored.

exception_type
str = Field(min_length=1, max_length=256)
http_status
int | None = Field(default=None, ge=100, le=599)
message
str = Field(max_length=2000)
model_config
= ConfigDict(extra='forbid', frozen=True)
nemo_gym.core.failure_records.validate_failure_event_map(
events: collections.abc.Mapping[str, nemo_gym.core.failure_records.FailureEvent]
) -> collections.abc.Mapping[str, nemo_gym.core.failure_records.FailureEvent]

Check that events is keyed by event id and that every cause_id resolves in it without a cycle.

nemo_gym.core.failure_records.FailureDetails = SandboxFailureDetails | ModelFailureDetails