nemo_gym.core.failure_kinds
nemo_gym.core.failure_kinds
Shared failure names and episode execution stages.
Gym describes failures with free text and component-local labels, so the same failure arrives at a collector, a log line and a metric under three different names and cannot be grouped. This module holds the one vocabulary those producers share.
A name says what kind of thing went wrong and nothing else. Whether to retry, whether
the request may be replayed, whether a completed result should be masked, and what to tell
a person are all properties of the occurrence, not of the name — they live on the failure
record, on the verify response, and in failure_reason respectively. Keeping them out is
deliberate: metadata attached to a name goes stale the moment one caller wants to retry
what another caller does not.
Names are low cardinality on purpose. They are safe as a metric label or a span dimension;
failure_reason never is, because it carries occurrence detail and can be unbounded.
Classification configuration uses Pydantic, but this module imports no server code.
Module Contents
Classes
Functions
Data
ENVIRONMENT_PROTOCOL_VIOLATION
HARNESS_TERMINATION_UNCONFIRMED
PROVIDER_COMMAND_OUTCOME_UNKNOWN
API
Bases: enum.Enum
Configurable responsibility for reporting; no class implies an action or health verdict.
Bases: BaseModel
One shared stage/kind mapping and explicit HTTP evidence rules.
Class overrides by HTTP status take precedence, followed by stage kind/default, HTTP-derived evidence, kind, and the unknown fallback. Stage rules require context admitted by the caller; a provider cannot establish agent ownership on its own. Partial overrides retain unspecified defaults. HTTP-derived labels are reporting projections only: never write them back into observed failure events.
Bases: BaseModel
Classification overrides for an established originating stage.
Private read-only default for classification; caller-created configs remain independent.
Merge partial stage configurations before their concrete models validate them.
Interpret only supplied evidence; never walk causes or mutate the observed kind/stage.
Whether a registered kind has a shared evaluation-side default without extra context.
Whether name is a class the acceptance gate derives from saved rows (GATE_DERIVED_KINDS).
Whether name is an environment’s own <server>:<kind> extension.
Whether name is part of the shared vocabulary.
Server subcategory for an already established evaluation failure, never a class decision.
Callers check the applied class first. In particular, OOM can be model-side or evaluation-side depending on its originating stage and admitted ownership.
Whether a recorded HTTP status is an integer in the 5xx class.
The UTC time a provider_quota_exhausted reason names (resets at YYYY-MM-DD HH:MM UTC), else None.
quota_reset_text as a unix time, else None.
Project observed HTTP evidence to a reporting label without changing the event.
Validate a canonical event kind; legacy outcome envelopes keep their migration validator.
A gate-derived class (GATE_DERIVED_KINDS) is no producer’s kind and is refused like any unregistered name.
Return name unchanged, warning once per unregistered value.
Producers call this at their boundary. It warns rather than raises so a component that still emits an old label stays visible during migration — rejecting it outright would replace an observable wrong name with an invisible dropped failure, which is worse than the problem this vocabulary exists to fix.
The judge failures whose saved answer is judged again, never regenerated: a failed judge call, a verdict that did not
parse (judge_unparseable), and collection’s own legacy class for an invalid judge response (judge_invalid).
Registered provider observations; membership does not assign responsibility.