nemo_gym.harness_capabilities.checks

Typed check definitions; execution state and reporting live in Results.

Module Contents

Classes

Name	Description
`BehavioralCheck`	Evaluate behavior or evidence against independent controlled observations.
`Check`	One rule bound to the input being inspected. TE labels do not set priority.
`Evaluation`	A predicate outcome and any specific failure locations, without payloads.
`SchemaCheck`	Validate the selected input against this check’s JSON Schema.
`SemanticCheck`	Evaluate a relationship between retained evidence records.
`_PredicateCheck`	-

Data

Kind

PriorityTier

API

class nemo_gym.harness_capabilities.checks.BehavioralCheck(
    id: str,
    tier: nemo_gym.harness_capabilities.checks.PriorityTier,
    evidence: tuple[str, ...],
    location: str,
    reason: str,
    depends_on: tuple[str, ...] = (),
    applies: bool = True,
    available: bool = True,
    predicate: collections.abc.Callable[[], bool]
)

Dataclass

Bases: _PredicateCheck

Evaluate behavior or evidence against independent controlled observations.

kind

Kind = 'behavioral'

class nemo_gym.harness_capabilities.checks.Check(
    id: str,
    tier: nemo_gym.harness_capabilities.checks.PriorityTier,
    evidence: tuple[str, ...],
    location: str,
    reason: str,
    depends_on: tuple[str, ...] = (),
    applies: bool = True,
    available: bool = True
)

DataclassAbstract

One rule bound to the input being inspected. TE labels do not set priority.

applies

bool = True

available

bool = True

depends_on

tuple[str, ...] = ()

evidence

tuple[str, ...]

str

kind

Kind

location

str

reason

str

tier

PriorityTier

nemo_gym.harness_capabilities.checks.Check.__post_init__() -> None

nemo_gym.harness_capabilities.checks.Check.evaluate() -> nemo_gym.harness_capabilities.checks.Evaluation

abstract

Evaluate only after applicability, availability and prerequisites pass.

class nemo_gym.harness_capabilities.checks.Evaluation(
    passed: bool,
    locations: tuple[str, ...] = ()
)

Dataclass

A predicate outcome and any specific failure locations, without payloads.

locations

tuple[str, ...] = ()

passed

bool

class nemo_gym.harness_capabilities.checks.SchemaCheck(
    id: str,
    tier: nemo_gym.harness_capabilities.checks.PriorityTier,
    evidence: tuple[str, ...],
    location: str,
    reason: str,
    depends_on: tuple[str, ...] = (),
    applies: bool = True,
    available: bool = True,
    schema: dict,
    value: object
)

Dataclass

Bases: Check

Validate the selected input against this check’s JSON Schema.

kind

Kind = 'schema'

schema

dict

value

object

nemo_gym.harness_capabilities.checks.SchemaCheck.evaluate() -> nemo_gym.harness_capabilities.checks.Evaluation

class nemo_gym.harness_capabilities.checks.SemanticCheck(
    id: str,
    tier: nemo_gym.harness_capabilities.checks.PriorityTier,
    evidence: tuple[str, ...],
    location: str,
    reason: str,
    depends_on: tuple[str, ...] = (),
    applies: bool = True,
    available: bool = True,
    predicate: collections.abc.Callable[[], bool]
)

Dataclass

Bases: _PredicateCheck

Evaluate a relationship between retained evidence records.

kind

Kind = 'semantic'

class nemo_gym.harness_capabilities.checks._PredicateCheck(
    id: str,
    tier: nemo_gym.harness_capabilities.checks.PriorityTier,
    evidence: tuple[str, ...],
    location: str,
    reason: str,
    depends_on: tuple[str, ...] = (),
    applies: bool = True,
    available: bool = True,
    predicate: collections.abc.Callable[[], bool]
)

Dataclass

Bases: Check

predicate

Callable[[], bool]

nemo_gym.harness_capabilities.checks._PredicateCheck.evaluate() -> nemo_gym.harness_capabilities.checks.Evaluation

nemo_gym.harness_capabilities.checks.Kind = Literal['schema', 'semantic', 'behavioral']

nemo_gym.harness_capabilities.checks.PriorityTier = Literal['P0', 'P1', 'P2']