nemo_gym.harness_capabilities.results

Independent check results. TE labels describe results; they never schedule checks.

Module Contents

Classes

Name	Description
`CheckResult`	-
`Results`	Combine repeated evaluations of one rule without hiding failures or blocked checks.

Functions

Name	Description
`gate_passes`	Require all applicable checks in the selected tier, independently of TE labels.
`render_matrices`	Render saved check results; never infer a pass from an absent failure message.

Data

Status

API

class nemo_gym.harness_capabilities.results.CheckResult(
    id: str,
    kind: nemo_gym.harness_capabilities.checks.Kind,
    status: nemo_gym.harness_capabilities.results.Status,
    tier: nemo_gym.harness_capabilities.checks.PriorityTier,
    evidence: tuple[str, ...] = (),
    locations: list[str] = list(),
    reasons: list[str] = list(),
    blocked_by: list[str] = list()
)

Dataclass

blocked_by

list[str] = field(default_factory=list)

evidence

tuple[str, ...] = ()

str

kind

Kind

locations

list[str] = field(default_factory=list)

reasons

list[str] = field(default_factory=list)

status

Status

tier

PriorityTier

class nemo_gym.harness_capabilities.results.Results()

Combine repeated evaluations of one rule without hiding failures or blocked checks.

rows

dict[str, CheckResult] = {}

nemo_gym.harness_capabilities.results.Results.add(
    check_id: str,
    kind: nemo_gym.harness_capabilities.checks.Kind,
    status: nemo_gym.harness_capabilities.results.Status,
    tier: nemo_gym.harness_capabilities.checks.PriorityTier,
    evidence: tuple[str, ...] = (),
    location: str = '',
    reason: str = '',
    blocked_by: collections.abc.Iterable[str] = ()
) -> bool

nemo_gym.harness_capabilities.results.Results.dump() -> list[dict]

nemo_gym.harness_capabilities.results.Results.run(
    check: nemo_gym.harness_capabilities.checks.Check
) -> bool

Execute a typed rule; excluded and blocked rules never evaluate their input.

nemo_gym.harness_capabilities.results.gate_passes(
    checks: list[dict],
    tier: nemo_gym.harness_capabilities.checks.PriorityTier = 'P0'
) -> bool

Require all applicable checks in the selected tier, independently of TE labels.

nemo_gym.harness_capabilities.results.render_matrices(
    harnesses: dict
) -> str

Render saved check results; never infer a pass from an absent failure message.

nemo_gym.harness_capabilities.results.Status = Literal['pass', 'fail', 'not_applicable']