> This page is for version 0.7.0.
> For other versions, use one of these documentation indexes:
> - Main (default): https://docs.nvidia.com/nemo/gym/main/llms.txt
> - 0.7.0: https://docs.nvidia.com/nemo/gym/v0.7.0/llms.txt
> - 0.6.0: https://docs.nvidia.com/nemo/gym/v0.6.0/llms.txt
> - 0.5.1: https://docs.nvidia.com/nemo/gym/v0.5.1/llms.txt
> - 0.5.0: https://docs.nvidia.com/nemo/gym/v0.5.0/llms.txt
> - 0.4.0: https://docs.nvidia.com/nemo/gym/v0.4.0/llms.txt
> - 0.3.0: https://docs.nvidia.com/nemo/gym/v0.3.0/llms.txt
> - 0.2.1: https://docs.nvidia.com/nemo/gym/v0.2.1/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/gym/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/gym/_mcp/server.

# nemo_gym.server_runtime.verifier_observability

Observe whole verify executions independently of their captured model calls.

## Module Contents

### Functions

| Name                                                                                                                 | Description                                                                                            |
| -------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------ |
| [`_finish`](#nemo_gym-server_runtime-verifier_observability-_finish)                                                 | -                                                                                                      |
| [`merge_verifier_call_observation`](#nemo_gym-server_runtime-verifier_observability-merge_verifier_call_observation) | Link a captured attempt to its caller-declared verifier invocation, when supplied.                     |
| [`observe_llm_judge_grading`](#nemo_gym-server_runtime-verifier_observability-observe_llm_judge_grading)             | Execute a reviewed async LLM-judge grading once and record its actual result in the active invocation. |
| [`observe_programmatic_grading`](#nemo_gym-server_runtime-verifier_observability-observe_programmatic_grading)       | Execute a reviewed synchronous scorer once and record its actual result in the active invocation.      |
| [`observe_verifier`](#nemo_gym-server_runtime-verifier_observability-observe_verifier)                               | Return verification evidence with its result, leaving persistence to collection.                       |
| [`observe_verifier_sandbox`](#nemo_gym-server_runtime-verifier_observability-observe_verifier_sandbox)               | Bind owner-observed sandbox facts to the active verification.                                          |

### Data

[`P`](#nemo_gym-server_runtime-verifier_observability-P)

[`T`](#nemo_gym-server_runtime-verifier_observability-T)

[`_active_verifier`](#nemo_gym-server_runtime-verifier_observability-_active_verifier)

[`logger`](#nemo_gym-server_runtime-verifier_observability-logger)

### API

```python
nemo_gym.server_runtime.verifier_observability._finish(
    invocation: nemo_gym.trajectory.trajectory_record.VerifierInvocation,
    result: object,
    verifier_name: str | None,
    graded_in_process: bool
) -> None
```

```python
nemo_gym.server_runtime.verifier_observability.merge_verifier_call_observation(
    invocations: dict[str, nemo_gym.trajectory.trajectory_record.VerifierInvocation],
    exchange: collections.abc.Mapping[str, object],
    model_call_id: str
) -> None
```

Link a captured attempt to its caller-declared verifier invocation, when supplied.

```python
nemo_gym.server_runtime.verifier_observability.observe_llm_judge_grading(
    calculate: collections.abc.Callable[[], collections.abc.Awaitable[nemo_gym.server_runtime.verifier_observability.T]]
) -> nemo_gym.server_runtime.verifier_observability.T
```

async

Execute a reviewed async LLM-judge grading once and record its actual result in the active invocation.

The judge replies parsed by `call_judge` during `calculate` become the operation's dependencies. It completes
only when every judge reply was identified, at least one was, the call site marked every reply with
`record_parsed_judge_verdict` and the result binds; otherwise it stays unknown. Errors propagate unchanged.

```python
nemo_gym.server_runtime.verifier_observability.observe_programmatic_grading(
    calculate: collections.abc.Callable[[], nemo_gym.server_runtime.verifier_observability.T]
) -> nemo_gym.server_runtime.verifier_observability.T
```

Execute a reviewed synchronous scorer once and record its actual result in the active invocation.

This is an explicit call-site contract for pure calculations, not a purity detector for arbitrary callbacks.
Judge/SDK/sandbox paths must not use it to label a returned result as programmatic.

```python
nemo_gym.server_runtime.verifier_observability.observe_verifier(
    verify_fn: collections.abc.Callable[nemo_gym.server_runtime.verifier_observability.P, collections.abc.Awaitable[nemo_gym.server_runtime.verifier_observability.T]],
    enabled: bool,
    verifier_name: str | None = None,
    uninstrumented_models: bool = False
) -> collections.abc.Callable[nemo_gym.server_runtime.verifier_observability.P, collections.abc.Awaitable[nemo_gym.server_runtime.verifier_observability.T | starlette.responses.Response]]
```

Return verification evidence with its result, leaving persistence to collection.

A verify execution named by `verifier_name` that completes without sending any model request also returns
its grading record: the verifier, the score and its verdict, bound to the graded response. A verifier that may
ask a model through a client that notes no request (`uninstrumented_models`) never returns one: missing notes
do not show that its grading was model-free.

HTTP failures keep their status and carry evidence in the error body. Cancellation
still propagates; a cancelled or lost response cannot deliver in-memory evidence.

```python
nemo_gym.server_runtime.verifier_observability.observe_verifier_sandbox(
    observation: nemo_gym.trajectory.trajectory_record.SandboxObservation | collections.abc.Mapping[str, object]
) -> None
```

Bind owner-observed sandbox facts to the active verification.

These facts do not certify remote liveness or dependency-free grading.

```python
nemo_gym.server_runtime.verifier_observability.P = ParamSpec('P')
```

```python
nemo_gym.server_runtime.verifier_observability.T = TypeVar('T')
```

```python
nemo_gym.server_runtime.verifier_observability._active_verifier: ContextVar[VerifierInvocation | None] = ContextVar('active_verifier_grading', default=None)
```

```python
nemo_gym.server_runtime.verifier_observability.logger = logging.getLogger(__name__)
```