> This page is for version 0.5.1.
> For other versions, use one of these documentation indexes:
> - Main (default): https://docs.nvidia.com/nemo/gym/main/llms.txt
> - 0.7.0: https://docs.nvidia.com/nemo/gym/v0.7.0/llms.txt
> - 0.6.0: https://docs.nvidia.com/nemo/gym/v0.6.0/llms.txt
> - 0.5.1: https://docs.nvidia.com/nemo/gym/v0.5.1/llms.txt
> - 0.5.0: https://docs.nvidia.com/nemo/gym/v0.5.0/llms.txt
> - 0.4.0: https://docs.nvidia.com/nemo/gym/v0.4.0/llms.txt
> - 0.3.0: https://docs.nvidia.com/nemo/gym/v0.3.0/llms.txt
> - 0.2.1: https://docs.nvidia.com/nemo/gym/v0.2.1/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/gym/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/gym/_mcp/server.

# nemo_gym.rollout_compare.schema

The vocabulary of the comparison.

The task key candidates, the embedded verifier fields, the sidecar flag, the file name
constants, the Category and Side literals, the Roots alias, the input error and the three
result dataclasses: one side of a task, one compared task and the whole result. Every other
module of the package imports from here and this module imports from none of them.

## Module Contents

### Classes

| Name                                                                      | Description                                                       |
| ------------------------------------------------------------------------- | ----------------------------------------------------------------- |
| [`CompareInputError`](#nemo_gym-rollout_compare-schema-CompareInputError) | An input file could not be read or does not look like rollouts.   |
| [`CompareResult`](#nemo_gym-rollout_compare-schema-CompareResult)         | -                                                                 |
| [`TaskComparison`](#nemo_gym-rollout_compare-schema-TaskComparison)       | -                                                                 |
| [`TaskSide`](#nemo_gym-rollout_compare-schema-TaskSide)                   | One side's rows for a task, reduced to what the comparison needs. |

### Data

[`Category`](#nemo_gym-rollout_compare-schema-Category)

[`EMBEDDED_VERIFIER_FIELDS`](#nemo_gym-rollout_compare-schema-EMBEDDED_VERIFIER_FIELDS)

[`FAILURES_SUFFIX`](#nemo_gym-rollout_compare-schema-FAILURES_SUFFIX)

[`Roots`](#nemo_gym-rollout_compare-schema-Roots)

[`SIDECAR_FLAG`](#nemo_gym-rollout_compare-schema-SIDECAR_FLAG)

[`Side`](#nemo_gym-rollout_compare-schema-Side)

[`TASK_KEY_CANDIDATES`](#nemo_gym-rollout_compare-schema-TASK_KEY_CANDIDATES)

[`VERIFIER_STDOUT_FILENAME`](#nemo_gym-rollout_compare-schema-VERIFIER_STDOUT_FILENAME)

[`_T`](#nemo_gym-rollout_compare-schema-_T)

### API

```python
class nemo_gym.rollout_compare.CompareInputError()
```

Exception

**Bases:** `Exception`

An input file could not be read or does not look like rollouts.

```python
class nemo_gym.rollout_compare.CompareResult(
    tasks: list[nemo_gym.rollout_compare.schema.TaskComparison],
    old_key: str,
    new_key: str,
    old_rows: int,
    new_rows: int,
    old_masked_rows: int = 0,
    new_masked_rows: int = 0,
    old_prefix_stripped: typing.Optional[str] = None,
    new_prefix_stripped: typing.Optional[str] = None,
    keyed_by_taskset: bool = False,
    notes: list[str] = list()
)
```

Dataclass

**`keyed_by_taskset`** `bool = False`

---

**`new_key`** `str`

---

**`new_masked_rows`** `int = 0`

---

**`new_prefix_stripped`** `Optional[str] = None`

---

**`new_rows`** `int`

---

**`notes`** `list[str] = field(default_factory=list)`

---

**`old_key`** `str`

---

**`old_masked_rows`** `int = 0`

---

**`old_prefix_stripped`** `Optional[str] = None`

---

**`old_rows`** `int`

---

**`tasks`** `list[TaskComparison]`

---

```python
nemo_gym.rollout_compare.CompareResult.by_category(
    category: nemo_gym.rollout_compare.schema.Category
) -> list[nemo_gym.rollout_compare.schema.TaskComparison]
```

```python
nemo_gym.rollout_compare.CompareResult.counts() -> dict[str, int]
```

```python
class nemo_gym.rollout_compare.TaskComparison(
    task: str,
    category: nemo_gym.rollout_compare.schema.Category,
    old: typing.Optional[nemo_gym.rollout_compare.schema.TaskSide] = None,
    new: typing.Optional[nemo_gym.rollout_compare.schema.TaskSide] = None,
    winner: typing.Optional[nemo_gym.rollout_compare.schema.Side] = None
)
```

Dataclass

**`category`** `Category`

---

**`loser`** `Optional[Side]`

---

**`new`** `Optional[TaskSide] = None`

---

**`old`** `Optional[TaskSide] = None`

---

**`task`** `str`

---

**`winner`** `Optional[Side] = None`

---

```python
class nemo_gym.rollout_compare.TaskSide(
    rows: list[dict[str, typing.Any]],
    reward: typing.Optional[float],
    masked: typing.Optional[str],
    agent_error: typing.Optional[str],
    reward_rows: int = 0,
    failure_kind: typing.Optional[str] = None
)
```

Dataclass

One side's rows for a task, reduced to what the comparison needs.

**`agent_error`** `Optional[str]`

---

**`failure_kind`** `Optional[str] = None`

---

**`masked`** `Optional[str]`

---

**`repeated`** `bool`

---

**`reward`** `Optional[float]`

---

**`reward_rows`** `int = 0`

---

**`rows`** `list[dict[str, Any]]`

---

```python
nemo_gym.rollout_compare.TaskSide.reward_text() -> str
```

```python
nemo_gym.rollout_compare.schema.Category = Literal['identical', 'flipped', 'flipped_agent_error', 'masked', 'missing']
```

```python
nemo_gym.rollout_compare.schema.EMBEDDED_VERIFIER_FIELDS: tuple[str, ...] = ('test_output', 'verifier_stdout', 'verifier_output')
```

```python
nemo_gym.rollout_compare.schema.FAILURES_SUFFIX = '_failures.jsonl'
```

```python
nemo_gym.rollout_compare.schema.Roots = list[Path] | dict[str, list[Path]]
```

```python
nemo_gym.rollout_compare.schema.SIDECAR_FLAG = '_ng_compare_sidecar'
```

```python
nemo_gym.rollout_compare.schema.Side = Literal['old', 'new']
```

```python
nemo_gym.rollout_compare.schema.TASK_KEY_CANDIDATES: tuple[str, ...] = ('_ng_task_id', 'task_name', 'task_id', 'verifier_metadata.task_id', 'verifier_m...
```

```python
nemo_gym.rollout_compare.schema.VERIFIER_STDOUT_FILENAME = 'test-stdout.txt'
```

```python
nemo_gym.rollout_compare.schema._T = TypeVar('_T')
```