> This page is for version 0.6.0.
> For other versions, use one of these documentation indexes:
> - Main (default): https://docs.nvidia.com/nemo/gym/main/llms.txt
> - 0.7.0: https://docs.nvidia.com/nemo/gym/v0.7.0/llms.txt
> - 0.6.0: https://docs.nvidia.com/nemo/gym/v0.6.0/llms.txt
> - 0.5.1: https://docs.nvidia.com/nemo/gym/v0.5.1/llms.txt
> - 0.5.0: https://docs.nvidia.com/nemo/gym/v0.5.0/llms.txt
> - 0.4.0: https://docs.nvidia.com/nemo/gym/v0.4.0/llms.txt
> - 0.3.0: https://docs.nvidia.com/nemo/gym/v0.3.0/llms.txt
> - 0.2.1: https://docs.nvidia.com/nemo/gym/v0.2.1/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/gym/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/gym/_mcp/server.

# nemo_gym.eval.comparison.pooling

Pool independent repeat observations without averaging run-level estimates.

## Module Contents

### Functions

| Name                                                                                 | Description                                                                          |
| ------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------ |
| [`_finite_number`](#nemo_gym-eval-comparison-pooling-_finite_number)                 | -                                                                                    |
| [`_pooled_groups`](#nemo_gym-eval-comparison-pooling-_pooled_groups)                 | Recompute per-task rewards from observations; never average task summaries.          |
| [`_pooled_metric_samples`](#nemo_gym-eval-comparison-pooling-_pooled_metric_samples) | Require complete quality samples and retain explicitly available token accounting.   |
| [`_quality_rows`](#nemo_gym-eval-comparison-pooling-_quality_rows)                   | Keep scored metric estimates while recognizing accounting-only producer rows.        |
| [`_validated_rows`](#nemo_gym-eval-comparison-pooling-_validated_rows)               | Validate the observation structure before collecting any metric samples.             |
| [`pool_loaded_runs`](#nemo_gym-eval-comparison-pooling-pool_loaded_runs)             | Concatenate complete repeat samples and recompute point estimates and 95% intervals. |

### API

```python
nemo_gym.eval.comparison.pooling._finite_number(
    value: object
) -> typing.TypeGuard[int | float]
```

```python
nemo_gym.eval.comparison.pooling._pooled_groups(
    runs: typing.Sequence[nemo_gym.eval.comparison.loading.LoadedRun]
) -> typing.Tuple[typing.List[typing.Dict[str, typing.Any]], typing.Optional[str]]
```

Recompute per-task rewards from observations; never average task summaries.

```python
nemo_gym.eval.comparison.pooling._pooled_metric_samples(
    name: str,
    rows_by_source: typing.Sequence[typing.List[typing.Dict[str, typing.Any]]],
    source_labels: typing.Sequence[str]
) -> typing.Tuple[typing.List[float], typing.List[str]]
```

Require complete quality samples and retain explicitly available token accounting.

```python
nemo_gym.eval.comparison.pooling._quality_rows(
    run: nemo_gym.eval.comparison.loading.LoadedRun,
    rows: typing.List[typing.Dict[str, typing.Any]],
    label: str
) -> typing.List[typing.Dict[str, typing.Any]]
```

Keep scored metric estimates while recognizing accounting-only producer rows.

```python
nemo_gym.eval.comparison.pooling._validated_rows(
    run: nemo_gym.eval.comparison.loading.LoadedRun,
    label: str
) -> typing.List[typing.Dict[str, typing.Any]]
```

Validate the observation structure before collecting any metric samples.

```python
nemo_gym.eval.comparison.pooling.pool_loaded_runs(
    runs: typing.Sequence[nemo_gym.eval.comparison.loading.LoadedRun],
    source_labels: typing.Sequence[str]
) -> nemo_gym.eval.comparison.loading.LoadedRun
```

Concatenate complete repeat samples and recompute point estimates and 95% intervals.

Each repeat has equal weight, even when sources have different repeat counts. Metrics
available only as whole-run aggregates (including pass\@k summaries) are omitted. A metric
other than token accounting must be finite in every scored row of every source.
Token metrics use the observations that record them, including accounting-only rows
from unscored repeats, without inventing a zero quality or token estimate.