> This page is for version 0.3.0.
> For other versions, use one of these documentation indexes:
> - Main (default): https://docs.nvidia.com/nemo/gym/main/llms.txt
> - 0.7.0: https://docs.nvidia.com/nemo/gym/v0.7.0/llms.txt
> - 0.6.0: https://docs.nvidia.com/nemo/gym/v0.6.0/llms.txt
> - 0.5.1: https://docs.nvidia.com/nemo/gym/v0.5.1/llms.txt
> - 0.5.0: https://docs.nvidia.com/nemo/gym/v0.5.0/llms.txt
> - 0.4.0: https://docs.nvidia.com/nemo/gym/v0.4.0/llms.txt
> - 0.3.0: https://docs.nvidia.com/nemo/gym/v0.3.0/llms.txt
> - 0.2.1: https://docs.nvidia.com/nemo/gym/v0.2.1/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/gym/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/gym/_mcp/server.

# nemo_gym.eval.comparison.suite.processing

Validate coverage and assemble statistical inputs from stored suite artifacts.

## Module Contents

### Functions

| Name                                                                                          | Description                                                                             |
| --------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------- |
| [`_collection_summary`](#nemo_gym-eval-comparison-suite-processing-_collection_summary)       | -                                                                                       |
| [`_metric_values`](#nemo_gym-eval-comparison-suite-processing-_metric_values)                 | -                                                                                       |
| [`_run_summary`](#nemo_gym-eval-comparison-suite-processing-_run_summary)                     | -                                                                                       |
| [`_samples`](#nemo_gym-eval-comparison-suite-processing-_samples)                             | -                                                                                       |
| [`_select_metric_repeats`](#nemo_gym-eval-comparison-suite-processing-_select_metric_repeats) | Omit token-only rows lacking selected estimates; score rows need not have sample_count. |
| [`_statistics`](#nemo_gym-eval-comparison-suite-processing-_statistics)                       | -                                                                                       |
| [`aggregate_suite_cells`](#nemo_gym-eval-comparison-suite-processing-aggregate_suite_cells)   | Aggregate eligible cells equally, retaining declared exclusions in the cell report.     |
| [`compare_suite_cell`](#nemo_gym-eval-comparison-suite-processing-compare_suite_cell)         | Compare concatenated independent repeat samples, retaining source-level evidence.       |

### API

```python
nemo_gym.eval.comparison.suite.processing._collection_summary(
    runs: tuple[nemo_gym.eval.comparison.suite.loading.LoadedSuiteRun, ...],
    coverages: list[nemo_gym.eval.comparison.suite.schema.SuiteCoverage],
    scores: tuple[float, ...],
    collection_notes: list[str]
) -> nemo_gym.eval.comparison.suite.schema.SuiteRunSummary
```

```python
nemo_gym.eval.comparison.suite.processing._metric_values(
    run: nemo_gym.eval.comparison.suite.loading.LoadedSuiteRun,
    metrics: list[nemo_gym.eval.comparison.suite.schema.SuiteMetricResult]
) -> list[list[float]]
```

```python
nemo_gym.eval.comparison.suite.processing._run_summary(
    run: nemo_gym.eval.comparison.suite.loading.LoadedSuiteRun,
    coverage: nemo_gym.eval.comparison.suite.schema.SuiteCoverage,
    scores: tuple[float, ...]
) -> nemo_gym.eval.comparison.suite.schema.SuiteRunSummary
```

```python
nemo_gym.eval.comparison.suite.processing._samples(
    cell: nemo_gym.eval.comparison.suite.schema.SuiteCellResult
) -> nemo_gym.eval.comparison.statistics.unpaired_welch_fixed_suite.BenchmarkSamples
```

```python
nemo_gym.eval.comparison.suite.processing._select_metric_repeats(
    run: nemo_gym.eval.comparison.suite.loading.LoadedSuiteRun,
    metrics: list[nemo_gym.eval.comparison.suite.schema.SuiteMetricResult]
) -> nemo_gym.eval.comparison.suite.loading.LoadedSuiteRun
```

Omit token-only rows lacking selected estimates; score rows need not have sample_count.

```python
nemo_gym.eval.comparison.suite.processing._statistics(
    cells: list[nemo_gym.eval.comparison.suite.schema.SuiteCellResult],
    confidence_level: float
) -> nemo_gym.eval.comparison.suite.schema.SuiteStatistics
```

```python
nemo_gym.eval.comparison.suite.processing.aggregate_suite_cells(
    name: str,
    cells: list[nemo_gym.eval.comparison.suite.schema.SuiteCellResult],
    confidence_level: float
) -> nemo_gym.eval.comparison.suite.schema.SuiteAggregateResult
```

Aggregate eligible cells equally, retaining declared exclusions in the cell report.

```python
nemo_gym.eval.comparison.suite.processing.compare_suite_cell(
    model: str,
    benchmark: str,
    baseline: nemo_gym.eval.comparison.suite.loading.LoadedSuiteRun | tuple[nemo_gym.eval.comparison.suite.loading.LoadedSuiteRun, ...],
    candidate: nemo_gym.eval.comparison.suite.loading.LoadedSuiteRun | tuple[nemo_gym.eval.comparison.suite.loading.LoadedSuiteRun, ...],
    metrics: list[nemo_gym.eval.comparison.suite.schema.SuiteMetricResult],
    confidence_level: float
) -> nemo_gym.eval.comparison.suite.schema.SuiteCellResult
```

Compare concatenated independent repeat samples, retaining source-level evidence.