> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/gym/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/gym/_mcp/server.

# nemo_gym.benchmarks

Benchmark discovery and preparation utilities.

## Module Contents

### Classes

| Name                                                      | Description |
| --------------------------------------------------------- | ----------- |
| [`BenchmarkConfig`](#nemo_gym-benchmarks-BenchmarkConfig) | -           |

### Functions

| Name                                                                                                                  | Description                                                                                                   |
| --------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------- |
| [`_load_benchmarks_from_config_paths`](#nemo_gym-benchmarks-_load_benchmarks_from_config_paths)                       | -                                                                                                             |
| [`_parse_no_environment_tolerating_unset_values`](#nemo_gym-benchmarks-_parse_no_environment_tolerating_unset_values) | `parse_no_environment` for *listing*: fill unset `???` values and undefined `${...}` interpolations |

### Data

[`BENCHMARKS_DIR`](#nemo_gym-benchmarks-BENCHMARKS_DIR)

[`_UNSET_VALUE_PLACEHOLDER`](#nemo_gym-benchmarks-_UNSET_VALUE_PLACEHOLDER)

[`__getattr__`](#nemo_gym-benchmarks-__getattr__)

### API

```python
class nemo_gym.benchmarks.BenchmarkConfig()
```

**Bases:** `BaseModel`

```python
nemo_gym.benchmarks.BenchmarkConfig.from_config_path(
    config_path: pathlib.Path,
    strict: bool = True
) -> typing.Optional[nemo_gym.benchmarks.BenchmarkConfig]
```

classmethod

```python
nemo_gym.benchmarks.BenchmarkConfig.from_initial_config_dict(
    path: pathlib.Path,
    initial_config_dict: omegaconf.DictConfig,
    strict: bool = True
) -> typing.Optional[nemo_gym.benchmarks.BenchmarkConfig]
```

classmethod

```python
nemo_gym.benchmarks._load_benchmarks_from_config_paths(
    config_paths: typing.List[pathlib.Path]
) -> typing.Dict[str, nemo_gym.benchmarks.BenchmarkConfig]
```

```python
nemo_gym.benchmarks._parse_no_environment_tolerating_unset_values(
    initial_config_dict: omegaconf.DictConfig
) -> omegaconf.DictConfig
```

`parse_no_environment` for *listing*: fill unset `???` values and undefined `${...}` interpolations
with a placeholder so a benchmark referencing runtime-only values can still be identified. Never mutates
the caller's config; errors other than those two propagate.

`???` is filled anywhere; `${...}` only where parse forces resolution (top-level and server sections),
not inside arbitrary non-server nested dicts — fine for listing, whose interpolations live in servers.

```python
nemo_gym.benchmarks.BENCHMARKS_DIR = PARENT_DIR / 'benchmarks'
```

```python
nemo_gym.benchmarks._UNSET_VALUE_PLACEHOLDER = '__unset_for_listing__'
```

```python
nemo_gym.benchmarks.__getattr__ = moved_attr_getter(__name__, {'list_benchmarks': 'nemo_gym.cli.eval', 'PrepareBen...
```