> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/gym/llms.txt.
> For full documentation content, see https://docs.nvidia.com/nemo/gym/llms-full.txt.

# nemo_gym.rollout_collection

## Module Contents

### Classes

| Name                                                                                          | Description                                                                                                             |
| --------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- |
| [`E2ERolloutCollectionConfig`](#nemo_gym-rollout_collection-E2ERolloutCollectionConfig)       | Spin up all necessary servers and perform a batch of rollout collection using each dataset inside the provided configs. |
| [`RolloutAggregationConfig`](#nemo_gym-rollout_collection-RolloutAggregationConfig)           | Aggregate metrics across rollout shards produced by `ng_collect_rollouts +disable_aggregation=true`.                    |
| [`RolloutAggregationHelper`](#nemo_gym-rollout_collection-RolloutAggregationHelper)           | -                                                                                                                       |
| [`RolloutCollectionConfig`](#nemo_gym-rollout_collection-RolloutCollectionConfig)             | Perform a batch of rollout collection.                                                                                  |
| [`RolloutCollectionHelper`](#nemo_gym-rollout_collection-RolloutCollectionHelper)             | -                                                                                                                       |
| [`SharedRolloutCollectionConfig`](#nemo_gym-rollout_collection-SharedRolloutCollectionConfig) | -                                                                                                                       |

### Functions

| Name                                                                                            | Description                                                                              |
| ----------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------- |
| [`_expand_input_glob`](#nemo_gym-rollout_collection-_expand_input_glob)                         | Expand a glob-or-comma-separated-globs string into a sorted, deduplicated list of paths. |
| [`_rollout_request_debug_summary`](#nemo_gym-rollout_collection-_rollout_request_debug_summary) | -                                                                                        |
| [`aggregate_rollouts`](#nemo_gym-rollout_collection-aggregate_rollouts)                         | -                                                                                        |
| [`collect_rollouts`](#nemo_gym-rollout_collection-collect_rollouts)                             | -                                                                                        |

### API

<Anchor id="nemo_gym-rollout_collection-E2ERolloutCollectionConfig">
  <CodeBlock showLineNumbers={false} wordWrap={true}>
    ```python
    class nemo_gym.rollout_collection.E2ERolloutCollectionConfig()
    ```
  </CodeBlock>
</Anchor>

<Indent>
  **Bases:** [SharedRolloutCollectionConfig](#nemo_gym-rollout_collection-SharedRolloutCollectionConfig)

  Spin up all necessary servers and perform a batch of rollout collection using each dataset inside the provided configs.

  Examples:

  <CodeBlock showLineNumbers={false}>
    ```python
    ng_collect_rollouts         +output_jsonl_fpath=weather_rollouts.jsonl         +num_samples_in_parallel=10
    ```
  </CodeBlock>

  <ParamField path="reuse_existing_data_preparation" type="bool = False" />

  <ParamField path="split" type="Union[Literal['train'], Literal['validation'], Literal['benchmark']]" />
</Indent>

<Anchor id="nemo_gym-rollout_collection-RolloutAggregationConfig">
  <CodeBlock showLineNumbers={false} wordWrap={true}>
    ```python
    class nemo_gym.rollout_collection.RolloutAggregationConfig()
    ```
  </CodeBlock>
</Anchor>

<Indent>
  **Bases:** [BaseNeMoGymCLIConfig](/nemo-gym/nemo_gym/config_types#nemo_gym-config_types-BaseNeMoGymCLIConfig)

  Aggregate metrics across rollout shards produced by `ng_collect_rollouts +disable_aggregation=true`.

  Reads every JSONL file matching `input_glob`, computes aggregate metrics by POSTing to each
  agent server's `/aggregate_metrics` endpoint over the global union of records, and writes a
  single `&lt;output_jsonl_fpath stem&gt;_aggregate_metrics.json` next to the rollouts. By default
  also concatenates all shards into `output_jsonl_fpath`.

  Examples:

  <CodeBlock showLineNumbers={false}>
    ```python
    ng_aggregate_rollouts         "+config_paths=[benchmarks/aime24/config.yaml,responses_api_models/vllm_model/configs/vllm_model.yaml]"         +input_glob='results/rollouts-rs*-chunk*.jsonl'         +output_jsonl_fpath=results/rollouts.jsonl
    ```
  </CodeBlock>

  <ParamField path="input_glob" type="str" />

  <ParamField path="merge_shards" type="bool" />

  <ParamField path="output_jsonl_fpath" type="str" />
</Indent>

<Anchor id="nemo_gym-rollout_collection-RolloutAggregationHelper">
  <CodeBlock showLineNumbers={false} wordWrap={true}>
    ```python
    class nemo_gym.rollout_collection.RolloutAggregationHelper()
    ```
  </CodeBlock>
</Anchor>

<Indent>
  **Bases:** `BaseModel`

  <Anchor id="nemo_gym-rollout_collection-RolloutAggregationHelper-run_from_config">
    <CodeBlock links={{"nemo_gym.rollout_collection.RolloutAggregationConfig":"#nemo_gym-rollout_collection-RolloutAggregationConfig"}} showLineNumbers={false} wordWrap={true}>
      ```python
      nemo_gym.rollout_collection.RolloutAggregationHelper.run_from_config(
          config: nemo_gym.rollout_collection.RolloutAggregationConfig
      ) -> typing.Optional[pathlib.Path]
      ```
    </CodeBlock>
  </Anchor>

  <Indent>
    <Badge>
      async
    </Badge>
  </Indent>
</Indent>

<Anchor id="nemo_gym-rollout_collection-RolloutCollectionConfig">
  <CodeBlock showLineNumbers={false} wordWrap={true}>
    ```python
    class nemo_gym.rollout_collection.RolloutCollectionConfig()
    ```
  </CodeBlock>
</Anchor>

<Indent>
  **Bases:** [SharedRolloutCollectionConfig](#nemo_gym-rollout_collection-SharedRolloutCollectionConfig)

  Perform a batch of rollout collection.

  Examples:

  <CodeBlock showLineNumbers={false}>
    ```python
    ng_collect_rollouts         +agent_name=example_single_tool_call_simple_agent         +input_jsonl_fpath=weather_query.jsonl         +output_jsonl_fpath=weather_rollouts.jsonl         +limit=100         +num_repeats=4         +num_samples_in_parallel=10
    ```
  </CodeBlock>

  <ParamField path="agent_name" type="Optional[str]" />

  <ParamField path="input_jsonl_fpath" type="str" />

  <ParamField path="limit" type="Optional[int]" />

  <ParamField path="materialized_jsonl_fpath" type="Path" />

  <ParamField path="num_repeats" type="Optional[int]" />

  <ParamField path="num_repeats_add_seed" type="bool" />

  <ParamField path="prompt_config" type="Optional[str]" />

  <ParamField path="resume_from_cache" type="bool" />
</Indent>

<Anchor id="nemo_gym-rollout_collection-RolloutCollectionHelper">
  <CodeBlock showLineNumbers={false} wordWrap={true}>
    ```python
    class nemo_gym.rollout_collection.RolloutCollectionHelper()
    ```
  </CodeBlock>
</Anchor>

<Indent>
  **Bases:** `BaseModel`

  <Anchor id="nemo_gym-rollout_collection-RolloutCollectionHelper-_call_aggregate_metrics">
    <CodeBlock showLineNumbers={false} wordWrap={true}>
      ```python
      nemo_gym.rollout_collection.RolloutCollectionHelper._call_aggregate_metrics(
          results: typing.List[typing.Dict],
          rows: typing.List[typing.Dict],
          output_fpath: pathlib.Path
      ) -> typing.Optional[pathlib.Path]
      ```
    </CodeBlock>
  </Anchor>

  <Indent>
    <Badge>
      async
    </Badge>

    Call /aggregate\_metrics on each agent server after rollouts complete.

    Writes a single \_aggregate\_metrics.json with one entry per agent (same shape
    as the old \_agent\_metrics.json). Returns the file path.
  </Indent>

  <Anchor id="nemo_gym-rollout_collection-RolloutCollectionHelper-_load_from_cache">
    <CodeBlock links={{"nemo_gym.rollout_collection.RolloutCollectionConfig":"#nemo_gym-rollout_collection-RolloutCollectionConfig"}} showLineNumbers={false} wordWrap={true}>
      ```python
      nemo_gym.rollout_collection.RolloutCollectionHelper._load_from_cache(
          config: nemo_gym.rollout_collection.RolloutCollectionConfig
      ) -> typing.Tuple[typing.List[typing.Dict], typing.List[typing.Dict], typing.List[typing.Dict], typing.List[typing.List[str]]]
      ```
    </CodeBlock>
  </Anchor>

  <Indent />

  <Anchor id="nemo_gym-rollout_collection-RolloutCollectionHelper-_preprocess_rows_from_config">
    <CodeBlock links={{"nemo_gym.rollout_collection.RolloutCollectionConfig":"#nemo_gym-rollout_collection-RolloutCollectionConfig"}} showLineNumbers={false} wordWrap={true}>
      ```python
      nemo_gym.rollout_collection.RolloutCollectionHelper._preprocess_rows_from_config(
          config: nemo_gym.rollout_collection.RolloutCollectionConfig
      ) -> typing.List[typing.Dict]
      ```
    </CodeBlock>
  </Anchor>

  <Indent />

  <Anchor id="nemo_gym-rollout_collection-RolloutCollectionHelper-run_examples">
    <CodeBlock links={{"nemo_gym.config_types.BaseServerConfig":"/nemo-gym/nemo_gym/config_types#nemo_gym-config_types-BaseServerConfig"}} showLineNumbers={false} wordWrap={true}>
      ```python
      nemo_gym.rollout_collection.RolloutCollectionHelper.run_examples(
          examples: typing.List[typing.Dict],
          head_server_config: typing.Optional[nemo_gym.config_types.BaseServerConfig] = None,
          semaphore: typing.Optional[asyncio.Semaphore] = None
      ) -> typing.Iterator[asyncio.Future]
      ```
    </CodeBlock>
  </Anchor>

  <Indent>
    We provide this function as a lower level interface for running rollout collection.
  </Indent>

  <Anchor id="nemo_gym-rollout_collection-RolloutCollectionHelper-run_from_config">
    <CodeBlock links={{"nemo_gym.rollout_collection.RolloutCollectionConfig":"#nemo_gym-rollout_collection-RolloutCollectionConfig"}} showLineNumbers={false} wordWrap={true}>
      ```python
      nemo_gym.rollout_collection.RolloutCollectionHelper.run_from_config(
          config: nemo_gym.rollout_collection.RolloutCollectionConfig
      ) -> typing.Tuple[typing.List[typing.Dict]]
      ```
    </CodeBlock>
  </Anchor>

  <Indent>
    <Badge>
      async
    </Badge>
  </Indent>

  <Anchor id="nemo_gym-rollout_collection-RolloutCollectionHelper-setup_server_client">
    <CodeBlock links={{"nemo_gym.config_types.BaseServerConfig":"/nemo-gym/nemo_gym/config_types#nemo_gym-config_types-BaseServerConfig","nemo_gym.server_utils.ServerClient":"/nemo-gym/nemo_gym/server_utils#nemo_gym-server_utils-ServerClient"}} showLineNumbers={false} wordWrap={true}>
      ```python
      nemo_gym.rollout_collection.RolloutCollectionHelper.setup_server_client(
          head_server_config: typing.Optional[nemo_gym.config_types.BaseServerConfig] = None
      ) -> nemo_gym.server_utils.ServerClient
      ```
    </CodeBlock>
  </Anchor>

  <Indent />
</Indent>

<Anchor id="nemo_gym-rollout_collection-SharedRolloutCollectionConfig">
  <CodeBlock showLineNumbers={false} wordWrap={true}>
    ```python
    class nemo_gym.rollout_collection.SharedRolloutCollectionConfig()
    ```
  </CodeBlock>
</Anchor>

<Indent>
  **Bases:** [BaseNeMoGymCLIConfig](/nemo-gym/nemo_gym/config_types#nemo_gym-config_types-BaseNeMoGymCLIConfig)

  <ParamField path="disable_aggregation" type="bool" />

  <ParamField path="num_samples_in_parallel" type="Optional[int]" />

  <ParamField path="output_jsonl_fpath" type="str" />

  <ParamField path="responses_create_params" type="Dict[str, Any]" />

  <ParamField path="upload_rollouts_to_wandb" type="bool" />
</Indent>

<Anchor id="nemo_gym-rollout_collection-_expand_input_glob">
  <CodeBlock showLineNumbers={false} wordWrap={true}>
    ```python
    nemo_gym.rollout_collection._expand_input_glob(
        input_glob: str
    ) -> typing.List[str]
    ```
  </CodeBlock>
</Anchor>

<Indent>
  Expand a glob-or-comma-separated-globs string into a sorted, deduplicated list of paths.

  **Examples:**

  <CodeBlock showLineNumbers={false}>
    ```python
    'results/rollouts.jsonl' -> ['results/rollouts.jsonl'] (if it exists)
    'a/*.jsonl, b/*.jsonl'   -> matches of both patterns, deduplicated
    ```
  </CodeBlock>
</Indent>

<Anchor id="nemo_gym-rollout_collection-_rollout_request_debug_summary">
  <CodeBlock showLineNumbers={false} wordWrap={true}>
    ```python
    nemo_gym.rollout_collection._rollout_request_debug_summary(
        row: typing.Dict[str, typing.Any]
    ) -> typing.Dict[str, typing.Any]
    ```
  </CodeBlock>
</Anchor>

<Indent />

<Anchor id="nemo_gym-rollout_collection-aggregate_rollouts">
  <CodeBlock showLineNumbers={false} wordWrap={true}>
    ```python
    nemo_gym.rollout_collection.aggregate_rollouts()
    ```
  </CodeBlock>
</Anchor>

<Indent />

<Anchor id="nemo_gym-rollout_collection-collect_rollouts">
  <CodeBlock showLineNumbers={false} wordWrap={true}>
    ```python
    nemo_gym.rollout_collection.collect_rollouts()
    ```
  </CodeBlock>
</Anchor>

<Indent />