> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/automodel/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/automodel/_mcp/server.

# nemo_automodel.components.datasets.llm.megatron.megatron_utils

## Module Contents

### Functions

| Name                                                                                                         | Description                                                                                                              |
| ------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------ |
| [`compile_helper`](#nemo_automodel-components-datasets-llm-megatron-megatron_utils-compile_helper)           | Compile helper function ar runtime. Make sure this                                                                       |
| [`get_blend_from_list`](#nemo_automodel-components-datasets-llm-megatron-megatron_utils-get_blend_from_list) | Get the megatron.core.datasets.blended\_megatron\_dataset\_config.BlendedMegatronDatasetConfig blend from the blend list |

### Data

[`logger`](#nemo_automodel-components-datasets-llm-megatron-megatron_utils-logger)

### API

```python
nemo_automodel.components.datasets.llm.megatron.megatron_utils.compile_helper()
```

Compile helper function ar runtime. Make sure this
is invoked on a single process.

```python
nemo_automodel.components.datasets.llm.megatron.megatron_utils.get_blend_from_list(
    blend: typing.Optional[typing.List[str]]
) -> typing.Optional[typing.Tuple[typing.List[str], typing.Optional[typing.List[float]]]]
```

Get the megatron.core.datasets.blended\_megatron\_dataset\_config.BlendedMegatronDatasetConfig blend from the blend list

**Parameters:**

The blend list, which can be either (1) a list of prefixes, e.g. \["path/to/dataset\_1\_prefix", "path/to/dataset\_2\_prefix"], or (2) a flattened, zipped list of weights and prefixes, e.g. \["30", "path/to/dataset\_1\_prefix", "70", "path/to/dataset\_2\_prefix"]

**Returns:** `Optional[Tuple[List[str], Optional[List[float]]]]`

Optional\[Tuple\[List\[str], Optional\[List\[float]]]]: The blend, consisting of a list of dataset prefixes and optionally a list of dataset weights, e.g. \[\["path/to/dataset\_1\_prefix", "path/to/dataset\_2\_prefix"], \[30.0, 70.0]].

```python
nemo_automodel.components.datasets.llm.megatron.megatron_utils.logger = logging.getLogger(__name__)
```