> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/automodel/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/automodel/_mcp/server.

# nemo_automodel.components.datasets.multimodal.packing

Packed-sequence iterable for BAGEL training.

## Module Contents

### Classes

| Name                                                                                    | Description                                                                    |
| --------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------ |
| [`PackedDataset`](#nemo_automodel-components-datasets-multimodal-packing-PackedDataset) | Greedy pack of samples drawn from weighted groups into token-budgeted batches. |

### Data

[`DataConfig`](#nemo_automodel-components-datasets-multimodal-packing-DataConfig)

[`logger`](#nemo_automodel-components-datasets-multimodal-packing-logger)

### API

```python
class nemo_automodel.components.datasets.multimodal.packing.PackedDataset(
    data_config: 'BagelDatasetConfig',
    tokenizer: 'PreTrainedTokenizerBase',
    special_tokens: collections.abc.Mapping[str, object],
    local_rank: int,
    world_size: int,
    num_workers: int,
    expected_num_tokens: int = 32768,
    max_num_tokens_per_sample: int = 16384,
    max_num_tokens: int = 36864,
    prefer_buffer_before: int = 16384,
    max_buffer_size: int = 50,
    interpolate_pos: bool = False,
    use_flex: bool = False,
    data_status: object | None = None,
    dataset_info: collections.abc.Mapping[str, object] | None = None,
    global_seed: int | None = None
)
```

**Bases:** `IterableDataset`

Greedy pack of samples drawn from weighted groups into token-budgeted batches.

The dataset reseeds at iterator start so AM sees a deterministic
BAGEL-compatible packed-data stream regardless of earlier RNG consumption
during model construction or checkpoint loading.

```python
nemo_automodel.components.datasets.multimodal.packing.PackedDataset.__iter__()
```

```python
nemo_automodel.components.datasets.multimodal.packing.PackedDataset._grouped_dataset_state_dicts()
```

```python
nemo_automodel.components.datasets.multimodal.packing.PackedDataset._load_grouped_dataset_state_dicts(
    states
)
```

```python
nemo_automodel.components.datasets.multimodal.packing.PackedDataset._load_rng_state_dict(
    state
)
```

```python
nemo_automodel.components.datasets.multimodal.packing.PackedDataset._log_drop(
    reason,
    message,
    args = (),
    every = 100
)
```

```python
nemo_automodel.components.datasets.multimodal.packing.PackedDataset._rng_state_dict()
```

```python
nemo_automodel.components.datasets.multimodal.packing.PackedDataset._set_resume_point(
    buffer,
    yielded_batches
)
```

```python
nemo_automodel.components.datasets.multimodal.packing.PackedDataset.build_datasets(
    datasets_metainfo,
    data_status
)
```

```python
nemo_automodel.components.datasets.multimodal.packing.PackedDataset.load_state_dict(
    state_dict
)
```

```python
nemo_automodel.components.datasets.multimodal.packing.PackedDataset.pack_sequence(
    sample,
    sequence_status
)
```

```python
nemo_automodel.components.datasets.multimodal.packing.PackedDataset.set_epoch(
    seed
)
```

```python
nemo_automodel.components.datasets.multimodal.packing.PackedDataset.set_sequence_status()
```

```python
nemo_automodel.components.datasets.multimodal.packing.PackedDataset.state_dict()
```

```python
nemo_automodel.components.datasets.multimodal.packing.PackedDataset.to_tensor(
    sequence_status
)
```

```python
nemo_automodel.components.datasets.multimodal.packing.DataConfig = BagelDatasetConfig
```

```python
nemo_automodel.components.datasets.multimodal.packing.logger = logging.getLogger(__name__)
```