> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/automodel/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/automodel/_mcp/server.

# nemo_automodel.components.datasets.vlm.loader

Typed construction for VLM processors, datasets, packing, and dataloaders.

## Module Contents

### Classes

| Name                                                                                              | Description                                                         |
| ------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------- |
| [`RankedVlmDatasetConfig`](#nemo_automodel-components-datasets-vlm-loader-RankedVlmDatasetConfig) | Dataset config whose build requires the runtime data-parallel rank. |
| [`VlmCollatorConfig`](#nemo_automodel-components-datasets-vlm-loader-VlmCollatorConfig)           | Declarative VLM collator factory.                                   |
| [`VlmDataloaderBuild`](#nemo_automodel-components-datasets-vlm-loader-VlmDataloaderBuild)         | Materialized VLM dataloader and its processor.                      |
| [`VlmDataloaderConfig`](#nemo_automodel-components-datasets-vlm-loader-VlmDataloaderConfig)       | Typed construction config for the complete VLM input pipeline.      |
| [`VlmProcessorConfig`](#nemo_automodel-components-datasets-vlm-loader-VlmProcessorConfig)         | Declarative processor factory and keyword arguments.                |

### Data

[`VlmCollateFn`](#nemo_automodel-components-datasets-vlm-loader-VlmCollateFn)

[`__all__`](#nemo_automodel-components-datasets-vlm-loader-__all__)

[`logger`](#nemo_automodel-components-datasets-vlm-loader-logger)

### API

```python
class nemo_automodel.components.datasets.vlm.loader.RankedVlmDatasetConfig()
```

Protocol

Dataset config whose build requires the runtime data-parallel rank.

```python
nemo_automodel.components.datasets.vlm.loader.RankedVlmDatasetConfig.build(
    rank: int,
    world_size: int
) -> object
```

Build the per-rank dataset shard.

```python
class nemo_automodel.components.datasets.vlm.loader.VlmCollatorConfig(
    factory: collections.abc.Callable[..., object],
    kwargs: dict[str, object] = dict()
)
```

Dataclass

Declarative VLM collator factory.

Function that accepts `examples` and the runtime `processor`.

Declarative keyword arguments bound once while building the dataloader.

```python
nemo_automodel.components.datasets.vlm.loader.VlmCollatorConfig.build(
    processor: transformers.ProcessorMixin | None
) -> nemo_automodel.components.datasets.vlm.loader.VlmCollateFn
```

Bind the runtime processor to the configured collator.

```python
class nemo_automodel.components.datasets.vlm.loader.VlmDataloaderBuild(
    dataloader: torch.utils.data.DataLoader,
    processor: transformers.ProcessorMixin | None
)
```

Dataclass

Materialized VLM dataloader and its processor.

```python
class nemo_automodel.components.datasets.vlm.loader.VlmDataloaderConfig(
    dataset_config: nemo_automodel.components.datasets.loader.DatasetConfig | nemo_automodel.components.datasets.vlm.loader.RankedVlmDatasetConfig,
    processor_config: nemo_automodel.components.datasets.vlm.loader.VlmProcessorConfig = VlmProcessorConfig(),
    pretokenization: nemo_automodel.components.datasets.vlm.datasets.PreTokenizedDatasetWrapperConfig | None = None,
    packing: nemo_automodel.components.datasets.vlm.neat_packing_vlm.NeatPackConfig | None = None,
    collator: nemo_automodel.components.datasets.vlm.loader.VlmCollatorConfig | None = None,
    chat_template: str | None = None,
    shuffle: bool = True,
    num_workers: int = 0,
    pin_memory: bool = False,
    persistent_workers: bool = False,
    prefetch_factor: int | None = None,
    drop_last: bool = False
)
```

Dataclass

Typed construction config for the complete VLM input pipeline.

```python
nemo_automodel.components.datasets.vlm.loader.VlmDataloaderConfig._build_source(
    pretrained_model_name_or_path: str,
    dp_rank: int,
    dp_world_size: int,
    dataset_build_context: contextlib.AbstractContextManager[object] | None
) -> tuple[object, transformers.ProcessorMixin | None]
```

Build the processor and raw dataset under the caller-owned ordering context.

```python
nemo_automodel.components.datasets.vlm.loader.VlmDataloaderConfig.build(
    pretrained_model_name_or_path: str,
    dp_rank: int,
    dp_world_size: int,
    batch_size: int,
    dataset_build_context: contextlib.AbstractContextManager[object] | None = None,
    get_rope_index: collections.abc.Callable[..., object] | None = None,
    packing_attn_implementation: str | None = None,
    pp_n_microbatches: int | None = None
) -> nemo_automodel.components.datasets.vlm.loader.VlmDataloaderBuild
```

Build the processor, dataset wrappers, sampler, collator, and dataloader.

**Parameters:**

Runtime model identifier used to build the processor.

Rank within the data-parallel group.

Size of the data-parallel group.

Runtime local training batch size.

Optional rank-ordering context used only for processor and source-dataset build.

Optional model callback used to create packed multimodal position IDs.

Resolved attention backend for packed-mask construction.

Optional pipeline microbatch count used to pre-chunk media tensors.

**Returns:** `VlmDataloaderBuild`

Named result containing the stateful dataloader and runtime processor.

```python
nemo_automodel.components.datasets.vlm.loader.VlmDataloaderConfig.resolve_packing_attn_implementation(
    model_attn_implementation: str | None,
    cp_size: int
) -> str | None
```

Resolve the packed-collator mask backend.

**Parameters:**

Attention implementation selected by the model config.

Runtime context-parallel world size.

**Returns:** `str | None`

Attention implementation used to choose the packed attention-mask representation.

```python
class nemo_automodel.components.datasets.vlm.loader.VlmProcessorConfig(
    factory: collections.abc.Callable[..., transformers.ProcessorMixin] | None = None,
    kwargs: dict[str, object] = dict()
)
```

Dataclass

Declarative processor factory and keyword arguments.

Configured processor factory; `None` selects `AutoProcessor.from_pretrained`.

Declarative keyword arguments for the configured processor factory.

```python
nemo_automodel.components.datasets.vlm.loader.VlmProcessorConfig.build(
    pretrained_model_name_or_path: str
) -> transformers.ProcessorMixin | None
```

Build the configured processor.

**Parameters:**

Runtime model identifier used by the default AutoProcessor factory.

**Returns:** `ProcessorMixin | None`

Processor instance, or `None` when the model has no compatible AutoProcessor.

```python
nemo_automodel.components.datasets.vlm.loader.VlmCollateFn = Callable[[list[object]], object]
```

```python
nemo_automodel.components.datasets.vlm.loader.__all__ = ['RankedVlmDatasetConfig', 'VlmCollatorConfig', 'VlmDataloaderBuild', 'VlmDatalo...
```

```python
nemo_automodel.components.datasets.vlm.loader.logger = logging.getLogger(__name__)
```