> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/automodel/_mcp/server.

# nemo_automodel.components.datasets

## Subpackages

* **[`nemo_automodel.components.datasets.audio`](/nemo-automodel/nemo_automodel/components/datasets/audio)**
* **[`nemo_automodel.components.datasets.diffusion`](/nemo-automodel/nemo_automodel/components/datasets/diffusion)**
* **[`nemo_automodel.components.datasets.dllm`](/nemo-automodel/nemo_automodel/components/datasets/dllm)**
* **[`nemo_automodel.components.datasets.llm`](/nemo-automodel/nemo_automodel/components/datasets/llm)**
* **[`nemo_automodel.components.datasets.multimodal`](/nemo-automodel/nemo_automodel/components/datasets/multimodal)**
* **[`nemo_automodel.components.datasets.vlm`](/nemo-automodel/nemo_automodel/components/datasets/vlm)**

## Submodules

* **[`nemo_automodel.components.datasets.datum`](/nemo-automodel/nemo_automodel/components/datasets/datum)**
* **[`nemo_automodel.components.datasets.lazy_mapped_dataset`](/nemo-automodel/nemo_automodel/components/datasets/lazy_mapped_dataset)**
* **[`nemo_automodel.components.datasets.loader`](/nemo-automodel/nemo_automodel/components/datasets/loader)**
* **[`nemo_automodel.components.datasets.packed_seq`](/nemo-automodel/nemo_automodel/components/datasets/packed_seq)**
* **[`nemo_automodel.components.datasets.packing`](/nemo-automodel/nemo_automodel/components/datasets/packing)**
* **[`nemo_automodel.components.datasets.reservoir_sampler`](/nemo-automodel/nemo_automodel/components/datasets/reservoir_sampler)**
* **[`nemo_automodel.components.datasets.utils`](/nemo-automodel/nemo_automodel/components/datasets/utils)**

## Docs

- [nemo_automodel.components.datasets.audio](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/audio.md)
- [nemo_automodel.components.datasets.audio.collate_fns](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/audio/collate_fns.md)
- [nemo_automodel.components.datasets.audio.datasets](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/audio/datasets.md)
- [nemo_automodel.components.datasets.audio.multi_en](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/audio/multi_en.md)
- [nemo_automodel.components.datasets.datum](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/datum.md)
- [nemo_automodel.components.datasets.diffusion](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/diffusion.md)
- [nemo_automodel.components.datasets.diffusion.base_dataset](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/diffusion/base_dataset.md)
- [nemo_automodel.components.datasets.diffusion.collate_fns](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/diffusion/collate_fns.md)
- [nemo_automodel.components.datasets.diffusion.image_edit_dataset](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/diffusion/image_edit_dataset.md)
- [nemo_automodel.components.datasets.diffusion.loader](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/diffusion/loader.md)
- [nemo_automodel.components.datasets.diffusion.meta_files_dataset](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/diffusion/meta_files_dataset.md)
- [nemo_automodel.components.datasets.diffusion.mock_dataloader](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/diffusion/mock_dataloader.md)
- [nemo_automodel.components.datasets.diffusion.multi_tier_bucketing](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/diffusion/multi_tier_bucketing.md)
- [nemo_automodel.components.datasets.diffusion.sampler](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/diffusion/sampler.md)
- [nemo_automodel.components.datasets.diffusion.text_to_image_dataset](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/diffusion/text_to_image_dataset.md)
- [nemo_automodel.components.datasets.diffusion.text_to_video_dataset](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/diffusion/text_to_video_dataset.md)
- [nemo_automodel.components.datasets.dllm](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/dllm.md)
- [nemo_automodel.components.datasets.dllm.collate](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/dllm/collate.md)
- [nemo_automodel.components.datasets.dllm.corruption](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/dllm/corruption.md)
- [nemo_automodel.components.datasets.lazy_mapped_dataset](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/lazy_mapped_dataset.md)
- [nemo_automodel.components.datasets.llm](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/llm.md)
- [nemo_automodel.components.datasets.llm.agent_chat](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/llm/agent_chat.md)
- [nemo_automodel.components.datasets.llm.chat_dataset](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/llm/chat_dataset.md)
- [nemo_automodel.components.datasets.llm.column_mapped_text_instruction_dataset](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/llm/column_mapped_text_instruction_dataset.md)
- [nemo_automodel.components.datasets.llm.column_mapped_text_instruction_iterable_dataset](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/llm/column_mapped_text_instruction_iterable_dataset.md)
- [nemo_automodel.components.datasets.llm.delta_lake_dataset](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/llm/delta_lake_dataset.md)
- [nemo_automodel.components.datasets.llm.dspark_cache](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/llm/dspark_cache.md)
- [nemo_automodel.components.datasets.llm.eagle3](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/llm/eagle3.md)
- [nemo_automodel.components.datasets.llm.eagle3_cache](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/llm/eagle3_cache.md)
- [nemo_automodel.components.datasets.llm.formatting_utils](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/llm/formatting_utils.md)
- [nemo_automodel.components.datasets.llm.hellaswag](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/llm/hellaswag.md)
- [nemo_automodel.components.datasets.llm.length_grouped_sampler](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/llm/length_grouped_sampler.md)
- [nemo_automodel.components.datasets.llm.megatron](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/llm/megatron.md)
- [nemo_automodel.components.datasets.llm.megatron.builder](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/llm/megatron/builder.md)
- [nemo_automodel.components.datasets.llm.megatron.gpt_dataset](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/llm/megatron/gpt_dataset.md)
- [nemo_automodel.components.datasets.llm.megatron.helpers](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/llm/megatron/helpers.md)
- [nemo_automodel.components.datasets.llm.megatron.indexed_dataset](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/llm/megatron/indexed_dataset.md)
- [nemo_automodel.components.datasets.llm.megatron.megatron_utils](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/llm/megatron/megatron_utils.md)
- [nemo_automodel.components.datasets.llm.megatron.sampler](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/llm/megatron/sampler.md)
- [nemo_automodel.components.datasets.llm.megatron_dataset](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/llm/megatron_dataset.md)
- [nemo_automodel.components.datasets.llm.mock](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/llm/mock.md)
- [nemo_automodel.components.datasets.llm.mock_iterable_dataset](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/llm/mock_iterable_dataset.md)
- [nemo_automodel.components.datasets.llm.mock_packed](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/llm/mock_packed.md)
- [nemo_automodel.components.datasets.llm.mock_prefix_tree](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/llm/mock_prefix_tree.md)
- [nemo_automodel.components.datasets.llm.mock_seq_cls](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/llm/mock_seq_cls.md)
- [nemo_automodel.components.datasets.llm.nanogpt_dataset](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/llm/nanogpt_dataset.md)
- [nemo_automodel.components.datasets.llm.neat_packing](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/llm/neat_packing.md)
- [nemo_automodel.components.datasets.llm.offline_cache](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/llm/offline_cache.md)
- [nemo_automodel.components.datasets.llm.packed_sequence](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/llm/packed_sequence.md)
- [nemo_automodel.components.datasets.llm.prefix_tree](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/llm/prefix_tree.md)
- [nemo_automodel.components.datasets.llm.retrieval_collator](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/llm/retrieval_collator.md)
- [nemo_automodel.components.datasets.llm.retrieval_dataset](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/llm/retrieval_dataset.md)
- [nemo_automodel.components.datasets.llm.retrieval_dataset_inline](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/llm/retrieval_dataset_inline.md)
- [nemo_automodel.components.datasets.llm.retrieval_dataset_normalized](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/llm/retrieval_dataset_normalized.md)
- [nemo_automodel.components.datasets.llm.retrieval_distill_collator](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/llm/retrieval_distill_collator.md)
- [nemo_automodel.components.datasets.llm.seq_cls](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/llm/seq_cls.md)
- [nemo_automodel.components.datasets.llm.seq2seq](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/llm/seq2seq.md)
- [nemo_automodel.components.datasets.llm.squad](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/llm/squad.md)
- [nemo_automodel.components.datasets.llm.xlam](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/llm/xlam.md)
- [nemo_automodel.components.datasets.loader](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/loader.md)
- [nemo_automodel.components.datasets.multimodal](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/multimodal.md)
- [nemo_automodel.components.datasets.multimodal.collate_fns](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/multimodal/collate_fns.md)
- [nemo_automodel.components.datasets.multimodal.datasets](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/multimodal/datasets.md)
- [nemo_automodel.components.datasets.multimodal.distributed_iterable](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/multimodal/distributed_iterable.md)
- [nemo_automodel.components.datasets.multimodal.interleave](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/multimodal/interleave.md)
- [nemo_automodel.components.datasets.multimodal.loader](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/multimodal/loader.md)
- [nemo_automodel.components.datasets.multimodal.packing](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/multimodal/packing.md)
- [nemo_automodel.components.datasets.multimodal.parquet_utils](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/multimodal/parquet_utils.md)
- [nemo_automodel.components.datasets.multimodal.transforms](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/multimodal/transforms.md)
- [nemo_automodel.components.datasets.multimodal.utils](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/multimodal/utils.md)
- [nemo_automodel.components.datasets.multimodal.video](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/multimodal/video.md)
- [nemo_automodel.components.datasets.packed_seq](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/packed_seq.md)
- [nemo_automodel.components.datasets.packing](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/packing.md)
- [nemo_automodel.components.datasets.reservoir_sampler](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/reservoir_sampler.md)
- [nemo_automodel.components.datasets.utils](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/utils.md)
- [nemo_automodel.components.datasets.vlm](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/vlm.md)
- [nemo_automodel.components.datasets.vlm.collate_fns](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/vlm/collate_fns.md)
- [nemo_automodel.components.datasets.vlm.datasets](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/vlm/datasets.md)
- [nemo_automodel.components.datasets.vlm.dspark_collate](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/vlm/dspark_collate.md)
- [nemo_automodel.components.datasets.vlm.fake_image](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/vlm/fake_image.md)
- [nemo_automodel.components.datasets.vlm.loader](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/vlm/loader.md)
- [nemo_automodel.components.datasets.vlm.mock](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/vlm/mock.md)
- [nemo_automodel.components.datasets.vlm.neat_packing_vlm](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/vlm/neat_packing_vlm.md)
- [nemo_automodel.components.datasets.vlm.pp_media](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/vlm/pp_media.md)
- [nemo_automodel.components.datasets.vlm.samplers](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/vlm/samplers.md)
- [nemo_automodel.components.datasets.vlm.utils](https://docs.nvidia.com/nemo/automodel/v0.5/nemo-automodel/nemo_automodel/components/datasets/vlm/utils.md)