nemo_automodel.components.datasets.llm

Subpackages

• nemo_automodel.components.datasets.llm.megatron
  • nemo_automodel.components.datasets.llm.megatron.builder
  • nemo_automodel.components.datasets.llm.megatron.gpt_dataset
  • nemo_automodel.components.datasets.llm.megatron.helpers
  • nemo_automodel.components.datasets.llm.megatron.indexed_dataset
  • nemo_automodel.components.datasets.llm.megatron.megatron_utils
  • nemo_automodel.components.datasets.llm.megatron.sampler

Submodules

• nemo_automodel.components.datasets.llm.chat_dataset
• nemo_automodel.components.datasets.llm.column_mapped_text_instruction_dataset
• nemo_automodel.components.datasets.llm.formatting_utils
• nemo_automodel.components.datasets.llm.hellaswag
• nemo_automodel.components.datasets.llm.megatron_dataset
• nemo_automodel.components.datasets.llm.mock
• nemo_automodel.components.datasets.llm.mock_iterable_dataset
• nemo_automodel.components.datasets.llm.mock_packed
• nemo_automodel.components.datasets.llm.nanogpt_dataset
• nemo_automodel.components.datasets.llm.packed_sequence
• nemo_automodel.components.datasets.llm.seq_cls
• nemo_automodel.components.datasets.llm.squad

Package Contents

Data

__all__

API

nemo_automodel.components.datasets.llm.__all__

[‘NanogptDataset’, ‘make_squad_dataset’, ‘ColumnMappedTextInstructionDataset’, ‘ChatDataset’]