nemo_automodel.components.datasets.llm.mock_seq_cls

Module Contents

Classes

Name	Description
`MockSequenceClassificationDataset`	Mock dataset for sequence classification functional tests.
`MockSequenceClassificationDatasetConfig`	Construction-time configuration for :class:`MockSequenceClassificationDataset`.

API

class nemo_automodel.components.datasets.llm.mock_seq_cls.MockSequenceClassificationDataset(
    num_samples: int = 64,
    num_labels: int = 2,
    vocab_size: int = 256,
    max_seq_len: int = 32,
    seed: int = 0,
    tokenizer = None
)

Bases: Dataset

Mock dataset for sequence classification functional tests.

Generates random token sequences with binary labels. Does not require a tokenizer or network access.

samples

= []

nemo_automodel.components.datasets.llm.mock_seq_cls.MockSequenceClassificationDataset.__getitem__(
    idx
)

nemo_automodel.components.datasets.llm.mock_seq_cls.MockSequenceClassificationDataset.__len__()

class nemo_automodel.components.datasets.llm.mock_seq_cls.MockSequenceClassificationDatasetConfig(
    num_samples: int = 64,
    num_labels: int = 2,
    vocab_size: int = 256,
    max_seq_len: int = 32,
    seed: int = 0
)

Dataclass

Construction-time configuration for :class:MockSequenceClassificationDataset.

accepts_tokenizer

bool = True

max_seq_len

int = 32

Maximum sequence length (each sample length is sampled in [4, max_seq_len]).

num_labels

int = 2

Number of classification labels.

num_samples

int = 64

Number of synthetic samples to generate.

seed

int = 0

Seed for the random generator.

vocab_size

int = 256

Vocabulary size for the random token ids.

nemo_automodel.components.datasets.llm.mock_seq_cls.MockSequenceClassificationDatasetConfig.build(
    tokenizer: 'PreTrainedTokenizerBase | None' = None
) -> 'MockSequenceClassificationDataset'

Build a :class:MockSequenceClassificationDataset from this :class:MockSequenceClassificationDatasetConfig.