nemo_automodel.components.datasets.llm.seq_cls

Module Contents

Classes

Name	Description
`GLUE_MRPC`	GLUE MRPC dataset (sentence pair classification).
`GLUE_MRPCConfig`	Construction-time configuration for :class:`GLUE_MRPC`.

API

class nemo_automodel.components.datasets.llm.seq_cls.GLUE_MRPC(
    tokenizer,
    split: str = 'train',
    num_samples_limit: typing.Optional[int] = None,
    trust_remote_code: bool = True,
    max_length: typing.Optional[int] = 256
)

GLUE MRPC dataset (sentence pair classification).

Produces tokenized inputs with both sentence1 and sentence2 using the provided tokenizer.

dataset

nemo_automodel.components.datasets.llm.seq_cls.GLUE_MRPC.__getitem__(
    idx
)

nemo_automodel.components.datasets.llm.seq_cls.GLUE_MRPC.__len__()

class nemo_automodel.components.datasets.llm.seq_cls.GLUE_MRPCConfig(
    split: str = 'train',
    num_samples_limit: int | None = None,
    trust_remote_code: bool = True,
    max_length: int | None = 256
)

Dataclass

Construction-time configuration for :class:GLUE_MRPC.

accepts_tokenizer

bool = True

max_length

int | None = 256

Maximum tokenization length (None derives one from the tokenizer).

num_samples_limit

int | None = None

If set, limit the dataset to this many samples via slicing.

split

str = 'train'

Dataset split to use (e.g. train, validation).

trust_remote_code

bool = True

Forwarded to datasets.load_dataset.

nemo_automodel.components.datasets.llm.seq_cls.GLUE_MRPCConfig.build(
    tokenizer: 'PreTrainedTokenizerBase | None'
) -> 'GLUE_MRPC'

Build a :class:GLUE_MRPC dataset from this :class:GLUE_MRPCConfig and a runtime tokenizer.