tasks.metrics

Module Contents

Classes

ANLI	Helper class that provides a standard way to create an ABC using inheritance.
ArcChallenge	Helper class that provides a standard way to create an ABC using inheritance.
ArcEasy	Helper class that provides a standard way to create an ABC using inheritance.
BigBenchHard	Helper class that provides a standard way to create an ABC using inheritance.
BigBenchLight	Helper class that provides a standard way to create an ABC using inheritance.
BoolQ	Helper class that provides a standard way to create an ABC using inheritance.
CB	Helper class that provides a standard way to create an ABC using inheritance.
COQA	Helper class that provides a standard way to create an ABC using inheritance.
Copa	Helper class that provides a standard way to create an ABC using inheritance.
Drop	Helper class that provides a standard way to create an ABC using inheritance.
Lambada	Helper class that provides a standard way to create an ABC using inheritance.
MMLU	Helper class that provides a standard way to create an ABC using inheritance.
MultiRC	Helper class that provides a standard way to create an ABC using inheritance.
Multilingual	Helper class that provides a standard way to create an ABC using inheritance.
NumDasc	Helper class that provides a standard way to create an ABC using inheritance.
OpenBookQA	Helper class that provides a standard way to create an ABC using inheritance.
PIQA	Helper class that provides a standard way to create an ABC using inheritance.
Quac	Helper class that provides a standard way to create an ABC using inheritance.
RTE	Helper class that provides a standard way to create an ABC using inheritance.
Race	Helper class that provides a standard way to create an ABC using inheritance.
Record	Helper class that provides a standard way to create an ABC using inheritance.
Squad	Helper class that provides a standard way to create an ABC using inheritance.
StoryCloze	Helper class that provides a standard way to create an ABC using inheritance.
TriviaQA	Helper class that provides a standard way to create an ABC using inheritance.
WSC	Helper class that provides a standard way to create an ABC using inheritance.
WebQA	Helper class that provides a standard way to create an ABC using inheritance.
WiC	Helper class that provides a standard way to create an ABC using inheritance.
Winogrande	Helper class that provides a standard way to create an ABC using inheritance.

API

class tasks.metrics.ANLI(min_ngram_size: int = 8, max_ngram_size: int = 13)

Bases: nemo_curator.tasks.downstream_task.DownstreamTask

Helper class that provides a standard way to create an ABC using inheritance.

Initialization

generate_ngrams() → dict[str, int]

class tasks.metrics.ArcChallenge(min_ngram_size: int = 8, max_ngram_size: int = 13)

Bases: nemo_curator.tasks.downstream_task.DownstreamTask

Helper class that provides a standard way to create an ABC using inheritance.

Initialization

generate_ngrams() → dict[str, int]

class tasks.metrics.ArcEasy(min_ngram_size: int = 8, max_ngram_size: int = 13)

Bases: nemo_curator.tasks.downstream_task.DownstreamTask

Helper class that provides a standard way to create an ABC using inheritance.

Initialization

generate_ngrams() → dict[str, int]

class tasks.metrics.BigBenchHard(

path: str | None = None,

min_ngram_size: int = 8,

max_ngram_size: int = 13,

)

Bases: nemo_curator.tasks.downstream_task.DownstreamTask

Helper class that provides a standard way to create an ABC using inheritance.

Initialization

generate_ngrams() → dict[str, int]

class tasks.metrics.BigBenchLight(

path: str | None = None,

min_ngram_size: int = 8,

max_ngram_size: int = 13,

)

Bases: nemo_curator.tasks.downstream_task.DownstreamTask

Helper class that provides a standard way to create an ABC using inheritance.

Initialization

generate_ngrams() → dict[str, int]

class tasks.metrics.BoolQ(min_ngram_size: int = 8, max_ngram_size: int = 13)

Bases: nemo_curator.tasks.downstream_task.DownstreamTask

Helper class that provides a standard way to create an ABC using inheritance.

Initialization

generate_ngrams() → dict[str, int]

class tasks.metrics.CB(min_ngram_size: int = 8, max_ngram_size: int = 13)

Bases: nemo_curator.tasks.downstream_task.DownstreamTask

Helper class that provides a standard way to create an ABC using inheritance.

Initialization

generate_ngrams() → dict[str, int]

class tasks.metrics.COQA(

file_path: str | None = None,

min_ngram_size: int = 8,

max_ngram_size: int = 13,

)

Bases: nemo_curator.tasks.downstream_task.DownstreamTask

Helper class that provides a standard way to create an ABC using inheritance.

Initialization

generate_ngrams() → dict[str, int]

class tasks.metrics.Copa(min_ngram_size: int = 8, max_ngram_size: int = 13)

Bases: nemo_curator.tasks.downstream_task.DownstreamTask

Helper class that provides a standard way to create an ABC using inheritance.

Initialization

generate_ngrams() → dict[str, int]

class tasks.metrics.Drop(min_ngram_size: int = 8, max_ngram_size: int = 13)

Bases: nemo_curator.tasks.downstream_task.DownstreamTask

Helper class that provides a standard way to create an ABC using inheritance.

Initialization

generate_ngrams() → dict[str, int]

class tasks.metrics.Lambada(

file_path: str,

min_ngram_size: int = 8,

max_ngram_size: int = 13,

)

Bases: nemo_curator.tasks.downstream_task.DownstreamTask

Helper class that provides a standard way to create an ABC using inheritance.

Initialization

generate_ngrams() → dict[str, int]

class tasks.metrics.MMLU(

path: str | None = None,

min_ngram_size: int = 8,

max_ngram_size: int = 13,

)

Bases: nemo_curator.tasks.downstream_task.DownstreamTask

Helper class that provides a standard way to create an ABC using inheritance.

Initialization

generate_ngrams() → dict[str, int]

class tasks.metrics.MultiRC(min_ngram_size: int = 8, max_ngram_size: int = 13)

Bases: nemo_curator.tasks.downstream_task.DownstreamTask

Helper class that provides a standard way to create an ABC using inheritance.

Initialization

generate_ngrams() → dict[str, int]

class tasks.metrics.Multilingual(

path: str | None = None,

min_ngram_size: int = 8,

max_ngram_size: int = 13,

)

Bases: nemo_curator.tasks.downstream_task.DownstreamTask

Helper class that provides a standard way to create an ABC using inheritance.

Initialization

generate_ngrams() → dict[str, int]

class tasks.metrics.NumDasc(

n: int,

file_path: str,

min_ngram_size: int = 8,

max_ngram_size: int = 13,

)

Bases: nemo_curator.tasks.downstream_task.DownstreamTask

Helper class that provides a standard way to create an ABC using inheritance.

Initialization

generate_ngrams() → dict[str, int]

class tasks.metrics.OpenBookQA(min_ngram_size: int = 8, max_ngram_size: int = 13)

Bases: nemo_curator.tasks.downstream_task.DownstreamTask

Helper class that provides a standard way to create an ABC using inheritance.

Initialization

generate_ngrams() → dict[str, int]

class tasks.metrics.PIQA(min_ngram_size: int = 8, max_ngram_size: int = 13)

Bases: nemo_curator.tasks.downstream_task.DownstreamTask

Helper class that provides a standard way to create an ABC using inheritance.

Initialization

generate_ngrams() → dict[str, int]

class tasks.metrics.Quac(min_ngram_size: int = 8, max_ngram_size: int = 13)

Bases: nemo_curator.tasks.downstream_task.DownstreamTask

Helper class that provides a standard way to create an ABC using inheritance.

Initialization

generate_ngrams() → dict[str, int]

class tasks.metrics.RTE(min_ngram_size: int = 8, max_ngram_size: int = 13)

Bases: nemo_curator.tasks.downstream_task.DownstreamTask

Helper class that provides a standard way to create an ABC using inheritance.

Initialization

generate_ngrams() → dict[str, int]

class tasks.metrics.Race(min_ngram_size: int = 8, max_ngram_size: int = 13)

Bases: nemo_curator.tasks.downstream_task.DownstreamTask

Helper class that provides a standard way to create an ABC using inheritance.

Initialization

generate_ngrams() → dict[str, int]

class tasks.metrics.Record(min_ngram_size: int = 8, max_ngram_size: int = 13)

Bases: nemo_curator.tasks.downstream_task.DownstreamTask

Helper class that provides a standard way to create an ABC using inheritance.

Initialization

generate_ngrams() → dict[str, int]

class tasks.metrics.Squad(min_ngram_size: int = 8, max_ngram_size: int = 13)

Bases: nemo_curator.tasks.downstream_task.DownstreamTask

Helper class that provides a standard way to create an ABC using inheritance.

Initialization

generate_ngrams() → dict[str, int]

class tasks.metrics.StoryCloze(

file_path: str,

min_ngram_size: int = 8,

max_ngram_size: int = 13,

)

Bases: nemo_curator.tasks.downstream_task.DownstreamTask

Helper class that provides a standard way to create an ABC using inheritance.

Initialization

generate_ngrams() → dict[str, int]

class tasks.metrics.TriviaQA(min_ngram_size: int = 8, max_ngram_size: int = 13)

Bases: nemo_curator.tasks.downstream_task.DownstreamTask

Helper class that provides a standard way to create an ABC using inheritance.

Initialization

generate_ngrams() → dict[str, int]

class tasks.metrics.WSC(min_ngram_size: int = 8, max_ngram_size: int = 13)

Bases: nemo_curator.tasks.downstream_task.DownstreamTask

Helper class that provides a standard way to create an ABC using inheritance.

Initialization

generate_ngrams() → dict[str, int]

class tasks.metrics.WebQA(min_ngram_size: int = 8, max_ngram_size: int = 13)

Bases: nemo_curator.tasks.downstream_task.DownstreamTask

Helper class that provides a standard way to create an ABC using inheritance.

Initialization

generate_ngrams() → dict[str, int]

class tasks.metrics.WiC(min_ngram_size: int = 8, max_ngram_size: int = 13)

Bases: nemo_curator.tasks.downstream_task.DownstreamTask

Helper class that provides a standard way to create an ABC using inheritance.

Initialization

generate_ngrams() → dict[str, int]

class tasks.metrics.Winogrande(min_ngram_size: int = 8, max_ngram_size: int = 13)

Bases: nemo_curator.tasks.downstream_task.DownstreamTask

Helper class that provides a standard way to create an ABC using inheritance.

Initialization

generate_ngrams() → dict[str, int]