Common Collection¶

The common collection contains things that could be used across all collections.

Tokenizers¶

Wrapper of HuggingFace AutoTokenizer https://huggingface.co/transformers/model_doc/auto.html#autotokenizer.

nemo.collections.common.tokenizers.AutoTokenizer.bos_id¶

nemo.collections.common.tokenizers.AutoTokenizer.cls_id¶

nemo.collections.common.tokenizers.AutoTokenizer.eos_id¶

nemo.collections.common.tokenizers.AutoTokenizer.mask_id¶

nemo.collections.common.tokenizers.AutoTokenizer.name¶

nemo.collections.common.tokenizers.AutoTokenizer.pad_id¶

nemo.collections.common.tokenizers.AutoTokenizer.sep_id¶

nemo.collections.common.tokenizers.AutoTokenizer.unk_id¶

nemo.collections.common.tokenizers.AutoTokenizer.vocab_size¶

Sentencepiecetokenizer https://github.com/google/sentencepiece.

nemo.collections.common.tokenizers.SentencePieceTokenizer.bos_id¶

nemo.collections.common.tokenizers.SentencePieceTokenizer.cls_id¶

nemo.collections.common.tokenizers.SentencePieceTokenizer.eos_id¶

nemo.collections.common.tokenizers.SentencePieceTokenizer.name¶

nemo.collections.common.tokenizers.SentencePieceTokenizer.pad_id¶

nemo.collections.common.tokenizers.SentencePieceTokenizer.sep_id¶

Inherit this class to implement a new tokenizer.

nemo.collections.common.tokenizers.TokenizerSpec.__init__¶: Initialize self. See help(type(self)) for accurate signature.

nemo.collections.common.tokenizers.TokenizerSpec.name¶

Losses¶

Sums several losses into one.

param num_inputs:
	number of input losses
param weights:	a list of coefficient for merging losses

nemo.collections.common.losses.AggregatorLoss.input_types¶: Returns definitions of module input ports.

nemo.collections.common.losses.AggregatorLoss.output_types¶: Returns definitions of module output ports.

CrossEntropyLoss

nemo.collections.common.losses.CrossEntropyLoss.input_types¶: Returns definitions of module input ports.

nemo.collections.common.losses.CrossEntropyLoss.output_types¶: Returns definitions of module output ports.

MSELoss

nemo.collections.common.losses.MSELoss.input_types¶: Returns definitions of module input ports.

nemo.collections.common.losses.MSELoss.output_types¶: Returns definitions of module output ports.

Calculates Cross-entropy loss with label smoothing for a batch of sequences.

SmoothedCrossEntropyLoss: 1) excludes padding tokens from loss calculation 2) allows to use label smoothing regularization 3) allows to calculate loss for the desired number of last tokens

param label_smoothing:
	label smoothing regularization coefficient
type label_smoothing:
	float
param predict_last_k:
	parameter which sets the number of last tokens to calculate the loss for, for example 0: (default) calculate loss on the entire sequence (e.g., NMT) 1: calculate loss on the last token only (e.g., LM evaluation) Intermediate values allow to control the trade-off between eval time (proportional to the number of batches) and eval performance (proportional to the number of context tokens)
type predict_last_k:
	int
param pad_id:	padding id
type pad_id:	int
param eps:	the small eps number to avoid division buy zero
type eps:	float

nemo.collections.common.losses.SmoothedCrossEntropyLoss.input_types¶: Returns definitions of module input ports.

nemo.collections.common.losses.SmoothedCrossEntropyLoss.output_types¶: Returns definitions of module output ports.

implements start and end loss of a span e.g. for Question Answering.

nemo.collections.common.losses.SpanningLoss.input_types¶: Returns definitions of module input ports.

nemo.collections.common.losses.SpanningLoss.output_types¶: Returns definitions of module output ports.