Common Collection

The common collection contains things that could be used across all collections.

• Callbacks
  • Exponential Moving Average (EMA)
• Losses
  • AggregatorLoss
    • AggregatorLoss.__init__()
    • AggregatorLoss.forward()
    • AggregatorLoss.input_types
    • AggregatorLoss.output_types
  • CrossEntropyLoss
    • CrossEntropyLoss.__init__()
    • CrossEntropyLoss.forward()
    • CrossEntropyLoss.input_types
    • CrossEntropyLoss.output_types
  • MSELoss
    • MSELoss.__init__()
    • MSELoss.forward()
    • MSELoss.input_types
    • MSELoss.output_types
  • SmoothedCrossEntropyLoss
    • SmoothedCrossEntropyLoss.__init__()
    • SmoothedCrossEntropyLoss.forward()
    • SmoothedCrossEntropyLoss.input_types
    • SmoothedCrossEntropyLoss.output_types
  • SpanningLoss
    • SpanningLoss.__init__()
    • SpanningLoss.forward()
    • SpanningLoss.input_types
    • SpanningLoss.output_types
• Metrics
  • Perplexity
    • Perplexity.compute()
    • Perplexity.full_state_update
    • Perplexity.update()
• Tokenizers
  • AutoTokenizer
    • AutoTokenizer.__init__()
    • AutoTokenizer.add_special_tokens()
    • AutoTokenizer.additional_special_tokens_ids
    • AutoTokenizer.bos_id
    • AutoTokenizer.cls_id
    • AutoTokenizer.eos_id
    • AutoTokenizer.ids_to_text()
    • AutoTokenizer.ids_to_tokens()
    • AutoTokenizer.mask_id
    • AutoTokenizer.name
    • AutoTokenizer.pad_id
    • AutoTokenizer.save_vocabulary()
    • AutoTokenizer.sep_id
    • AutoTokenizer.text_to_ids()
    • AutoTokenizer.text_to_tokens()
    • AutoTokenizer.token_to_id()
    • AutoTokenizer.tokens_to_ids()
    • AutoTokenizer.tokens_to_text()
    • AutoTokenizer.unk_id
    • AutoTokenizer.vocab
    • AutoTokenizer.vocab_size
  • SentencePieceTokenizer
    • SentencePieceTokenizer.__init__()
    • SentencePieceTokenizer.add_special_tokens()
    • SentencePieceTokenizer.additional_special_tokens_ids
    • SentencePieceTokenizer.bos_id
    • SentencePieceTokenizer.cls_id
    • SentencePieceTokenizer.eos_id
    • SentencePieceTokenizer.ids_to_text()
    • SentencePieceTokenizer.ids_to_tokens()
    • SentencePieceTokenizer.mask_id
    • SentencePieceTokenizer.pad_id
    • SentencePieceTokenizer.sep_id
    • SentencePieceTokenizer.text_to_ids()
    • SentencePieceTokenizer.text_to_tokens()
    • SentencePieceTokenizer.token_to_id()
    • SentencePieceTokenizer.tokens_to_ids()
    • SentencePieceTokenizer.tokens_to_text()
    • SentencePieceTokenizer.unk_id
    • SentencePieceTokenizer.vocab
  • TokenizerSpec
    • TokenizerSpec.add_special_tokens()
    • TokenizerSpec.ids_to_text()
    • TokenizerSpec.ids_to_tokens()
    • TokenizerSpec.name
    • TokenizerSpec.text_to_ids()
    • TokenizerSpec.text_to_tokens()
    • TokenizerSpec.tokens_to_ids()
    • TokenizerSpec.tokens_to_text()
• Data
  • ConcatDataset
    • ConcatDataset.get_iterable()
    • ConcatDataset.random_generator()
    • ConcatDataset.round_robin_generator()
    • ConcatDataset.temperature_generator()
  • ConcatMapDataset