core.activations

Module Contents

Functions

squared_relu	Squared ReLU activation
quick_gelu	Quick GELU activation
fast_gelu	Fast GELU activation

API

core.activations.squared_relu(x: torch.Tensor) → torch.Tensor

Squared ReLU activation

core.activations.quick_gelu(x: torch.Tensor) → torch.Tensor

Quick GELU activation

core.activations.fast_gelu(x: torch.Tensor) → torch.Tensor

Fast GELU activation