• Overview
• Release 0.1.0 (BETA)
• Release 0.2.0 (BETA)

Overview

Transformer Engine (TE) is a library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit floating point (FP8) precision on Hopper GPUs, to provide better performance with lower memory utilization in both training and inference.

TE provides a collection of highly optimized building blocks for popular Transformer architectures and an automatic mixed precision-like API that can be used seamlessly with your PyTorch code.

TE also includes a framework-agnostic C++ API that can be integrated with other deep learning libraries to enable FP8 support for Transformers.