Transformer Engine
0.1.0
Home
Getting Started
Installation
Prerequisites
Transformer Engine in NGC Containers
pip - from GitHub
Additional Prerequisites
Installation (stable release)
Installation (development build)
Getting Started
Overview
Let’s build a Transformer layer!
Meet Transformer Engine
Fused TE Modules
Enabling FP8
Python API documentation
Common API
Classes
Framework-specific API
pyTorch
Modules
Functions
Examples and Tutorials
Using FP8 with Transformer Engine
Introduction to FP8
Structure
Mixed precision training - a quick introduction
Mixed precision training with FP8
Using FP8 with Transformer Engine
FP8 recipe
FP8 autocasting
Handling backward pass
Precision
Advanced
C/C++ API
activation.h
cast.h
gemm.h
layer_norm.h
transformer_engine.h
transpose.h
Transformer Engine
»
Framework-specific API
View page source
Framework-specific API
¶
pyTorch
Modules
Functions