Transformer Engine
0.2.0 -3b16125
Version select:
  • Home

Getting Started

  • Installation
    • Prerequisites
    • Transformer Engine in NGC Containers
    • pip - from GitHub
      • Additional Prerequisites
      • Installation (stable release)
      • Installation (development build)
  • Getting Started
    • Overview
    • Let’s build a Transformer layer!
    • Meet Transformer Engine
    • Fused TE Modules
    • Enabling FP8

Python API documentation

  • Common API
    • Classes
      • Format
      • DelayedScaling
  • Framework-specific API
    • pyTorch
      • Modules
        • Linear
        • LayerNorm
        • LayerNormLinear
        • LayerNormMLP
        • TransformerLayer
      • Functions
        • fp8_autocast()

Examples and Tutorials

  • Using FP8 with Transformer Engine
    • Introduction to FP8
      • Structure
      • Mixed precision training - a quick introduction
      • Mixed precision training with FP8
    • Using FP8 with Transformer Engine
      • FP8 recipe
      • FP8 autocasting
      • Handling backward pass
      • Precision

Advanced

  • C/C++ API
    • activation.h
    • cast.h
    • gemm.h
    • layer_norm.h
    • transformer_engine.h
    • transpose.h
Transformer Engine
  • »
  • Search

© Copyright 2022, NVIDIA CORPORATION & AFFILIATES. All rights reserved..

Built with Sphinx using a theme provided by Read the Docs.