Release Notes Release 0.5.0 (BETA)

Key Features and Enhancements

Added a C API for RMSNorm and GeGLU operations as well as their gradients.

Added support for NVTX markers.

Fixed Issues in This Release

Fixed an issue preventing Transformer Engine from work with an interleaved pipeline parallelism scheme.

Fixed an issue which resulted in increasing checkpoint size during evaluation when using activation checkpointing.

Known Issues in This Release

There are no known issues in this release.

Breaking Changes in This Release

There are no breaking changes in this release.

Deprecated Features

There are no deprecated features in this release.