.. include:: /content/common.rsts

Release Notes |ndash| Release 0.6.0 (BETA)
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Key Features and Enhancements
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

Added ONNX export support for Transformer Engine modules.

Added the ability to calibrate the Linear layer for FP8 from a higher-precision checkpoint.

Added a ``zero_centered_gamma`` option to ``LayerNorm()`` (and other methods using ``LayerNorm()``) for increased precision when the parameters of ``LayerNorm()`` are stored in ``FP16`` or ``BFloat16``.

Added ``DotProductAttention()``, with optimized implementations of attention, including integration with ``FlashAttention()``.

Optimized handling of separate QKV parameters in ``TransformerLayer()``.


Fixed Issues in This Release
@@@@@@@@@@@@@@@@@@@@@@@@@@@@

Fixed an issue with gradients not being propagated properly when training under AMP.

Fixed an issue occurring for very large inputs in the custom Softmax kernel.


Known Issues in This Release
@@@@@@@@@@@@@@@@@@@@@@@@@@@@

There are no known issues in this release.


Breaking Changes in This Release
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

There are no breaking changes in this release.


Deprecated Features
@@@@@@@@@@@@@@@@@@@

There are no deprecated features in this release.