Important

NeMo 2.0 is an experimental feature and currently released in the dev container only: nvcr.io/nvidia/nemo:dev. Please refer to NeMo 2.0 overview for information on getting started.

Software Component Versions

This section identifies the versions of software components and services that are used in this release of the NVIDIA NeMo™ Framework.

NeMo Framework 24.07

Software Component	Version
PyTorch	2.3.0a0
PyTorch Lightning	2.3.3
Megatron Core	0.8
Transformer Engine	1.8.0.dev0
NeMo	2.0.0rc1
NeMo Aligner	0.4.0rc0
NeMo Data Curator	0.4.0
TRT-LLM	0.10.0

NeMo Framework 24.05

Software Component	Version
PyTorch	2.3.0a0
PyTorch Lightning	2.2.4
Megatron Core	0.7.0
Transformer Engine	1.6.0.dev0
NeMo	2.0.0rc0
NeMo Aligner	0.3.0.dev0
NeMo Data Curator	0.3.0

NVIDIA NeMo™ Framework Training container is built on top of NVIDIA Optimized Frameworks PyTorch 24.02 container: https://docs.nvidia.com/deeplearning/frameworks/pytorch-release-notes/index.html

NeMo Framework 24.01

Software Component	Version
Transformer Engine	v1.3
PyTorch	2.2.0
NeMo	1.23.0
NeMo Aligner	0.2.0
NeMo Data Curator	0.1.0
Megatron Core	0.5.0
PyTorch Lightning	2.0.7
Hydra	1.2.0
Kubernetes	1.27.4
Helm	3.12.1
GPU Operator	23.3.2
Network Operator	23.1.0
KubeFlow Operator	1.6.0

NVIDIA NeMo™ Framework Training container is built on top of NVIDIA Optimized Frameworks PyTorch 24.01 container: https://docs.nvidia.com/deeplearning/frameworks/pytorch-release-notes/index.html

NeMo Framework 23.11

Software Component	Version
Transformer Engine	v1.1
PyTorch	2.1.0
NeMo	1.22.0
Megatron Core	0.4.0
PyTorch Lightning	2.0.7
Hydra	1.2.0
Kubernetes	1.27.4
Helm	3.12.1
GPU Operator	23.3.2
Network Operator	23.1.0
KubeFlow Operator	1.6.0

NVIDIA NeMo™ Framework Training container is built on top of NVIDIA Optimized Frameworks PyTorch 23.11 container: https://docs.nvidia.com/deeplearning/frameworks/pytorch-release-notes/index.html