Important
You are viewing the NeMo 2.0 documentation. This release introduces significant changes to the API and a new library, NeMo Run. We are currently porting all features from NeMo 1.0 to 2.0. For documentation on previous versions or features not yet available in 2.0, please refer to the NeMo 24.07 documentation.
Install NeMo Framework#
The NeMo Framework can be installed in the following ways, depending on your needs:
(Recommended) Docker Container
NeMo Framework supports Large Language Models (LLMs), Multimodal Models (MMs), Automatic Speech Recognition (ASR), and Text-to-Speech (TTS) modalities in a single consolidated Docker container.
Recommended for LLM and MM domains.
Conda / Pip
If you are using a Nvidia PyTorch container as the base, this is the recommended method for all domains.
NeMo containers are launched concurrently with NeMo version updates. You can find additional information about released containers on the NeMo releases page.
Using a Pre-built Container (Recommended)
This is suitable for most users who want to use a stable version of NeMo. To get access to the container, follow these steps:
Log in, or create a free account here: NVIDIA GPU Cloud (NGC).
Once logged in, you can view all container releases here: NVIDIA NGC NeMo Framework.
In your terminal, run the following code:
docker pull nvcr.io/nvidia/nemo:24.12
Please use the latest tag in the form “yy.mm.(patch)”.
If you are interested in latest experimental features, get the container with “dev” tag.
(Alternatively) Building a Container from the Latest GitHub Branch
To build a nemo container with Dockerfile from a NeMo Github branch, clone the branch, and run the following code:
DOCKER_BUILDKIT=1 docker build -f Dockerfile -t nemo:latest
If you choose to work with the main branch, we recommend using NVIDIA’s PyTorch container version 23.10-py3 as base and then installing from source on GitHub.
docker run --gpus all -it --rm -v <nemo_github_folder>:/NeMo --shm-size=8g \ -p 8888:8888 -p 6006:6006 --ulimit memlock=-1 --ulimit \ stack=67108864 --device=/dev/snd nvcr.io/nvidia/pytorch:23.10-py3
Important
We strongly recommend that you start with a base NVIDIA PyTorch container:
nvcr.io/nvidia/pytorch:24.07-py3
, and install NeMo and it’s dependencies inside. If doing so, run the container, and go to step 3 directly for pip instructions.Create a fresh Conda environment:
conda create --name nemo python==3.10.12 conda activate nemo
Install PyTorch using their configurator:
conda install pytorch==2.2.0 torchvision torchaudio pytorch-cuda=12.1 -c pytorch -c nvidia
The command to install PyTorch may depend on your system. Use the configurator linked above to find the right command for your system.
Then, install NeMo via Pip or from Source. We do not provide NeMo on the conda-forge or any other Conda channel.
(Option 1) Install NeMo via Pip.
To install the nemo_toolkit, use the following installation method:
Install all Domains
apt-get update && apt-get install -y libsndfile1 ffmpeg pip install Cython packaging pip install nemo_toolkit['all']
Install a Specific Domain
If you are just interested in the Speech AI domain, you can just install asr and/or tts components. You must first install the nemo_toolkit dependencies using.
apt-get update && apt-get install -y libsndfile1 ffmpeg pip install Cython packaging
Then, you can run the following domain-specific commands:
pip install nemo_toolkit['asr'] # For ASR pip install nemo_toolkit['tts'] # For TTS
To install the LLM domain, you should use the command
pip install nemo_toolkit['all']
.(Option 2) Install NeMo and Megatron Core from source.
To work with a specific version of NeMo from a particular GitHub branch (e.g main), use the following installation method:
export BRANCH="main" apt-get update && apt-get install -y libsndfile1 ffmpeg pip install Cython packaging pip install git+https://github.com/NVIDIA/NeMo.git@$BRANCH#egg=nemo_toolkit[all]
Depending on the shell used, you may need to use the “nemo_toolkit[all]” specifier instead in the above command.
Install additional dependencies (Only for LLM, Multimodal domains)
If you work with the LLM and Multimodal domains, there are three additional dependencies: NVIDIA Megatron Core, NVIDIA Apex and NVIDIA Transformer Engine.
Follow the latest instructions at Github: Install LLM and Multimodal dependencies
Note
Apex and Transformer Engine are optional for LLM collections but recommended for optimal performance, while they are required dependencies for Multimodal collections.
Note
RMSNorm is currently not supported when Apex is not installed. Non-Apex support for RMSNorm will be added in a future release.