NVIDIA NeMo Framework User Guide

NeMo Framework

• Overview
  • Speech AI
  • Other Resources
    • GitHub Repos
    • Getting Help
  • Programming Languages and Frameworks
  • Licenses

Releases

• Software Component Versions
  • NeMo Framework 26.02
  • NeMo Framework 25.11
  • NeMo Framework 25.09
  • NeMo Framework 25.07
  • NeMo Framework 25.04
  • NeMo Framework 25.02
  • NeMo Framework 24.12
  • NeMo Framework 24.09
  • NeMo Framework 24.07
  • NeMo Framework 24.05
  • NeMo Framework 24.01
  • NeMo Framework 23.11
• Changelog
  • 26.02 NeMo Framework Container
  • 25.11.01 NeMo Framework Container
  • 25.11 NeMo Framework Container
    • Evaluator
  • 25.09.02 NeMo Framework Container
    • NeMo 2
    • Megatron Bridge
    • Automodel
    • Export & Deploy
  • 25.09 NeMo Framework Container
    • NeMo 2
    • Megatron Bridge
    • Automodel
    • Evaluator
    • Export & Deploy
    • Speech
  • 25.09 NeMo Curator Container
  • 25.07.02 NeMo Framework Container
  • 25.04.03 NeMo Framework Container
  • 25.07.01 NeMo Framework Container
  • 25.07 NeMo Framework Container
    • Existing Repository
      • Training Performance (Speed)
      • Collections
      • Speech
    • New Repository
      • Export & Deploy
      • Megatron Bridge (Alpha)
      • Automodel (Alpha)
      • Eval (Alpha)
      • nvFSDP (Alpha)
  • 25.07 NeMo Curator Container
  • Updates to Non-Container Repositories
    • RL
  • NeMo Framework 25.04.02
  • NeMo Framework 25.04.01
    • Collections
    • Export & Deploy
    • Automodel
    • Fault Tolerance
  • NeMo Framework 25.04.00
    • Curator
    • Export & Deploy
    • Evaluation
    • Automodel
    • Fault Tolerance
    • Collections
    • Performance
    • Deprecations
  • NeMo Framework 25.02.01
    • Training
  • NeMo Framework 25.02
    • Training
    • Aligner
    • Curator
    • Export & Deploy
    • Evaluations
    • Collections
    • Automodel
    • ASR/TTS
  • NeMo Framework 24.12
    • Training
    • LLM & MM
    • Aligner
    • Export
    • Curator
    • ASR
  • NeMo Framework 24.09
    • LLM & MM
    • Aligner
    • Export
    • Curator
    • ASR
  • NeMo Framework 24.07
    • Training
    • ASR
    • Aligner
    • Curator
    • Export & Deploy
  • NeMo Framework 24.05
    • LLM and MM
      • Models
    • Performance
    • Export
    • Curator
    • Aligner
    • Speech (ASR & TTS)
      • Models
    • Perf Improvements
    • Customization
    • Longform Inference
    • Misc
  • NeMo Framework 24.03.01
  • NeMo Framework 24.03
  • NeMo Framework 24.01.01
  • NeMo Framework 24.01
  • NeMo Framework 23.11
  • NeMo Framework 23.10
  • NeMo Framework 23.08.03
  • NeMo Framework 23.08.02
  • NeMo Framework 23.08.01
  • NeMo Framework 23.08
  • NeMo Framework 23.07
  • NeMo Framework 23.05
  • NeMo Framework 23.04.1
  • NeMo Framework 23.04
  • NeMo Framework 23.03
  • NeMo Framework 23.01
  • NeMo Framework 22.11
  • NeMo Framework 22.09
  • NeMo Framework 22.08.01
  • NeMo Framework 22.08
  • NeMo Framework 22.06-hotfix.01
  • NeMo Framework 22.06
  • NeMo Framework 22.05.01
  • NeMo Framework 22.05
  • NeMo Framework 22.04
  • NeMo Framework 22.03
  • NeMo Framework 22.02
  • NeMo Framework 22.01
• Known Issues

Library Documentation

• Overview
• NeMo
  • Introduction
    • Prerequisites
    • Installation
    • Quick Start Guide
    • Discussion Board
    • Contribute to NeMo
    • License
  • NeMo Fundamentals
    • NeMo Models
    • Configuring and Training NeMo Models
    • Example Training Script
    • Overriding Configs
    • Running NeMo Scripts
      • NeMo Inference Scripts
    • Specifying Training Data
    • Model Checkpoints
    • Fine-Tuning
    • Where To Go Next?
  • Tutorials
    • Running Tutorials on Colab
    • Tutorial Overview
  • Mixed Precision Training
    • Half-Precision Training
      • Implement Half-Precision Training
    • FP8 Training
      • Implement FP8 Training
      • Resources
  • Parallelisms
    • Data Parallelism
      • Distributed Data Parallelism
      • Distributed Optimizer
    • Model Parallelism
      • Tensor Parallelism
      • Pipeline Parallelism
      • Expert Parallelism
    • Activation Partitioning
      • Sequence Parallelism
      • Context Parallelism
      • Parallelism Nomenclature
  • Mixture of Experts
    • Overview
    • Use MoE
    • Configure MoE-specific Loss Functions
  • Optimizations
    • Attention Optimizations
      • Flash Attention
      • Multi-query Attention (MQA) and Grouped-query Attention (GQA)
    • Activation Recomputation
      • Transformer Layer Recomputation
      • Self-attention Recomputation
    • Communication Overlap
      • Data-parallel Communication Overlap
      • Tensor-parallel Communication Overlap
      • Pipeline-parallel Communication Overlap
      • Context-parallel Communication Overlap
    • CPU Offloading
      • Overview
      • Features
      • Usage
  • Checkpoints
    • Checkpoint Formats
    • Sharded Model Weights
    • Quantized Checkpoints
    • Community Checkpoint Converter
      • NeMo Distributed Checkpoint User Guide
      • Converting from Megatron-LM
  • NeMo APIs
    • NeMo Models
      • Basics
      • Pretrained
      • Training
      • Configuration
      • Optimization
      • Save and Restore
      • Register Artifacts
      • Push to Hugging Face Hub
      • Nested NeMo Models
      • Profiling
    • Neural Modules
      • NeuralModule
      • Dynamic Layer Freezing
    • Experiment Manager
      • Resume Training
      • Experiment Loggers
      • Exponential Moving Average
      • Hydra Multi-Run with NeMo
      • Tips and Tricks
      • ExpManagerConfig
    • Neural Types
      • Motivation
      • NeuralType class
      • Type Comparison Results
      • Examples
    • Adapters
      • What are Adapters?
      • torch.nn.Module with Adapters
      • AdapterModuleMixin
      • Using the Adapter Module
      • Adapter Compatible Models
      • References
    • NeMo Core APIs
      • Base class for all NeMo models
      • Base Neural Module class
      • Base Mixin classes
      • Base Connector classes
      • Base Mixin Classes
      • Neural Type checking
      • Neural Type classes
      • Experiment manager
      • Exportable
    • NeMo Common Collection API
      • Callbacks
      • Losses
      • Metrics
      • Tokenizers
      • Data
      • S3 Checkpointing
    • NeMo ASR API
      • Model Classes
      • Modules
      • Parts
      • Mixins
      • Datasets
      • Audio Preprocessors
      • Audio Augmentors
      • Miscellaneous Classes
    • NeMo TTS API
      • Model Classes
      • Base Classes
      • Dataset Processing Classes
  • NeMo Collections
    • Automatic Speech Recognition (ASR)
      • Transcribe speech with 3 lines of code
      • Transcribe speech via command line
      • Improve ASR transcriptions by incorporating a language model (LM)
      • Use real-time transcription
      • Try different ASR models
      • Try out NeMo ASR transcription in your browser
      • ASR tutorial notebooks
      • ASR model configuration
      • Preparing ASR datasets
      • NeMo ASR Documentation
    • Speech Classification
      • Models
      • Datasets
      • Checkpoints
      • NeMo Speech Classification Configuration Files
      • Resource and Documentation Guide
      • Resource and Documentation Guide
    • Speaker Recognition (SR)
      • Models
      • NeMo Speaker Recognition Configuration Files
      • Datasets
      • Checkpoints
      • NeMo Speaker Recognition API
      • Resource and Documentation Guide
      • Resource and Documentation Guide
    • Speaker Diarization
      • Speaker Diarization Overview
      • Types of Speaker Diarization Systems
      • Resource and Documentation Guide
    • Speech Self-Supervised Learning
      • Models
      • Datasets
      • Checkpoints
      • NeMo SSL Configuration Files
      • NeMo SSL collection API
      • Resources and Documentation
      • Resources and Documentation
    • Speech Intent Classification and Slot Filling
      • Resources and Documentation
    • SpeechLM2
      • Using Pretrained Models
      • Inference with Pretrained Models
      • Training a Model
      • Example Using Command-Line Training Script
      • Collection Structure
      • SpeechLM2 Documentation
    • Text-to-Speech (TTS)
      • Models
      • Data Preprocessing
      • Checkpoints
      • NeMo TTS Configuration Files
      • Grapheme-to-Phoneme Models
      • Magpie-TTS
      • Magpie-TTS Preference Optimization
      • Magpie-TTS Longform Inference
    • Resources and Documentation
    • Speech and Audio Processing
      • Models
      • Datasets
      • Checkpoints
      • NeMo Audio Configuration Files
      • NeMo Audio API
    • Resources and Documentation
  • Speech AI Tools
    • NeMo Forced Aligner (NFA)
      • Demos & Tutorials
      • Quickstart
      • How do I use NeMo Forced Aligner?
      • Input manifest file format
      • Output CTM file format
      • Output ASS file format
      • Output JSON manifest file format
      • How do I evaluate the alignment accuracy?
    • Dataset Creation Tool Based on CTC-Segmentation
      • References
    • Speech Data Explorer
      • Getting Started
      • User Interface
      • Analysis of Speech Datasets
    • Comparison tool for ASR Models
      • Getting Started
      • User Interface
      • Analysis of Speech Datasets
    • ASR Evaluator
    • Speech Data Processor
• NeMo AutoModel
• NeMo Curator
• NeMo Evaluator
• NeMo Export and Deploy
• NeMo Megatron Bridge
• NeMo RL
• NeMo Run

Getting Started

• Tutorials
  • Speech AI
    • Running Tutorials on Colab
    • Speech AI Fundamentals
    • Automatic Speech Recognition (ASR) Tutorials
    • Text-to-Speech (TTS) Tutorials
    • Tools and Utilities
    • Text Processing (TN/ITN) Tutorials

Model Optimization

• Quantization
  • Post-Training Quantization
    • Support Matrix
    • Example
      • Use the NeMo CLI
      • Use the PTQ script with torchrun or Slurm
  • References
• Pruning
  • Use with torchrun or Slurm
    • Drop Model Layers (Depth Pruning)
    • Drop Model Width (Width Pruning)
    • Drop Model Depth and Width
  • Use NeMo-Run Recipes
  • Tips for Pruning
  • Troubleshooting
• Distillation
  • Knowledge Distillation Process
  • Limitations
  • Configuration
  • Use NeMo-Run Recipes
  • Use with torchrun or Slurm
  • Perform SFT Knowledge Distillation
• Speculative Decoding
  • Speculative Decoding Process
  • Limitations
  • Converting a Model
  • Exporting

Models

• Speech AI Models
  • Automatic Speech Recognition (ASR)
  • Speaker Diarization
  • Text-to-Speech (TTS)
  • Speech AI Tools