NVIDIA NeMo Framework User Guide

NeMo Framework

• Overview
  • Large Language Models and Multimodal Models
    • Data Curation
    • Training and Customization
    • RL
    • Multimodal Models
    • Deployment and Inference
      • Deploy with NVIDIA NIM
      • Deploy with TensorRT-LLM or vLLM
    • Supported Models
      • Large Language Models
      • Vision Language Models
      • Embedding Models
      • World Foundation Models
  • Speech AI
  • Other Resources
    • GitHub Repos
    • Getting Help
  • Programming Languages and Frameworks
  • Licenses
• Install NeMo Framework
• Performance
  • Performance Summary for Large Language Models
    • Pre-training
    • Fine-Tuning
      • Performance Numbers for previous NeMo Container Releases
• Why NeMo Framework?
  • Large Language Models & Multimodal (LLM & MM)
  • Speech AI
    • Data Augmentation
    • Speech Data Explorer
    • Using Kaldi Formatted Data
    • Speech Command Recognition
  • General Optimizations
    • Mixed Precision Training
    • Multi-GPU Training
    • NeMo, PyTorch Lightning, and Hydra
    • Optimized Pretrained Models
  • Resources

Getting Started

• Quickstart with NeMo-Run
  • Minimum Requirements
  • Pretraining
    • Set Up the Prerequisites
    • Configure the Recipe
      • Override the Attributes
      • Swap Recipes
    • Execute Locally
      • Change the Number of GPUs
    • Execute on a Slurm Cluster
    • Continue Pretraining
  • Fine-Tuning
    • Set Up the Prerequisites
    • Configure the Recipe
    • Execute Locally
      • Switch from PEFT to Full Fine-Tuning
      • Use a NeMo 2.0 Pretraining Checkpoint as the Base
      • Execute on a Slurm Cluster with More Nodes
• Quickstart with NeMo 2.0 API
  • Install NeMo 2.0
  • Quickstart
  • Extend Quickstart with NeMo-Run
    • Prerequisites
    • Launch the Experiment Locally
    • Launch the Experiment on Slurm
  • Where to Find NeMo 2.0
  • Additional Resources
• Tutorials
  • Large Language Models
    • Data Curation
    • Training and Customization
  • World Foundation Models
    • Post Training
  • Speech AI
    • Running Tutorials on Colab
    • Speech AI Fundamentals
    • Automatic Speech Recognition (ASR) Tutorials
    • Text-to-Speech (TTS) Tutorials
    • Tools and Utilities
    • Text Processing (TN/ITN) Tutorials

Developer Guides

• Migration Guide
  • Pre-Training
    • NeMo 1.0 (Previous Release)
    • NeMo 2.0 (New Release)
      • Command Line Interface
    • Continue Training
      • Command Line Interface
    • Migration Steps
  • SFT Training and Inference
    • NeMo 1.0 (Previous Release)
    • NeMo 2.0 (New Release)
    • Migration Steps
  • PEFT Training and Inference
    • NeMo 1.0 (Previous Release)
    • NeMo 2.0 (New Release)
    • Migration Steps
    • Some Notes on Migration
  • Trainer Configuration
    • NeMo 1.0 (Previous Release)
    • NeMo 2.0 (New Release)
    • Migration Steps
  • Precision Configuration
    • NeMo 1.0 (Previous Release)
    • NeMo 2.0 (New Release)
    • Migrate Precision Configurations
  • Parallelisms
    • NeMo 1.0 (Previous Release)
    • NeMo 2.0 (New Release)
    • Migrate Parallelism Configurations
  • Experiment Manager
    • NeMo 1.0 (Previous Release)
    • NeMo 2.0 (New Release)
    • Migration Steps
  • Checkpointing Configurations
    • NeMo 1.0 (Previous Release)
    • NeMo 2.0 (New Release)
    • Migrate Distributed Checkpoint Setup Settings
    • Convert NeMo 1.0 Checkpoint to NeMo 2.0 Checkpoint
  • Optimizer Configuration
    • NeMo 1.0 (Previous Release)
    • NeMo 2.0 (New Release)
    • Migration Steps
  • Data Configuration
    • NeMo 1.0 (Previous Release)
    • NeMo 2.0 (New Release)
    • Migration Steps
    • Some Notes on Migration
  • Nsys Profiling
    • NeMo 1.0 (Previous Release)
    • NeMo 2.0 (New Release)
    • Migration Steps
  • Tokenizers
    • NeMo 1.0 (Previous Release)
    • NeMo 2.0 (New Release)
• Feature Guide
  • The Bridge Between Lightning and Megatron Core
    • MegatronStrategy
    • MegatronParallel
    • MegatronMixedPrecision
  • Logging and Checkpointing
    • ModelCheckpoint
    • NeMoLogger
    • Experiment Logging
      • TensorBoard Logging
      • Weights & Biases (wandb) Logging
    • AutoResume
    • Putting it All Together
  • Serialization
    • IOMixin
  • Parameter Efficient Fine-Tuning (PEFT)
    • ModelTransform Class
    • PEFT Class
    • LoRA Class (or Any Other PEFT Method)
    • Set Up a LoRA Training Run
  • Hugging Face Integration
    • Fine-Tune a Model using a Hugging Face Checkpoint
    • Create a Model Importer
  • Profiling
    • Nsys Profiling
      • Use Nsys Profiling in NeMo 2.0
      • Analyze Nsys Profiling Results
    • CUDA Memory Profiling
      • Use CUDA Memory Profiling in NeMo 2.0
      • Analyze CUDA Memory Profiling Results
• Best Practices
  • Import Guarding
  • Working with Hugging Face Models
  • Working with scripts in NeMo 2.0

Training and Customization

• Long Context Training
  • Context Parallelism
    • Enable Context Parallelism
    • Implement Context Parallelism
  • Provided Long Context Recipes in NeMo
    • Llama 3
    • Mixtral
    • Nemotron 4
  • Use Long Context Recipes
    • Use with the NeMo-Run API
      • Customize Parameters
    • Use with the NeMo-Run CLI
      • Customize Parameters
    • Add a New Recipe
  • Train a Long Context Model Pipeline
    • Use with the NeMo-Run API
      • Pretrain the Llama 3 Model from Scratch
      • Extend the Context Length of a Pretrained Checkpoint
    • Use with the NeMo-Run CLI
      • Pretrain the Llama 3 Model from Scratch
      • Extend the Context Length of a Pretrained Checkpoint
• Optimal Configuration with Auto Configurator
  • Auto Configurator Capabilities
    • Model Size Recommendation
    • Training Time Estimation
    • Hyperparameters Recommendation
    • Optimal Configuration Recommendation
    • Configurations Generation Example
    • Calculate Performance
    • End-To-End Example
• Parameter-Efficient Fine-tuning (PEFT)
  • Model Customization
    • Supported PEFT Methods
      • NeMo 2.0
      • NeMo 1.0 (Legacy)
  • Run PEFT Training in NeMo 2.0
  • Train with a Custom Dataset
  • Run PEFT Inference in NeMo 2.0
  • Load PEFT Checkpoint for Evaluation
  • Merge LoRA Weights with Base Model
  • Export LoRA Weights to Hugging Face
    • A Comparison of Performant and Canonical LoRA Variants
  • Explore PEFT Design in NeMo 2.0
• Sequence Packing
  • Sequence Packing for SFT/PEFT
    • Overview
    • Run SFT/PEFT with Packed Sequences in LLM
      • Prepare the Dataset
      • Train with Predefined Fine-Tune Recipes
      • Migrate Your Own Recipe
    • Run SFT/PEFT with Packed Sequences in NeVA
• Resiliency
  • Fault Tolerance
    • Use Fault Tolerance Features
    • Default Settings
  • Straggler Detection
    • Use Straggler Detection Features
    • Default Settings
  • Local Checkpointing
    • Use Local Checkpointing Features
    • Configuration
  • Preemption
    • Use Preemption Features
    • Default Settings
• Continual Training
  • Acquire Pretrained Checkpoints
  • Configure Continual Learning
  • Adjust Training Configurations
  • Execute Continual Learning
• Custom Datasets
  • PreTrainingDataModule
  • FineTuningDataModule
    • Pre-Training Data Module
      • Overview
      • Data Preprocessing
      • Preprocess for GPT Model
      • Preprocess for BERT Model
      • Use PreTrainingDataModule with Preprocessed Data
    • Fine-Tuning Data Module
      • Overview
      • Option 1: Create a Custom DataModule
      • Option 2: Use FineTuningDataModule with Preprocessed Data
      • Advanced Features

NeMo AutoModel

• Overview
  • Quickstart
  • Known Issues
• Parameter-Efficient Fine-tuning (PEFT)
  • Introduction
  • Run PEFT with NeMo AutoModel
  • Run PEFT Inference with NeMo AutoModel-trained Adapters
  • Publish PEFT Adapters to Hugging Face Hub
  • Evaluate with the LM Evaluation Harness
  • Export to vLLM
• Supervised Fine-tuning (SFT)
  • Introduction
  • Run SFT with NeMo AutoModel
  • Run Inference with the NeMo AutoModel Fine-Tuned Checkpoint
  • Publish the SFT Checkpoint to the Hugging Face Hub
  • Evaluate with the LM Evaluation Harness
  • Export to vLLM
• Large Language Models with NeMo AutoModel
  • Introduction
  • Run LLMs with NeMo Automodel
  • Supported Models
• Vision Language Models with NeMo AutoModel
  • Introduction
  • Installation Notes
  • Models
  • rdr items dataset
  • cord-v2 dataset
  • Train the Model
• Add a New AutoModel
  • Where To Put the Class
  • Create Your AutoModel Class
  • Configure the Underlying HF Model
  • Training and Validation Loops
  • Checkpointing Utilities
  • Checkpoint I/O Wrapper
  • Create a LightningDataModule
  • Quick Checklist

Model Optimization

• Quantization
  • Post-Training Quantization
    • Support Matrix
    • Example
      • Use the NeMo CLI
      • Use the PTQ script with torchrun or Slurm
  • References
• Pruning
  • Use with torchrun or Slurm
    • Drop Model Layers (Depth Pruning)
    • Drop Model Width (Width Pruning)
    • Drop Model Depth and Width
  • Use NeMo-Run Recipes
  • Tips for Pruning
  • Troubleshooting
• Distillation
  • Logits-Distillation Process
  • Limitations
  • Use NeMo-Run Recipes
  • Use with torchrun or Slurm

Models

• Large Language Models
  • Baichuan 2
    • NeMo 2.0 Finetuning Recipes
  • ChatGLM 3
    • NeMo 2.0 Finetuning Recipes
  • DeepSeek V2
    • NeMo 2.0 Pretraining Recipes
    • NeMo 2.0 Finetuning Recipes
  • DeepSeek V3
    • NeMo 2.0 Pretraining Recipes
    • NeMo 2.0 Finetuning Recipes
  • Gemma
    • NeMo 2.0 Finetuning Recipes
  • Gemma 2
    • NeMo 2.0 Finetuning Recipes
  • Hyena
    • Introduction to Hyena and Evo 2
      • Introduction
    • Hyena-Based Models
      • Available Models
      • Training Recipes
    • BioNeMo Integration with Evo 2
  • Llama 3
    • NeMo 2.0 Pretraining Recipes
    • NeMo 2.0 Finetuning Recipes
  • Llama Nemotron
    • NeMo 2.0 Pretraining Recipes
    • NeMo 2.0 Finetuning Recipes
  • Mamba 2
    • New in Mamba: Nemotron-H Models
      • Introduction
      • Nemotron-H Training Recipes
      • Nemotron-H Inference
    • Mamba2 and Mamba2-Hybrid Models
      • Introduction
      • Available Models
      • Mamba2 Training Recipes
  • Mixtral
    • NeMo 2.0 Pretraining Recipes
    • NeMo 2.0 Finetuning Recipes
  • Nemotron
    • NeMo 2.0 Pretraining Recipes
  • Phi 3
    • NeMo 2.0 Pretraining Recipes
    • NeMo 2.0 Finetuning Recipes
  • Qwen2/2.5
    • NeMo 2.0 Pretraining Recipes
    • NeMo 2.0 Fine-tuning Recipes
  • Starcoder
    • NeMo 2.0 Pretraining Recipes
    • NeMo 2.0 Finetuning Recipes
  • Starcoder 2
    • NeMo 2.0 Pretraining Recipes
    • NeMo 2.0 Finetuning Recipes
  • T5
    • NeMo 2.0 Pretraining Recipes
    • NeMo 2.0 Finetuning Recipes
  • BERT
    • NeMo 2.0 Pretraining Recipes
• Vision Language Models
  • NeVA (LLaVA)
    • Import from Hugging Face to NeMo 2.0
    • NeMo 2.0 Fine-Tuning Recipes
      • Bring Your Own Data
    • Use the Energon Dataloader
      • Example Code
      • Explanation of Parameters
      • Key Components
    • Supported Features
  • LLaVA-Next
    • Import from Hugging Face to NeMo 2.0
    • NeMo 2.0 Pretraining Recipes
    • NeMo 2.0 Fine-Tuning Recipes
    • Use the Energon Dataloader
  • Llama 3.2 Vision Models
    • Import from Hugging Face to NeMo 2.0
    • NeMo 2.0 Fine-Tuning Recipes
      • Bring Your Own Data
  • Llama 4 Models
    • Import from Hugging Face to NeMo 2.0
    • NeMo 2.0 Llama 4 Scripts
    • NeMo 2.0 Fine-Tuning Recipes
      • Bring Your Own Data
  • Qwen2-VL
    • Import from Hugging Face to NeMo 2.0
    • NeMo 2.0 Qwen2-VL Scripts
    • NeMo 2.0 Fine-Tuning Recipes
      • Bring Your Own Data
    • Use the Energon Dataloader
      • Data Preparation
      • Example Code
  • Data Preparation to Use Megatron-Energon Dataloader
    • Step 1: Download the Dataset
    • Step 2: Convert JSON Data to WebDataset Format
    • Step 3: Generate Metadata for Megatron-Energon
      • .nv-meta Directory Contents
      • Example Content of dataset.yaml
  • CLIP
    • Import from Hugging Face to NeMo 2.0
    • NeMo 2.0 Pretraining Recipes
    • Use the Energon Dataloader
• Speech AI Models
  • Automatic Speech Recognition (ASR)
  • Speaker Diarization
  • Text-To-Speech (TTS)
  • Speech AI Tools
• Diffusion Models
  • Flux
    • Import from Hugging Face to NeMo 2.0
      • Training
      • Inference
    • Training Recipes
    • Flux ControlNet
      • Training
      • Inference
    • Energon-Compatible Dataset
  • Diffusion Training Framework
    • Overview
    • Performance
      • Features Status
      • Training Stages
    • Energon Dataloader for Webscale Dataloading
      • Webscale Dataloading
      • Dataloader Checkpointing
    • Mixed Image-Video Training
    • Model and Data Parallelism
    • Model Architectures
      • DiT
    • Model Training
    • Citations
• Embedding Models
  • SBERT
    • NeMo 2.0 Finetuning Recipes
  • Llama Embedding
    • NeMo 2.0 Fine-Tuning Recipes
  • Exporting Llama Embedding To ONNX and TensorRT
    • ONNX Export
    • TensorRT Export

Deploy Models

• Overview
  • Deploy NeMo Models by Exporting Inference Optimized libraries
  • NVIDIA NIM for LLMs
  • Deploy PyTorch-level Models (In-Framework)
• Large Language Models
  • Deploy NeMo Models
    • Deploy NeMo Models Using NIM LLM Containers
      • Exporting NeMo 2.0 Models to Hugging Face
      • Exporting NeMo 2.0 Models to TensorRT-LLM in NIM LLM Container
      • Exporting NeMo 2.0 Models to TensorRT-LLM in NeMo FW Container
    • Deploy NeMo Models by Exporting to Inference Optimized Libraries
      • Supported LLMs
    • Deploy NeMo Models in the Framework
      • Quick Example
      • Use a Script to Deploy NeMo LLMs on a Triton Server
    • Send Queries to the NVIDIA Triton Server for NeMo LLMs
      • Send a Query using the Script
      • Send a Query using the NeMo APIs
  • Deploy Hugging Face Models
  • Supported NeMo Checkpoint Formats
  • Supported GPUs
  • Generate a NeMo 2.0 Checkpoint
• Multimodal Models
  • Optimized Inference for Multimodal Models using TensorRT
  • Supported GPUs
  • Supported Models
    • Access the Models with a Hugging Face Token
    • Export and Deploy a NeMo Multimodal Checkpoint to TensorRT
    • Use a Script to Run Inference on a Triton Server
      • Export and Deploy a Multimodal Model to TensorRT
      • Send a Query
    • Use NeMo Export and Deploy Module APIs to Run Inference
      • Export a Multimodal Model to TensorRT
      • Deploy a Multimodal Model to TensorRT
      • Send a Query
      • Other Examples

Library Documentation

• Overview
• NeMo
  • Introduction
    • Prerequisites
    • Installation
    • Quick Start Guide
    • Discussion Board
    • Contribute to NeMo
    • License
  • NeMo Fundamentals
    • NeMo Models
    • Configuring and Training NeMo Models
    • Example Training Script
    • Overriding Configs
    • Running NeMo Scripts
      • NeMo Inference Scripts
    • Specifying Training Data
    • Model Checkpoints
    • Fine-Tuning
    • Where To Go Next?
  • Tutorials
    • Running Tutorials on Colab
    • Tutorial Overview
  • Mixed Precision Training
    • Half-Precision Training
      • Implement Half-Precision Training
    • FP8 Training
      • Implement FP8 Training
      • Resources
  • Parallelisms
    • Data Parallelism
      • Distributed Data Parallelism
      • Distributed Optimizer
    • Model Parallelism
      • Tensor Parallelism
      • Pipeline Parallelism
      • Expert Parallelism
    • Activation Partitioning
      • Sequence Parallelism
      • Context Parallelism
      • Parallelism Nomenclature
  • Mixture of Experts
    • Overview
    • Use MoE
    • Configure MoE-specific Loss Functions
  • Optimizations
    • Attention Optimizations
      • Flash Attention
      • Multi-query Attention (MQA) and Grouped-query Attention (GQA)
    • Activation Recomputation
      • Transformer Layer Recomputation
      • Self-attention Recomputation
    • Communication Overlap
      • Data-parallel Communication Overlap
      • Tensor-parallel Communication Overlap
      • Pipeline-parallel Communication Overlap
      • Context-parallel Communication Overlap
    • CPU Offloading
      • Overview
      • Features
      • Usage
  • Checkpoints
    • Checkpoint Formats
    • Sharded Model Weights
    • Quantized Checkpoints
    • Community Checkpoint Converter
      • NeMo Distributed Checkpoint User Guide
      • Converting from Megatron-LM
  • Evaluate NeMo 2.0 Checkpoints
    • Introduction
    • Run Evaluations without NeMo-Run
    • Run Evaluations with NeMo-Run
      • Launch Evaluations with NeMo-Run
      • Run Locally with NeMo-Run
      • Run on Slurm-based Clusters
    • Run Legacy Evaluations with lm-evaluation-harness
  • NeMo APIs
    • NeMo Models
      • Basics
      • Pretrained
      • Training
      • Configuration
      • Optimization
      • Save and Restore
      • Register Artifacts
      • Push to Hugging Face Hub
      • Nested NeMo Models
      • Profiling
    • Neural Modules
      • NeuralModule
      • Dynamic Layer Freezing
    • Experiment Manager
      • Resume Training
      • Experiment Loggers
      • Exponential Moving Average
      • Hydra Multi-Run with NeMo
      • Tips and Tricks
      • ExpManagerConfig
    • Neural Types
      • Motivation
      • NeuralType class
      • Type Comparison Results
      • Examples
    • Exporting NeMo Models
      • Exporting Models
      • How to Use Model Export
      • How to Make Model Exportable
    • Adapters
      • What are Adapters?
      • torch.nn.Module with Adapters
      • AdapterModuleMixin
      • Using the Adapter Module
      • Adapter Compatible Models
      • References
    • NeMo Core APIs
      • Base class for all NeMo models
      • Base Neural Module class
      • Base Mixin classes
      • Base Connector classes
      • Base Mixin Classes
      • Neural Type checking
      • Neural Type classes
      • Experiment manager
      • Exportable
    • NeMo Common Collection API
      • Callbacks
      • Losses
      • Metrics
      • Tokenizers
      • Data
      • S3 Checkpointing
    • NeMo ASR API
      • Model Classes
      • Modules
      • Parts
      • Mixins
      • Datasets
      • Audio Preprocessors
      • Audio Augmentors
      • Miscellaneous Classes
    • NeMo TTS API
      • Model Classes
      • Base Classes
      • Dataset Processing Classes
  • NeMo Collections
    • Large Language Models
      • GPT Model Training
      • Batching
      • Positional embeddings
      • Positional interpolation
      • References
      • Megatron Core Customization
      • Reset Learning Rate
      • Parameters
      • Use Cases
      • Ramp Up Batch Size
      • Usage
      • Ramp Up Stages and Training Interruption
      • Automatic Node Scheduling
      • Example
      • References
    • Machine Translation Models
      • Quick Start Guide
      • Data Format
      • Data Cleaning, Normalization & Tokenization
      • Training a BPE Tokenization
      • Applying BPE Tokenization, Batching, Bucketing and Padding
      • Tarred Datasets for Large Corpora
      • Model Configuration and Training
      • Multi-Validation
      • Bottleneck Models and Latent Variable Models (VAE, MIM)
      • Model Inference
      • Inference Improvements
      • Pretrained Encoders
      • References
    • Automatic Speech Recognition (ASR)
      • Transcribe speech with 3 lines of code
      • Transcribe speech via command line
      • Incorporate a language model (LM) to improve ASR transcriptions
      • Use real-time transcription
      • Try different ASR models
      • Try out NeMo ASR transcription in your browser
      • ASR tutorial notebooks
      • ASR model configuration
      • Preparing ASR datasets
      • NeMo ASR Documentation
    • Speech Classification
      • Models
      • Datasets
      • Checkpoints
      • NeMo Speech Classification Configuration Files
      • Resource and Documentation Guide
      • Resource and Documentation Guide
    • Speaker Recognition (SR)
      • Models
      • NeMo Speaker Recognition Configuration Files
      • Datasets
      • Checkpoints
      • NeMo Speaker Recognition API
      • Resource and Documentation Guide
      • Resource and Documentation Guide
    • Speaker Diarization
      • Speaker Diarization Overview
      • Types of Speaker Diarization Systems
      • Resource and Documentation Guide
    • Speech Self-Supervised Learning
      • Models
      • Datasets
      • Checkpoints
      • NeMo SSL Configuration Files
      • NeMo SSL collection API
      • Resources and Documentation
      • Resources and Documentation
    • Speech Intent Classification and Slot Filling
      • Resources and Documentation
    • Text-to-Speech (TTS)
      • Models
      • Data Preprocessing
      • Checkpoints
      • NeMo TTS Configuration Files
      • Grapheme-to-Phoneme Models
    • Resources and Documentation
    • Speech and Audio Processing
      • Models
      • Datasets
      • Checkpoints
      • NeMo Audio Configuration Files
      • NeMo Audio API
    • Resources and Documentation
  • Speech AI Tools
    • NeMo Forced Aligner (NFA)
      • Demos & Tutorials
      • Quickstart
      • How do I use NeMo Forced Aligner?
      • Input manifest file format
      • Output CTM file format
      • Output ASS file format
      • Output JSON manifest file format
      • How do I evaluate the alignment accuracy?
    • Dataset Creation Tool Based on CTC-Segmentation
      • References
    • Speech Data Explorer
      • Getting Started
      • User Interface
      • Analysis of Speech Datasets
    • Comparison tool for ASR Models
      • Getting Started
      • User Interface
      • Analysis of Speech Datasets
    • ASR Evaluator
    • Speech Data Processor
    • (Inverse) Text Normalization
      • WFST-based (Inverse) Text Normalization
      • Neural Models for (Inverse) Text Normalization
• NeMo RL
• NeMo Curator
  • Text Curation
  • Image Curation
  • Reference
    • Text Curation
      • Download and Extract Text
      • Working with DocumentDataset
      • CPU and GPU Modules with Dask
      • Classifier and Heuristic Quality Filtering
      • Language Identification and Unicode Fixing
      • Stop Words in Text Processing
      • GPU Accelerated Exact and Fuzzy Deduplication
      • Semantic Deduplication
      • Synthetic Data Generation
      • Downstream Task Decontamination/Deduplication
      • PII Identification and Removal
      • Distributed Data Classification
    • Image Curation
      • Get Started
      • Image-Text Pair Datasets
      • Aesthetic Classifier
      • NSFW Classifier
      • Semantic Deduplication
    • Reference
      • Running NeMo Curator on Kubernetes
      • Reading and writing datasets with NeMo Curator and Apache Spark
      • Best Practices
      • Next Steps
      • API Reference
• NeMo Run
  • Guides
    • Configure NeMo-Run
      • Python Meets YAML
      • Raw Scripts
    • Execute NeMo Run
      • Executors
    • Management
      • Creating an Experiment
      • Adding Tasks
      • Launching an Experiment
      • Experiment Status
      • Canceling a Task
      • Viewing Logs
      • Experiment output
    • Why should I use NemoRun?
      • Configuration
      • Execution
      • Management
      • Conclusion
  • Frequently Asked Questions
    • Configuration
      • Q: UnserializableValueError when using run.Partial or run.Config
      • Q: Deserialization error when using run.Partial or run.Config
      • Q: How to use control flow in autoconvert?
      • Q: I made a change locally in my git repo and tested it using the local executor. However, the change is not reflected in the remote job.
      • Q: I made a change locally outside my git repo and tested it using the local executor. However, the change is not reflected in the remote job.
    • Execution
      • Q: For SlurmExecutor, how can I execute directly from the login node of the cluster.
    • Management
      • Q: I can’t retrieve logs for an experiment.
  • Installation
  • Tutorials

Releases

• Software Component Versions
  • NeMo Framework 25.04
  • NeMo Framework 25.02
  • NeMo Framework 24.12
  • NeMo Framework 24.09
  • NeMo Framework 24.07
  • NeMo Framework 24.05
  • NeMo Framework 24.01
  • NeMo Framework 23.11
• Changelog
  • NeMo Framework 25.04.01
    • Collections
    • Export & Deploy
    • AutoModel
    • Fault Tolerance
  • NeMo Framework 25.04.00
    • Curator
    • Export & Deploy
    • Evaluation
    • AutoModel
    • Fault Tolerance
    • Collections
    • Performance
    • Deprecations
    • Long-term support (LTS)
  • NeMo Framework 25.02.01
    • Training
  • NeMo Framework 25.02
    • Training
    • Aligner
    • Curator
    • Export & Deploy
    • Evaluations
    • Collections
    • Automodel
    • ASR/TTS
  • NeMo Framework 24.12
    • Training
    • LLM & MM
    • Aligner
    • Export
    • Curator
    • ASR
  • NeMo Framework 24.09
    • LLM & MM
    • Aligner
    • Export
    • Curator
    • ASR
  • NeMo Framework 24.07
    • Training
    • ASR
    • Aligner
    • Curator
    • Export & Deploy
  • NeMo Framework 24.05
    • LLM and MM
      • Models
    • Performance
    • Export
    • Curator
    • Aligner
    • Speech (ASR & TTS)
      • Models
    • Perf Improvements
    • Customization
    • Longform Inference
    • Misc
  • NeMo Framework 24.03.01
  • NeMo Framework 24.03
  • NeMo Framework 24.01.01
  • NeMo Framework 24.01
  • NeMo Framework 23.11
  • NeMo Framework 23.10
  • NeMo Framework 23.08.03
  • NeMo Framework 23.08.02
  • NeMo Framework 23.08.01
  • NeMo Framework 23.08
  • NeMo Framework 23.07
  • NeMo Framework 23.05
  • NeMo Framework 23.04
  • NeMo Framework 23.03
  • NeMo Framework 23.01
  • NeMo Framework 22.11
  • NeMo Framework 22.09
  • NeMo Framework 22.08.01
  • NeMo Framework 22.08
  • NeMo Framework 22.06
  • NeMo Framework 22.05.01
  • NeMo Framework 22.05
  • NeMo Framework 22.04
  • NeMo Framework 22.03
  • NeMo Framework 22.02
  • NeMo Framework 22.01
• Known Issues