NVIDIA NeMo User Guide#
Getting Started
- Introduction
- Tutorials
- Best Practices
- Why NeMo?
- NeMo, PyTorch Lightning, And Hydra
- Using Optimized Pretrained Models With NeMo
- ASR Guidance
- Data Augmentation
- Speech Data Explorer
- Using Kaldi Formatted Data
- Using Speech Command Recognition Task For ASR Models
- NLP Fine-Tuning BERT
- BioMegatron Medical BERT
- Efficient Training With NeMo
- Recommendations For Optimization And FAQs
- Resources
- Migration guide to use lightning 2.0
NeMo Core
Speech Processing
Natural Language Processing
- NeMo Megatron
- Migrating from Megatron-LM
- GPT model training
- Batching
- Parallelisms
- Prompt Learning
- Terminology
- Prompt Tuning
- P-Tuning
- Using Both Prompt and P-Tuning
- Dataset Preprocessing
- Prompt Formatting
model.task_templatesConfig Parameters- Prompt Learning Specific Config Values
- Setting New Tasks
- Example Multi-Task Prompt Tuning Config and Command
- Example Multi-Task P-Tuning Config and Command After Prompt-Tuning
- Example Multi-Task Inference
- NeMo RETRO Model
- Hiddens Module
- Parameter-Efficient Fine-Tuning (PEFT)
- References
- Machine Translation Models
- Quick Start Guide
- Data Format
- Data Cleaning, Normalization & Tokenization
- Training a BPE Tokenization
- Applying BPE Tokenization, Batching, Bucketing and Padding
- Tarred Datasets for Large Corpora
- Model Configuration and Training
- Multi-Validation
- Bottleneck Models and Latent Variable Models (VAE, MIM)
- Model Inference
- Inference Improvements
- Pretrained Encoders
- References
- (Inverse) Text Normalization
- NeMo Megatron API
- ONNX Export of Megatron Models
- Tasks
- Punctuation And Capitalization Models
- SpellMapper (Spellchecking ASR Customization) Model
- Token Classification (Named Entity Recognition) Model
- Joint Intent and Slot Classification
- Text Classification model
- BERT
- Language Modeling
- Prompt Learning
- Terminology
- Prompt Tuning
- P-Tuning
- Using Both Prompt and P-Tuning
- Dataset Preprocessing
- Prompt Formatting
model.task_templatesConfig Parameters- Prompt Learning Specific Config Values
- Setting New Tasks
- Example Multi-Task Prompt Tuning Config and Command
- Example Multi-Task P-Tuning Config and Command After Prompt-Tuning
- Example Multi-Task Inference
- Question Answering
- Dialogue tasks
- GLUE Benchmark
- Information Retrieval
- Entity Linking
- Model NLP
- Machine Translation Models
- Quick Start Guide
- Data Format
- Data Cleaning, Normalization & Tokenization
- Training a BPE Tokenization
- Applying BPE Tokenization, Batching, Bucketing and Padding
- Tarred Datasets for Large Corpora
- Model Configuration and Training
- Multi-Validation
- Bottleneck Models and Latent Variable Models (VAE, MIM)
- Model Inference
- Inference Improvements
- Pretrained Encoders
- References
Text To Speech (TTS)
Text Processing