Automatic Speech Recognition (ASR)#
Automatic Speech Recognition (ASR), also known as Speech To Text (STT), refers to the problem of automatically transcribing spoken language. NeMo provides open-sourced pretrained models in 25+ languages. Browse the full list in ASR Model Checkpoints.
Quick Start#
After installing NeMo, transcribe an audio file in 3 lines:
import nemo.collections.asr as nemo_asr
asr_model = nemo_asr.models.ASRModel.from_pretrained("nvidia/parakeet-tdt-0.6b-v2")
transcript = asr_model.transcribe(["path/to/audio_file.wav"])[0].text
Timestamps#
Obtain word, segment, or character timestamps with any Parakeet model (CTC/RNNT/TDT):
hypotheses = asr_model.transcribe(["path/to/audio_file.wav"], timestamps=True)
for stamp in hypotheses[0].timestamp['word']:
print(f"{stamp['start']}s - {stamp['end']}s : {stamp['word']}")
See Inference for full details on timestamps, long audio, streaming, and multi-task inference.
Key Features#
50+ Pretrained Models — NeMo offers open-source checkpoints across 14+ languages, available on HuggingFace and NGC. Browse the full list in All Checkpoints.
Timestamps — Character, word, and segment-level timestamps are supported for all Parakeet models with CTC, RNNT, and TDT decoders.
Streaming — Real-time transcription with cache-aware streaming Conformer models, supporting configurable latency-accuracy tradeoffs. See Cache-aware Streaming Conformer.
Multi-task (Canary) — The Canary model family supports ASR and speech translation (AST) across 25 European languages, with built-in punctuation and capitalization. See Featured Models.
Language Modeling — GPU-accelerated n-gram LM fusion (NGPU-LM) for CTC, RNN-T, TDT, and AED models improves transcription accuracy without retraining. See ASR Language Modeling and Customization.
Word Boosting — Bias decoding toward specific words or phrases without retraining. Supports global and per-stream (per-utterance) boosting. See Word Boosting.
Multitalker — Streaming multi-speaker ASR with speaker kernel injection handles overlapping speech in real time. See Multitalker Parakeet.
Long Audio — Inference on audio over 1 hour via local attention or buffered chunked processing.
Decoder Types — NeMo supports CTC, RNN-T, TDT, AED, and Hybrid decoders. For a comparison of decoder types, see ASR Language Modeling and Customization.
ASR Customization#
NeMo supports decoding-time customization techniques to improve accuracy without retraining, including GPU-accelerated language model fusion (NGPU-LM), neural rescoring, and word boosting (GPU-PB, per-stream, Flashlight, CTC-WS). See ASR Language Modeling and Customization for full documentation.