Getting Started

• Overview
• Quick Start Guide
• NVIDIA AI Enterprise Trial
• Release Notes

Installation

• Best Practices
  • Security
  • Autoscaling Configurations
  • Monitoring GPU and CPU Telemetry
  • Load-Balancing Types
  • Optimizing Resource Usage for Embedded Platforms
• Local (Docker)
  • Configuring
  • Downloading Required Models and Containers from NGC
  • Launching the Servers and Client Container
  • Stopping
  • Clean-up
• Kubernetes
  • Installation with Helm
  • Kubernetes Secrets
  • Riva Settings
  • Deploying Custom RMIR Models with Helm
  • Enabling Model Cache for Scalable Deployment
  • Ingress Controller
• How to Deploy Riva at Scale on AWS with EKS
  • Prerequisites
  • Creating the EKS Cluster
  • Deploying the Riva API
  • Deploying the Traefik edge router
  • Creating the IngressRoute
  • Deploying a Sample Client
  • Scaling the cluster
• NVIDIA Fleet Command
  • Application Setup
  • Deployment Setup

Tutorials

• Speech Recognition
  • How do I use Riva ASR APIs with out-of-the-box models?
    • NVIDIA Riva Overview
    • Transcription with Riva ASR APIs
    • Go deeper into Riva capabilities
  • Creating Grammars for Speech Hints
    • Dependencies
    • Prerequisites
    • Overview
    • Using Existing Speech Hint Grammars in Python
    • Sample Grammar for Handling Alphabet Sequences
  • How to Customize Riva ASR Vocabulary and Pronunciation with Lexicon Mapping
    • Overview
    • What can be customized?
    • Extending the vocabulary
    • Customizing pronunciation with lexicon mapping
    • Go deeper into Riva capabilities
  • How to Deploy a Custom Language Model (n-gram) Trained with NeMo on Riva
    • NVIDIA Riva Overview
    • NeMo (Neural Modules) and nemo2riva
    • Prerequisites
    • Riva ServiceMaker
    • Start the Riva Server
    • Run Inference
  • How to Deploy a Custom Acoustic Model (Citrinet) Trained with NeMo on Riva
    • NVIDIA Riva Overview
    • NeMo (Neural Modules) and nemo2riva
    • Prerequisites
    • Riva ServiceMaker
    • Start the Riva Server
    • Run Inference
  • How to Deploy a Custom Acoustic Model (Conformer-CTC) Trained with NeMo on Riva
    • NVIDIA Riva Overview
    • NeMo (Neural Modules) and nemo2riva
    • Prerequisites
    • Riva ServiceMaker
    • Start the Riva Server
    • Run Inference
  • How to Deploy a Conformer-CTC Acoustic Model with WFST Decoders
    • NVIDIA Riva Overview
    • NeMo (Neural Modules) and nemo2riva
    • Prerequisites
    • Riva ServiceMaker
    • Start the Riva Server
    • Run Inference
  • How to Customize a Riva ASR Acoustic Model (Conformer-CTC) with Adapters
    • NVIDIA Riva Overview
    • Neural Module (NeMo)
  • ASR with Adapters
  • What are Adapters?
  • Advantages and Limitations of Adapter Training
  • Preparing the Acoustic Encoder for Adapter Training
  • Preparing the Model and Dataset for Adaptation
  • Creating and Training an Adapter
  • Evaluating the Model
  • Export the Model to Riva
  • What’s Next?
  • How to Fine-Tune a Riva ASR Acoustic Model with NVIDIA NeMo
    • NVIDIA Riva Overview
    • NeMo (Neural Modules)
    • Fine-Tuning an ASR model with NeMo
    • More Resources
    • What’s Next?
  • How to Improve Recognition of Specific Words
    • Overview of Riva customization techniques
    • 1. Word boosting
    • 2. Custom vocabulary
    • 3. Custom pronunciation (Lexicon mapping)
    • 4. Retrain language model
    • 5. Fine-tune the acoustic model
  • Conclusion
  • How to Synthesize a Noisy Dataset that can be used to Train a Noise Robust ASR Model
  • How to Improve the Accuracy on Noisy Speech by Fine-Tuning the Acoustic Model (Conformer-CTC) in the Riva ASR Pipeline
    • NVIDIA Riva Overview
  • How To Train, Evaluate, and Fine-Tune an n-gram Language Model
    • n-gram Language Model
    • Prerequisites
    • Training and Fine-tuning LM with KenLM and NeMo
    • Download the Evaluation Dataset
    • Generating the Base Language Model
    • Load the ASR Model
    • Fine-Tuning and Interpolation
  • How do I Use Speaker Diarization with Riva ASR?
    • NVIDIA Riva Overview
    • Speaker Diarization with Riva ASR APIs
  • Requirements and Setup
    • Import the Riva Client Libraries
    • Create a Riva Client and Connect to the Riva Speech API Server
    • Go Deeper into Riva Capabilities
  • How do I boost specific words at runtime with word boosting?
    • NVIDIA Riva Overview
    • Word boosting with Riva ASR APIs
    • Go deeper into Riva capabilities
  • Support for Class Based n-gram Language Models in Riva (WFST Decoder)
  • WFST Decoding
    • Supporting Class Language Models in the WFST Framework
    • Process to Create TLG.fst Supporting Word Classes
    • Evaluation
    • Deploying in Riva
• Speech Recognition - New Language Adaptation
• Cloud Deployment
  • How do I Deploy Riva at Scale on Azure Cloud with AKS?
    • Prerequisites
    • Creating the AKS Cluster
    • Deploying the Riva API
    • Deploying the Traefik Edge Router
    • Creating the IngressRoute
    • Deploying a Sample Client
    • Scaling the Cluster
  • How to Deploy Riva at Scale on AWS with EKS
    • Prerequisites
    • Creating the EKS Cluster
    • Deploying the Riva API
    • Deploying the Traefik edge router
    • Creating the IngressRoute
    • Deploying a Sample Client
    • Scaling the cluster
  • How do I Deploy Riva at Scale on Google Cloud with GKE?
    • Prerequisites
    • Creating the GKE Cluster
    • Deploying the Riva API
    • Deploying the Traefik Edge Router
    • Creating the IngressRoute
    • Deploying a Sample Client
    • Scaling the Cluster
  • How to Deploy Riva at Scale on OCI with OKE
    • Prerequisites
    • Creating the OKE Cluster
    • Accessing the OKE Cluster
    • Deploying the Riva API
    • Deploying the Traefik edge router
    • Creating the IngressRoute
    • Deploying a Sample Client
    • Scaling the cluster
• Speech Synthesis
  • How do I use Riva TTS APIs with out-of-the-box models?
    • NVIDIA Riva Overview
    • Basics: Generating Speech with Riva TTS APIs
    • Customizing Riva TTS audio output with SSML
  • TTS Deploy
    • Learning Objectives
    • Prerequisties
    • Riva ServiceMaker
    • Run riva-build
    • Run riva-deploy
  • Run Inference
    • Connect to the Riva server and run inference
  • Evaluate a TTS Pipeline
    • Download data
    • Synthesize text from asr.
    • Calculate character error rate (CER).
    • Calculate WER(Word error rate)
    • Conclusion
  • Text to Speech Finetuning using NeMo
    • Text to Speech
    • Let’s Dig in: TTS using NeMo
  • Calculate and Plot the Distribution of Phonemes in a TTS Dataset
    • Get arpabet file
  • Reference distribution:
    • Load reference distribution
    • Sample corpus Phonemes
    • Plot distribution together
    • Phoneme comparison to a reference distribution
    • Calculate key differences between reference distribution and total phonemes
  • Conclusion
  • Guidelines to Record a TTS Dataset at Home
    • Recommended Data
    • Hardware Requirements
    • Software Requirements
    • Recording Prerequisites
    • Adjusting the Microphone Level and Body Position Before Recording
    • Positioning Yourself Just Right, Too Far or Too Close
    • Recording the TTS Data
• Translation
  • How do I perform Language Translation using Riva NMT APIs with out-of-the-box models?
    • NVIDIA Riva Overview
    • Introduction to Language Translation with Riva NMT
    • Language Translation with Riva NMT APIs
    • Riva S2T and S2S APIs
  • How to deploy a NeMo-finetuned NMT model on Riva Speech Skills server?
    • NVIDIA Riva Overview
    • NVIDIA NeMo Overview
    • Prerequisites
    • Riva ServiceMaker
    • Start the Riva Server
    • Run Inference
  • How to fine-tune a Riva NMT Bilingual model with Nvidia NeMo
    • NVIDIA Riva Overview
    • NVIDIA NeMo Overview
    • Fine-tuning Riva NMT Bilingual model with NVIDIA NeMo
  • How to perform synthetic data generation using Riva NMT Multilingual model with Nvidia NeMo
    • NVIDIA Riva Overview
    • NVIDIA NeMo Overview
    • Generating synthetic data using Riva NMT Multilingual model with NVIDIA NeMo
  • How to fine-tune a Riva NMT Multilingual model with Nvidia NeMo
    • NVIDIA Riva Overview
    • NVIDIA NeMo Overview
    • Fine-tuning Riva NMT Multilingual model with NVIDIA NeMo

Architecture

• Overview
  • Using Your Own Data
  • Model Development with NeMo
  • Riva Build
  • Riva Deploy
  • Deploying Your Custom Model into Riva
  • Build Process
  • Deploy Process
• Clients in a New Programming Language
  • Download the Riva API
  • Generate Language Bindings
  • Write the Client

Speech Recognition

• ASR Overview
  • Try It Out
  • Pretrained ASR Models
  • Language Support
  • Features
  • Checking deployed models
• Basics of Speech Recognition and Customization of Riva ASR
• Basics of Automatic Speech Recognition
  • Speech Recognition by Humans
  • Speech Recognition by Machines
• Evaluation of ASR Accuracy
• Riva ASR
  • The Alignment Problem
  • The Greedy Search Decoder
  • The Beam Search Decoder
• Riva Speech Recognition Pipeline
  • Summary of Riva ASR Customizations
  • Basic Customization of Riva ASR
  • Advanced Customization of Riva ASR
  • WFST Decoding
• Pipeline Configuration
  • Streaming/Offline Recognition
  • Language Models
  • Flashlight Decoder Lexicon
  • Flashlight Decoder Lexicon Free
  • OpenSeq2Seq Decoder
  • Beginning/End of Utterance Detection
  • Neural-Based Voice Activity Detection
  • Generating Multiple Transcript Hypotheses
  • Impact of Chunk Size and Padding Size on Performance and Accuracy (Advanced)
  • Sharing Acoustic and Feature Extractor Models Across Multiple ASR Pipelines (Advanced)
  • Riva-build Optional Parameters
  • Offline Speaker Diarization
• Performance
  • Evaluation Process
  • Results
  • On-Prem Hardware Specifications
• ASR Advanced Details
  • Confidence Estimates

Speech Synthesis

• TTS Overview
  • Try It Out
  • Pretrained TTS Models
  • Language Support
  • Zero Shot TTS (Beta Feature)
  • Emotion mixing (Beta feature)
  • Checking deployed models
  • Output Audio Encoding
• TTS Inference and Customization
  • NVIDIA Riva Overview
  • Basics: Generating Speech with Riva TTS APIs
  • Customizing Riva TTS audio output with SSML
• TTS Zero Shot
  • OOTB Voices
  • Setting up the TTS Python API parameters
  • Understanding TTS Python API parameters
  • Make a gRPC request to the Riva server
  • Binary Client Example
• Speaker Adapter for Custom Voice
  • Download the Models
  • Fine-Tune the Existing Model
  • Generate the Riva Checkpoint
  • Generate RMIR
  • Conclusion
• Custom Models
  • Model Deployment
  • Creating Riva Files
  • Customization
  • Riva-build Pipeline Instructions
• Performance
  • Evaluation Process
  • Results
  • On-Prem Hardware Specifications
  • Performance Considerations
• TTS Deploy
  • Learning Objectives
  • Prerequisties
  • Riva ServiceMaker
  • Run riva-build
  • Run riva-deploy
• Run Inference
  • Connect to the Riva server and run inference
• Phoneme Support
  • English-US
• Data Collection - Script Generation
  • Identifying Data Sources:
  • Phoneme Coverage and Distribution
  • Technical Requirements
  • Heuristics for Manually Normalizing
  • Conclusion

Natural Language Processing

• NLP Overview
  • Punctuation and captilization
  • Checking deployed models
• Custom Models
  • Pretrained Models

Translation

• Translation Overview
  • Language Pairs Supported
  • Translation Features
  • Speech-to-Speech Translation (S2S)
  • Speech-to-Text Translation (S2T)
  • Model Deployment
  • Multiple Deployed Models
  • Punctuation and Inverse Text Normalization (ITN) with S2S and S2T
• Custom Models
  • Example
  • Supported Models
• Performance
  • Evaluation Process
  • Results
  • Hardware Specifications

SDKs and Sample Apps

• Python
  • Installation
  • Python Example
• Command-line Clients
  • Speech Recognition
  • Natural Language Processing
  • Speech Synthesis
  • Machine Translation
  • Speech-to-Speech Translation
  • Speech-to-Text Translation
• Sample Apps
  • Riva Contact
  • Riva Virtual Assistant Example
  • Virtual Assistant (with Rasa)
  • Virtual Assistant (with Google Dialogflow)
  • SpeechSquad
  • AudioCodes VoiceGateway Sample

Reference

• Models
  • Speech Recognition
  • Natural Language Processing
  • Natural Machine Translation(NMT)
  • Speech Synthesis
• gRPC & Protocol Buffers
  • riva/proto/riva_asr.proto
  • riva/proto/riva_nlp.proto
  • riva/proto/riva_nmt.proto
  • riva/proto/riva_tts.proto
  • riva/proto/riva_common.proto
  • riva/proto/riva_audio.proto
  • riva/proto/health.proto
• Troubleshooting
  • NGC
  • Model Export and ServiceMaker
  • Services
  • Client Integration
  • Riva Helm Deployment
  • SpeechSquad
• Support Matrix
  • Riva 2.16.0
  • Riva 2.15.0
  • Riva 2.14.0
  • Riva 2.13.0
  • Riva 2.12.0
  • Riva 2.11.0
  • Riva 2.10.0
  • Riva 2.9.0
  • Riva 2.8.0
  • Riva 2.7.0
  • Riva 2.6.0
  • Riva 2.5.0
  • Riva 2.4.0
  • Riva 2.3.0
  • Riva 2.2.0
  • Riva 2.1.0
  • Riva 2.0.0
  • Riva 1.10.0 Beta
  • Riva 1.9.0 Beta
  • Riva 1.8.0 Beta
  • Riva 1.7.0 Beta
  • Riva 1.6.0 Beta
  • Riva 1.5.0 Beta
  • Riva 1.4.0 Beta
• Upgrading
• Acknowledgements
  • Google APIs
  • GoogleTest
  • gflags
  • Google Logging Library (glog)
  • speexdsp
  • libFLAC
  • gRPC
  • Triton Inference Server
  • NVlabs cub
  • KenLM
  • Kaldi
  • grpc_health_probe
  • OpenFST
  • Yamale
  • PyTorch
  • requests
  • PyCUDA
  • RapidJSON
  • protobuf
  • onnx
  • librosa
  • omegaconf
  • utf8proc
  • re2
  • thrax
  • Sparrowhawk
  • SentencePiece
  • YouTokenToMe
  • MS-SNSD
• End User License Agreement
• Notice
  • Google
  • Trademarks
  • Copyright