Local (Docker)

Configuring

A general configuration for the services deployed using the Quick Start scripts is done by editing the config.sh file. By default, the configuration file is set to launch all available services on the supported GPU.

Attention

By default, the Riva Speech AI Skills API server listens on port 50051.

All of the configuration options are documented within the configuration file itself. Follow the instructions in the config.sh file to change the default deployment behavior of the script. Advanced users can select which specific models to deploy for each service by commenting out lines corresponding to the prebuilt model configuration files.

Downloading Required Models and Containers from NGC

The riva_init.sh script downloads all required models and containers from NGC and generates the model repository. The download can take some time depending on your internet bandwidth. You need to provide an NGC API key for it to work. The key can be provided either through the environment variable NGC_API_KEY or as a configuration file (which is automatically generated by running ngc config set).

If the NGC key cannot automatically be discovered from your environment, the init script prompts you to enter it.

Run the script with the command bash riva_init.sh. Upon successful completion of this command, you should see the following output:

Logging into NGC Docker registry if necessary...
Pulling required Docker images if necessary...
 > Pulling Riva Speech Server images.
 > Pulling nvcr.io/nvidia/riva/riva-speech:2.17.0. This may take some time...
Riva initialization complete. Run bash riva_start.sh to launch services.

Launching the Servers and Client Container

After downloading the required models and containers, the Riva Speech AI servers can be started by running bash riva_start.sh. This launches the Riva Speech AI API server.

Example output:

Starting Riva Speech Services
 > Waiting for Triton server to load all models...retrying in 10 seconds
 > Waiting for Triton server to load all models...retrying in 10 seconds
 > Waiting for Triton server to load all models...retrying in 10 seconds
 > Triton server is ready…

To verify that the servers have started correctly, check that the output of docker logs riva-speech shows:

I0428 03:14:50.440955 1 riva_server.cc:71] Riva Conversational AI Server listening on 0.0.0.0:50051

For data center, to start a container with sample clients for each service, run bash riva_start_client.sh. For embedded, you can try the sample clients for each service from the server container that has already been launched by running bash riva_start.sh.

Try walking through the different tutorials on GitHub at nvidia-riva/tutorials.

Stopping

To shut down the Riva Speech AI server containers, run bash riva_stop.sh.

Clean-up

To clean-up the local Riva installation, run bash riva_clean.sh. This stops and removes all Riva related containers, as well as deletes the Docker volume or directory used to store the model files. The Docker images can also be removed, however, you will be asked for confirmation before removal.