Gst-nvds_text_to_speech (Alpha) =============================== The `Gst-nvds_text_to_speech` plugin performs speech synthesis on the input text. Currently it supports only x86 platform. By default, the plugin loads DS-Riva Text To Speech library (``libnvds_riva_tts.so``) to perform speech synthesis. The plugin provides a mechanism to load custom TTS low level library at runtime. .. note:: The `Gst-nvds_text_to_speech` plugin is being released as an alpha feature. .. note:: DS-Riva Text To Speech library uses gRPC API to access the Riva TTS service. The Riva TTS service should be started before using this plugin. And the gRPC C++ installation is required on the client side. The plugin accepts text (UTF8) Gst Buffers from upstream component. It transforms the text into audio Gst Buffer output. The DS-Riva TTS library (``libnvds_riva_tts.so``) generates raw audio data with F32LE format (float 32 bit Little Endian) at 22050 Hz sample rate. Library settings can be configured via YAML format file (by setting a property on `nvds_text_to_speech` gst plugin) which has multi-part settings for plugin control, Riva TTS service configurations. As shown in the diagram below, input text is send to Riva TTS service for speech synthesis. The final output is available as F32LE PCM audio at 22050 Hz. .. image:: /content/DS_plugin_gst-nvds_text_to_speech.png :align: center :alt: Gst-Nvds_text_to_speech Inputs and Outputs ------------------- This section summarizes the inputs, outputs, and communication facilities of the `Gst-nvds_text_to_speech` plugin with DS-Riva TTS implementation. * Input * Text GStreamer buffers * Control parameters * ``customlib-name``: Set a custom TTS library that the plugin loads to perform speech synthesis. By default, DS-Riva TTS library (``libnvds_riva_tts.so``) is set * ``create-speech-ctx-func``: Symbol name to create TTS speech context. Default: ``create_text_to_speech_ctx`` * ``config-file``: A text file to configure the plugin, DS-Riva TTS service requests. * Output * Raw audio GStreamer buffers containing the synthesized speech Features --------- The following table summarizes the features of the plugin. .. csv-table:: Gst-nvds_text_to_speech plugin features :file: ../text/tables/Gst-nvds_text_to_speech tables/DS_Plugin_gst-nvds_text_to_speech_features.csv :widths: 30, 30, 30 :header-rows: 1 DS-Riva TTS Yaml File Configuration Specifications ----------------------------------------------------- DS-Riva TTS configuration file uses YAML 1.2 file format: https://yaml.org/spec/1.2/spec.html * There are multiple parts in the configuration file. An example is located at ``/opt/nvidia/deepstream/deepstream/sources/apps/audio_apps/deepstream_asr_tts_app/riva_tts_conf.yml``. Each part has a ``name`` indicating a unique part name and a ``detail`` indicating the setting details. * ``name: riva_server`` part configures the Riva server URI in its corresponding node ``detail:``. * ``name: riva_tts_stream`` part configures Riva TTS service supported features in its corresponding node ``detail:``. * ``name: ds_riva_tts_plugin`` part configures DS-Riva TTS settings in its corresponding node ``detail:``. * A separator line with ``---`` is inserted between the 2 neighbor parts according to YAML specification. Gst Properties ---------------- The following tables describes the Gst properties of the `Gst-nvds_text_to_speech` plugin. .. csv-table:: riva_server: Configuration properties for Riva low level library :file: ../text/tables/Gst-nvds_text_to_speech tables/DS_Plugin_gst-nvds_text_to_speech_gst_riva_server_properties.csv :widths: 20, 20, 20, 20 :header-rows: 1 .. csv-table:: ds_riva_tts_stream: Configuration properties for Riva TTS service request :file: ../text/tables/Gst-nvds_text_to_speech tables/DS_Plugin_gst-nvds_text_to_speech_gst_riva_tts_stream_properties.csv :widths: 20, 20, 20, 20 :header-rows: 1 .. csv-table:: ds_riva_tts_plugin: Configuration properties for DS-Riva TTS library settings :file: ../text/tables/Gst-nvds_text_to_speech tables/DS_Plugin_gst-nvds_text_to_speech_gst_riva_tts_plugin_properties.csv :widths: 20, 20, 20, 20 :header-rows: 1 Riva TTS Service Initiation ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Please refer to https://docs.nvidia.com/deeplearning/riva/user-guide/docs/quick-start-guide.html#local-deployment-using-quick-start-scripts for the procedure to start Riva TTS service. The DS-Riva TTS library (``libnvds_riva_tts.so``) works with Riva Speech Skills 1.4.0-beta release or later. gRPC C++ Installation ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ For installing gRPC C++ shared libraries, please follow steps given at below link. Add -DBUILD_SHARED_LIBS=ON to the cmake build options. (Recommend to use 'make -j4' instead of 'make -j') https://grpc.io/docs/languages/cpp/quickstart/#install-grpc Ensure that the LD_LIBRARY_PATH environment variable includes the path to the installed gRPC libraries. Note: The gRPC C++ libraries are already installed on the DeepStream docker images and the corresponding commands to update the environment variables are added to the ``~/.profile`` file. After starting a new docker terminal, please run below command to update the LD_LIBRARY_PATH environment variable with gRPC installation path: ``$ source ~/.profile`` Sample Application ------------------ A sample application using the plugin is available here: ``sources/apps/audio_apps/deepstream_asr_tts_app.`` Please follow ``README`` to run the tests.