Gst-nvds_text_to_speech (Alpha) =============================== The `Gst-nvds_text_to_speech` plugin performs speech synthesis on the input text. Currently it supports only x86 platform. By default, the plugin loads `DS-Riva` Text To Speech library (``libnvds_riva_tts.so``) to perform speech synthesis. The plugin provides a mechanism to load custom TTS low level library at runtime. .. note:: * The `Gst-nvds_text_to_speech` plugin is being released as an alpha feature. * `DS-Riva` Text To Speech library uses gRPC API to access the Riva TTS service. The Riva TTS service should be started before using this plugin. And the gRPC C++ installation is required on the client side. The plugin accepts text (UTF8) Gst Buffers from upstream component. It transforms the text into audio Gst Buffer output. The DS-Riva TTS library (``libnvds_riva_tts.so``) generates raw audio data with F32LE format (float 32 bit Little Endian) at 22050 Hz sample rate. Library settings can be configured via YAML format file (by setting a property on `nvds_text_to_speech` gst plugin) which has multi-part settings for plugin control, Riva TTS service configurations. As shown in the diagram below, input text is send to Riva TTS service for speech synthesis. The final output is available as F32LE PCM audio at 22050 Hz. .. image:: /content/DS_plugin_gst-nvds_text_to_speech.png :align: center :alt: Gst-Nvds_text_to_speech Inputs and Outputs ------------------- This section summarizes the inputs, outputs, and communication facilities of the `Gst-nvds_text_to_speech` plugin with DS-Riva TTS implementation. * Input * Text GStreamer buffers * Control parameters * ``customlib-name``: Set a custom TTS library that the plugin loads to perform speech synthesis. By default, DS-Riva TTS library (``libnvds_riva_tts.so``) is set * ``create-speech-ctx-func``: Symbol name to create TTS speech context. Default: ``create_text_to_speech_ctx`` * ``config-file``: A text file to configure the plugin, DS-Riva TTS service requests. * Output * Raw audio GStreamer buffers containing the synthesized speech Features --------- The following table summarizes the features of the plugin. .. csv-table:: Gst-nvds_text_to_speech plugin features :file: ../text/tables/Gst-nvds_text_to_speech tables/DS_Plugin_gst-nvds_text_to_speech_features.csv :widths: 30, 30, 30 :header-rows: 1 DS-Riva TTS Yaml File Configuration Specifications ----------------------------------------------------- DS-Riva TTS configuration file uses YAML 1.2 file format: https://yaml.org/spec/1.2/spec.html. * There are multiple parts in the configuration file. An example is located at ``/opt/nvidia/deepstream/deepstream/sources/apps/audio_apps/deepstream_asr_tts_app/riva_tts_conf.yml``. Each part has a ``name`` indicating a unique part name and a ``detail`` indicating the setting details. * ``name: riva_server`` part configures the Riva server URI in its corresponding node ``detail:``. * ``name: riva_tts_stream`` part configures Riva TTS service supported features in its corresponding node ``detail:``. * ``name: ds_riva_tts_plugin`` part configures DS-Riva TTS settings in its corresponding node ``detail:``. * A separator line with ``---`` is inserted between the 2 neighbor parts according to YAML specification. Gst Properties ---------------- The following tables describes the Gst properties of the `Gst-nvds_text_to_speech` plugin. .. csv-table:: riva_server: Configuration properties for Riva low level library :file: ../text/tables/Gst-nvds_text_to_speech tables/DS_Plugin_gst-nvds_text_to_speech_gst_riva_server_properties.csv :widths: 20, 20, 20, 20 :header-rows: 1 .. csv-table:: ds_riva_tts_stream: Configuration properties for Riva TTS service request :file: ../text/tables/Gst-nvds_text_to_speech tables/DS_Plugin_gst-nvds_text_to_speech_gst_riva_tts_stream_properties.csv :widths: 20, 20, 20, 20 :header-rows: 1 .. csv-table:: ds_riva_tts_plugin: Configuration properties for DS-Riva TTS library settings :file: ../text/tables/Gst-nvds_text_to_speech tables/DS_Plugin_gst-nvds_text_to_speech_gst_riva_tts_plugin_properties.csv :widths: 20, 20, 20, 20 :header-rows: 1 Riva TTS Service Initiation ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Refer to https://docs.nvidia.com/deeplearning/riva/user-guide/docs/quick-start-guide.html#local-deployment-using-quick-start-scripts for the procedure to start Riva TTS service. The DS-Riva TTS library (``libnvds_riva_tts.so``) works with Riva Speech Skills 1.4.0-beta release or later. gRPC C++ Installation ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 1. To install the `gRPC C++` shared libraries, follow the steps here: https://grpc.io/docs/languages/cpp/quickstart/#install-grpc. Add ``-DBUILD_SHARED_LIBS=ON`` to the `cmake` build options. (Recommend to use ``'make -j4'`` instead of ``'make -j'``) 2. Ensure that the ``LD_LIBRARY_PATH`` environment variable includes the path to the installed `gRPC` libraries. .. Note:: The gRPC C++ libraries are already installed on the DeepStream docker images and the corresponding commands to update the environment variables are added to the ``~/.profile`` file. 3. After starting a new docker terminal, run the below command to update the ``LD_LIBRARY_PATH`` environment variable with `gRPC` installation path: :: $ source ~/.profile Sample Application ------------------ A sample application using the plugin is available here: ``sources/apps/audio_apps/deepstream_asr_tts_app``. Follow the ``README`` to run the tests.