Important

You are viewing the NeMo 2.0 documentation. This release introduces significant changes to the API and a new library, NeMo Run. We are currently porting all features from NeMo 1.0 to 2.0. For documentation on previous versions or features not yet available in 2.0, please refer to the NeMo 24.07 documentation.

Deploy to Production with C++ backend#

Warning

TN/ITN transitioned from NVIDIA/NeMo repository to a standalone NVIDIA/NeMo-text-processing repository. All updates and discussions/issues should go to the new repository.

NeMo-text-processing provides tools to deploy TN and ITN for production [TEXTPROCESSING-DEPLOYMENT2]. It uses Sparrowhawk [TEXTPROCESSING-DEPLOYMENT1] – an open-source C++ framework by Google. The grammars written with NeMo-text-processing can be exported into an OpenFST Archive File (FAR) and dropped into Sparrowhawk.

Requirements#

Quick Start#

Examples how to run:

# export English TN grammars and return prompt inside docker container
cd NeMo-text-processing/tools/text_processing_deployment
bash export_grammars.sh --GRAMMARS=tn_grammars --LANGUAGE=en --INPUT_CASE=cased

# export English ITN grammars and return prompt inside docker container
cd NeMo-text-processing/tools/text_processing_deployment
bash export_grammars.sh --GRAMMARS=itn_grammars --LANGUAGE=en

Arguments:#

  • GRAMMARS - tn_grammars or itn_grammars to export either TN or ITN grammars.

  • LANGUAGE - en for English. Click here for full list of languages.

  • INPUT_CASE - cased or lower_cased (ITN has no differentiation between these two, only used for TN).

  • MODE - By default export which returns prompt inside the docker. If --MODE=test runs NeMo-text-processing pytests inside container.

  • OVERWRITE_CACHE - Whether to re-export grammars or load from cache. By default True.

  • FORCE_REBUILD - Whether to rebuild docker image in cased of updated dependencies. By default False.

Detailed pipeline#

export_grammars.sh runs the following steps in sequence:

Go to script folder:

cd NeMo-text-processing/tools/text_processing_deployment

  1. Grammars written in Python are exported to OpenFST archive files (FAR). Specifically, grammars ClassifyFst and VerbalizeFst from nemo_text_processing are exported and saved to ./LANGUAGE/classify/tokenize_and_classify.far and ./LANGUAGE/verbalize/verbalize.far respectively.

python pynini_export.py <--output_dir .> <--grammars GRAMMARS> <--input_case INPUT_CASE> <--language LANGUAGE>

Warning

TN and ITN grammars are saved to the same file by default.

  1. Docker image is built with dependencies, including Thrax and Sparrowhawk.

bash docker/build.sh

  1. Plugs in grammars into production backend by mounting grammar directory LANGUAGE/classify/ and LANGUAGE/verbalize/ inside docker. Returns docker prompt.

# launch container with the exported grammars
bash docker/launch.sh

  1. Runs system in docker container.

echo "ITN result: two dollars fifty. TN result: $2.50" | ../../src/bin/normalizer_main --config=sparrowhawk_configuration.ascii_proto

This returns “ITN result: $2.50. TN result: two dollars fifty cents”

See WFST Resources for more details.

References#

[TEXTPROCESSING-DEPLOYMENT1]

Alexander Gutkin, Linne Ha, Martin Jansche, Knot Pipatsrisawat, and Richard Sproat. Tts for low resource languages: a bangla synthesizer. In 10th Language Resources and Evaluation Conference. 2016.

[TEXTPROCESSING-DEPLOYMENT2]

Yang Zhang, Evelina Bakhturina, Kyle Gorman, and Boris Ginsburg. Nemo inverse text normalization: from development to production. 2021. arXiv:2104.05055.

On this page