Custom Models#

The following NLP task is supported in Riva:

  • Punctuation and capitalization

Custom, supported NLP models trained with NVIDIA NeMo can be deployed in Riva using the riva-build and riva-deploy commands as documented in the Riva Build and Riva Deploy sections. You can now use .nemo files directly with riva-build through integrated nemo2riva conversion.

For punctuation models, you need to specify the Hydra configuration:

riva-build \
    --config-path=pkg://servicemaker.configs.punctuation \
    --config-name=base \
    output_path=<rmir_filename>:<encryption_key> \
    source_path=[<riva_filename>:<encryption_key>]

Or with a .nemo file:

riva-build \
    --config-path=pkg://servicemaker.configs.punctuation \
    --config-name=base \
    output_path=<rmir_filename>:<encryption_key> \
    'source_path=[{path: <nemo_filename>, nemo2riva: {format: onnx, key: tlt_encode}}]'

Where:

  • --config-path specifies the Hydra configuration path for punctuation models (can be pkg://servicemaker.configs.punctuation or an absolute path to a directory containing config files)

  • --config-name specifies the Hydra configuration name (use base for punctuation)

  • output_path specifies the path to the Riva rmir file that will be generated

  • source_path specifies a list of .riva or .nemo files to use as input

  • <rmir_filename> is the name of the RMIR file that will be generated

  • <riva_filename> or <nemo_filename> is the name of the .riva or .nemo file to use as input

  • <encryption_key> is the key used to encrypt the files. The encryption key for the pre-trained Riva models uploaded on NGC is tlt_encode.

The punctuation and capitalization task supports an optional parameter called language_code which must be set to the BCP-47 (https://www.rfc-editor.org/rfc/bcp/bcp47.txt) language code of the language the target model was trained on. When receiving ASR requests with the enable_automatic_punctuation Boolean flag set to true, the Riva server will look for a punctuation and capitalization model with the requested language code, and use it to add punctuation and capitalization to the ASR transcript.

Each of the tasks supports a set of arguments that enables you to configure your settings using the CLI. Use the format riva-build -h to view a list of available CLI inputs for each task.

Pretrained Models#