Export Models Trained with NeMo/TLT to Jarvis

Models trained from frameworks such as NVIDIA NeMo and NVIDIA Transfer Learning Toolkit (TLT) normally have the format .nemo and .tlt, respectively. To use those models in Jarvis, users needs to convert the model checkpoints to .ejrvs format for building and deploying with Jarvis ServiceMaker.

Note: If users trained their models with the recent NeMo release (1.0.0.b4), they can directly use tlt … export from the TLT launcher to export the NeMo models to the Jarvis required format. For older NeMo releases (1.0.0.b1-b3), this export path might work for some but not all models, users should use at their own risk. For even older NeMo releases (before 1.0.0.b1), it will not work due to missing artifacts that Jarvis ServiceMaker requires.

TLT Export for NeMo/TLT

1. Follow the TLT Launcher Quick Start Guide instructions to setup.

2. Mount the directory that contains the model checkpoints to the TLT Docker image. The TLT launcher uses Docker containers under the hood. The launcher instance can be configured in the ~/.tlt_mounts.json file. A sample is provided below.

   {
      "Mounts":[
             {
                 "source": "/home/nemo_files",
                 "destination": "/workspace/tlt/samples/nemo_files"
             }
       ],
       "Envs": [
             {
                 "variable": "TEST_VARIABLE",
                 "value": "1"
             },
             {
                 "variable": "SAMPLE_VARIABLE",
                 "value": "1"
             }
       ],
       "DockerOptions":{
             "shm_size": "16G",
             "ulimits": {
                 "memlock": -1,
                 "stack": 67108864
             }
       }
   }
   

3. Use tlt … export to convert either NeMo or TLT checkpoints to the Jarvis format. The example below shows how users can export a Jasper model trained in NeMo, where -m is used to specify the Jasper model checkpoints location and -e is used to specify the path to a experiment spec file.

   tlt speech_to_text -m /workspace/tlt/samples/nemo_files/jasper.nemo -e /workspace/tlt/samples/nemo_files/export.yaml
   

   Here is an example experiment spec file (export.yaml):

   # Name of the .nemo EFF archive to be loaded/exported.
   restore_from: /workspace/tlt/samples/nemo_files/jasper.nemo
   
   # Set export format: JARVIS
   export_format: JARVIS
   
   # Output EFF archive containing JARVIS.
   export_to: /workspace/tlt/samples/nemo_files/jasper.erjvs
   

Other conversational AI tasks supported by TLT includes:

• intent_slot_classification

• punctuation_and_capitalization

• question_answering

• text_classification

• token_classification

More details can be found in tlt --help.