Joint Intent and Slot Classification =============================================== Joint Intent and Slot classification is a method for classifying an Intent and detecting all relevant Slots (Entities) for the Intent in a query. For example, in the query "What is the weather in Santa Clara tomorrow morning?", we would like to classify the query as a weather Intent, detect Santa Clara as a location Slot, and tomorrow morning as a date_time Slot. Intent and Slot names are usually task-specific and defined as labels in the training data. This is a fundamental step that is executed in any task-driven conversational assistant. Our BERT-based model implementation allows you to train and detect both of these tasks together. TLT provides a sample notebook to outline the end-to-end workflow on how to train an Intent and Slot classification model using TLT and deploy it in Jarvis format on `NGC resources`_. .. _NGC resources: https://ngc.nvidia.com/catalog/resources/nvidia:tlt-jarvis:intentslotclassification_notebook Downloading Sample Spec files ----------------------------- Before proceeding, let's download sample spec files that we would need for the rest of the subtasks. .. code:: tlt intent_slot_classification download_specs -r /results/intent_slot_classification/default_specs/ \ -o /specs/nlp/intent_slot_classification Data Format ----------- When training the model, the dataset should be first converted to the required data format, which requires these files: - :code:`dict.intents.csv` - A list of all Intent names in the data. Only one line per intent name is allowed. - :code:`dict.slots.csv` - A list of all Slot names in the data. Only one line per slot name is allowed. - :code:`train.tsv/test.tsv` - A list of original queries, one per line, with the intent number separated by a tab (e.g., "what alarms do i have set right now 0"). Intent numbers are set according to the intent line in the intent dictionary file (:code:`dict.intents.csv`), starting from 0. The first line in these files should contain the header line "sentence label". - :code:`train_slot.tvs/test_slot.tsv` - A list that contains one line per a query, when each word from the original text queried, is replaced by a token number from the slots dictionary file (dict.slots.csv), counted starting from 0. All the words that do not contain a relevant slot are replaced by an 'out-of-scope' token number, which is also a part of the slot dictionary file, usually as the last entry there. Example of a line from these files: "54 0 0 54 54 12 12" (the numbers are separated by a space). These files do not contain any header line. Dataset Conversion ------------------ To convert to the format of the model data, use the dataset_convert utility, which implements conversion for the Assistant dataset, which you can download `here `_. Or you can write your own converter for the format that you are using for data annotation. For a dataset that follows your own annotation format, we recommend using one text file for all samples of the same intent, with the name of the file as the name of the intent. Use one line per query, with brackets to define slot names. This is very similar to the assistant format, and you can adapt this converter utility or your own format with small changes: :: did i set an alarm to [alarm_type : wake up] in the [timeofday : morning] To run the dataset_converter, use this command: .. code:: !tlt intent_slot_classification dataset_convert \ source_data_dir=`source_data_dir` \ target_data_dir=`target_data_dir` - :code:`source_data_dir`: The directory location of the your dataset - :code:`target_data_dir`: The directory location where the converted dataset should be saved After conversion, the `target_data_dir` should contain the following files: .. code:: . |--target_data_dir |-- dict.intents.csv |-- dict.slots.csv |-- train.tsv |-- train_slots.tsv |-- test.tsv |-- test_slots.tsv Model Training -------------- .. _model_training_intent_and_slot: The following is an example of the config spec for training :code:`train.yaml` file. You can change any of these parameters and pass them to the :code:`train` command. .. code:: # Name of the file where trained model will be saved. save_to: trained-model.tlt data_dir: ??? model: class_balancing: null # choose from [null, weighted_loss]. weighted_loss enables the weighted class balancing of the loss, may be used for handling unbalanced classes intent_loss_weight: 0.6 # relation of intent to slot loss in total loss (between 0 to 1) pad_label: -1 # value to pad the inputs ignore_extra_tokens: false ignore_start_end: true # do not use first and last token for slot training tokenizer: tokenizer_name: ${model.language_model.pretrained_model_name} # or sentencepiece vocab_file: null # path to vocab file tokenizer_model: null # only used if tokenizer is sentencepiece special_tokens: null language_model: max_seq_length: 50 pretrained_model_name: bert-base-uncased lm_checkpoint: null config_file: null # json file, precedence over config config: null head: num_output_layers: 2 fc_dropout: 0.1 training_ds: prefix: train batch_size: 32 num_workers: 2 validation_ds: prefix: dev batch_size: 32 num_workers: 2 optim: name: adam lr: 2e-5 # optimizer arguments betas: [0.9, 0.999] weight_decay: 0.01 # scheduler setup sched: name: WarmupAnnealing # Scheduler params warmup_steps: null warmup_ratio: 0.1 last_epoch: -1 # pytorch lightning args monitor: val_loss reduce_on_plateau: false +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ | **Parameter** | **Data Type** | **Default** | **Description** | +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ | save_to | string | trained-model.tlt | The filename of the trained model | +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ | data_dir | string | -- | The path of the data converted to the specified format | +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ | model.class_balancing | string | null | Choose from [null, weighted_loss]. The weighted_loss enables weighted class balancing of the loss | +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ | model.intent_loss_weight | float | 0.6 | The relation of intent-to-slot loss in the total loss | +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ | model.pad_label | integer | -1 | A value to pad the inputs | +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ | model.ignore_extra_tokens | boolean | false | A flag that specifies whether to ignore extra tokens | +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ | model.ignore_start_end | boolean | true | A flag that specifies whether to not use the first and last token for slot training | +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ | model.tokenizer.tokenizer_name | string | Will be filled automatically based on model.language_model.pretrained_model_name | The tokenizer name | +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ | model.tokenizer.vocab_file | string | null | The path to the tokenizer vocabulary | +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ | model.tokenizer.tokenizer_model | string | null | The path to tokenizer model (only for the sentencepiece tokenizer) | +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ | model.tokenizer.special_tokens | string | null | Special tokens for the tokenizer (if they exist) | +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ | model.language_model.max_seq_length | integer | 50 | The maximum length of the input queries (in tokens) | +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ | model.language_model.pretrained_model_name| string | bert-base-uncased | The pre-trained language model name (choose from `bert-base-cased`, `bert-base-uncased`, | | | | | `megatron-bert-345m-cased` and `megatron-bert-345m-uncased`) | +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ | model.language_model.lm_checkpoint | string | null | The path to the pre-trained language-model checkpoint | +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ | model.language_model.config_file | string | null | The path to the pre-trained language-model config file | +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ | model.language_model.config | dictionary | null | The config for the pre-trained language model | +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ | model.head.num_output_layers | integer | 2 | The number of fully connected layers of the Classifier on top of the BERT model | +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ | model.head.fc_dropout | float | 0.1 | The dropout ratio of the fully connected layers | +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ | training_ds.prefix | string | train | A prefix for the training file names | +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ | training_ds.num_workers | integer | 2 | The number of worker threads for training | +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ | training_ds.batch_size | integer | 32 | The training data batch size | +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ | validation_ds.prefix | string | dev | A prefix for the validation file names | +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ | validation_ds.num_workers | integer | 2 | The number of worker threads for validation | +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ | validation_ds.batch_size | integer | 32 | The validation data batch size | +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ | optim.name | string | adam | An optimizer to use for training | +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ | optim.lr | float | 2e-5 | The learning rate to use for training | +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ | optim.weight_decay | float | 0.01 | The weight decay to use for training | +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ | optim.sched.name | string | WarmupAnnealing | The warmup schedule | +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ | optim.sched.warmup_ratio | float | 0.1 | The warmup ratio | +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ The following is an example of the command for training the model: .. code:: !tlt intent_slot_classification train -e /specs/nlp/intent_slot_classification/train.yaml \ data_dir=PATH_TO_DATA \ trainer.max_epochs=50 \ -g 1 \ -k $KEY Required Arguments for Training ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ * :code:`-e`: The experiment-specification file to set up training * :code:`data_dir`: The dataset directory * :code:`-k`: The encryption key Optional Arguments ^^^^^^^^^^^^^^^^^^ * :code:`trainer.max_epochs`: The number of training epochs * :code:`-g`: The number of GPUs to use for training .. Note:: You can use other arguments to override fields in the specification file. To do so, use the name of the config parameter with a desired value and pass it as a parameter in the script call (e.g., :code:`model.class_balancing=weighted_loss`). Training Procedure ^^^^^^^^^^^^^^^^^^ At the start of evaluation, TLT will print out a log of the experiment specification, a summary of the training dataset, and the model architecture. As the model starts training, you should see a progress bar per epoch. During training, after each epoch TLT will display accuracy metrics on the validation dataset for every Intent and Slot separately, as well as the total accuracy. You can expect these numbers to grow up to 50-100 epochs, depending on the size of the trained data. At the end of training, TLT will save the best checkpoint on the validation dataset at the path specified by the experiment spec file before finishing. .. code:: GPU available: True, used: True TPU available: None, using: 0 TPU cores LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0,1,2] [NeMo W 2021-01-28 14:52:19 exp_manager:299] There was no checkpoint folder at checkpoint_dir :results/checkpoints. Training from scratch. [NeMo I 2021-01-28 14:52:19 exp_manager:186] Experiments will be logged at results ... label precision recall f1 support weather.weather (label_id: 0) 0.00 0.00 0.00 128 weather.temprature (label_id: 1) 0.00 0.00 0.00 0 weather.temperature_yes_no (label_id: 2) 0.00 0.00 0.00 0 weather.rainfall (label_id: 3) 0.00 0.00 0.00 0 weather.rainfall_yes_no (label_id: 4) 0.00 0.00 0.00 0 weather.snow (label_id: 5) 0.00 0.00 0.00 0 weather.snow_yes_no (label_id: 6) 0.00 0.00 0.00 0 weather.humidity (label_id: 7) 0.00 0.00 0.00 0 weather.humidity_yes_no (label_id: 8) 0.00 0.00 0.00 0 weather.windspeed (label_id: 9) 0.00 0.00 0.00 0 weather.sunny (label_id: 10) 0.00 0.00 0.00 0 weather.cloudy (label_id: 11) 0.00 0.00 0.00 0 weather.alert (label_id: 12) 0.00 0.00 0.00 0 context.weather (label_id: 13) 0.00 0.00 0.00 0 context.continue (label_id: 14) 0.00 0.00 0.00 0 context.navigation (label_id: 15) 0.00 0.00 0.00 0 context.rating (label_id: 16) 0.00 0.00 0.00 0 context.distance (label_id: 17) 0.00 0.00 0.00 0 ------------------- micro avg 0.00 0.00 0.00 128 macro avg 0.00 0.00 0.00 128 weighted avg 0.00 0.00 0.00 128 Model Fine-tuning ----------------- The fine-tuning process will continue training using a previously trained model; however, fine-tuning is not used much in this task since training is fast and the trained model is specific to a dataset. The following is an example spec for fine-tuning of the model: .. code:: # Name of the .tlt file where trained model will be saved or restored. save_to: finetuned-model.tlt restore_from: trained-model.tlt # Directory containing both fine-tuning and validation data. data_dir: ??? # Training data set configurations finetuning_ds: prefix: train batch_size: 32 shuffle: true num_samples: -1 num_workers: 2 # Validation data set configurations validation_ds: prefix: dev batch_size: 32 shuffle: false num_samples: -1 num_workers: 2 optim: name: adam lr: 2e-5 weight_decay: 0.01 +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ | **Parameter** | **Data Type** | **Default** | **Description** | +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ | restore_from | string | trained-model.tlt | The path to the pre-trained model | +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ | save_to | string | finetuned-model.tlt | The path to saved, trained model | +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ | data_dir | string | -- | The path to the data that is converted to the specified format | +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ | trainer.max_epochs | integer | 5 | The maximum number of epochs to train the model | +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ | finetuning_ds.prefix | string | train | A prefix for the training filenames | +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ | finetuning_ds.num_workers | integer | 2 | The number of worker threads for training | +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ | finetuning_ds.batch_size | integer | 32 | The training data batch size | +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ | finetuning_ds.shuffle | bool | True | A flag specifying whether to shuffle the training data | +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ | finetuning_ds.num_samples | integer | -1 | The number of samples to use from the training dataset (use -1 to specify all samples) | +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ | validation_ds.prefix | string | dev | A prefix for the validation filenames | +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ | validation_ds.num_workers | integer | 2 | The number of worker threads for validation | +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ | validation_ds.batch_size | integer | 32 | The validation data batch size | +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ | validation_ds.shuffle | bool | False | A flag specifying whether to shuffle the validation data | +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ | validation_ds.num_samples | integer | -1 | The number of samples to use from the validation dataset (use -1 to specify all samples) | +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ | optim.name | string | adam | The optimizer to use for training | +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ | optim.lr | float | 1e-5 | The learning rate to use for training | +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ | optim.weight_decay | float | 0.01 | The weight decay to use for training | +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ Use the following command to fine-tune the model: .. code:: !tlt intent_slot_classification finetune \ -e /specs/nlp/intent_slot_classification/finetune.yaml \ -g 1 \ data_dir=PATH_TO_DATA \ trainer.max_epochs=5 \ -k $KEY Required Arguments for Fine-tuning ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ * :code:`-e`: The experiment specification file to set up fine-tuning * :code:`data_dir`: The dataset directory * :code:`-k`: The encryption key Optional Arguments ^^^^^^^^^^^^^^^^^^ * :code:`trainer.max_epochs`: The number of training epochs * :code:`-g`: The number of GPUs to use in evaluation in a multi-GPU scenario (default: 1) .. Note:: You can use other arguments to override fields in the specification file. To do so, use the name of the config parameter with a desired value and pass it as a parameter in the script call (e.g., :code:`model.class_balancing=weighted_loss`). Fine-tuning Procedure ^^^^^^^^^^^^^^^^^^^^^ Fine-tuning the procedure and logs will look similar to the procedure described in the :ref:`Model Training ` section, with the addition of the model that .. TODO: CTaylor: the link above doesn't seem to be formatted correctly is initially loaded from a previously trained checkpoint. Model Evaluation ---------------- The following is an example spec to evaluate the pre-trained model: .. code:: restore_from: trained-model.tlt data_dir: ??? test_ds: prefix: test num_workers: 2 batch_size: 32 shuffle: false num_samples: -1 +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ | **Parameter** | **Data Type** | **Default** | **Description** | +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ | restore_from | string | trained-model.tlt | The path to the pre-trained model | +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ | data_dir | string | -- | The path to the data converted to the specified format | +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ | test_ds.prefix | string | test | A prefix for the training file names | +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ | test_ds.num_workers | integer | 2 | The number of worker threads for training | +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ | test_ds.batch_size | integer | 32 | The training data batch size | +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ | test_ds.shuffle | bool | False | A flag specifying whether to shuffle the training data | +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ | test_ds.num_samples | integer | -1 | The number of samples to use from the training dataset (use -1 to specify all samples) | +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ Use the following command to evaluate the model: .. code:: !tlt intent_slot_classification evaluate \ -e /specs/nlp/intent_slot_classification/evaluate.yaml \ data_dir=PATH_TO_DATA Required Arguments for Evaluation ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ * :code:`-e`: The experiment specification file to set up evaluation * :code:`data_dir`: Path to the pre-processed data to run evaluation on Evaluation Procedure ^^^^^^^^^^^^^^^^^^^^ After the previously trained model is initialized, it will run evaluation against the provided test set. You should see metrics for each Intent and Slot in the dataset and accumulative metrics, as shown below: .. code:: ... social_post (label_id: 55) 100.00 36.84 53.85 19 social_query (label_id: 56) 57.69 83.33 68.18 18 takeaway_order (label_id: 57) 72.00 94.74 81.82 19 takeaway_query (label_id: 58) 87.50 73.68 80.00 19 transport_query (label_id: 59) 82.35 73.68 77.78 19 transport_taxi (label_id: 60) 100.00 100.00 100.00 18 transport_ticket (label_id: 61) 79.17 100.00 88.37 19 transport_traffic (label_id: 62) 100.00 89.47 94.44 19 weather_query (label_id: 63) 92.31 63.16 75.00 19 ------------------- micro avg 66.54 66.54 66.54 1076 macro avg 58.77 60.11 56.48 1076 weighted avg 64.62 66.54 62.32 1076 Model Inference ---------------- To run inference on the model, specify the list of examples in the spec: .. code:: input_batch: - 'set alarm for seven thirty am' - 'lower volume by fifty percent' - 'what is my schedule for tomorrow' In the output, for each input query, the recognized intent and a list of slots for each word will be printed, as shown in the following example: .. code:: Query : set alarm for seven thirty am Predicted Intent: alarm_set Predicted Slots: O O O time time time Use the following command to run inference: .. code:: !tlt intent_slot_classification infer \ -e /specs/nlp/intent_slot_classification/infer.yaml \ -m trained-model.tlt \ Required Arguments for Inference ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ * :code:`-e`: The experiment specification file to set up inference. This requires the :code:`input_batch` with a list of examples to run inference on. * :code:`-m`: The path to the pre-trained model checkpoint from which to infer. The file should have a :code:`.tlt` extension. Inference Procedure ^^^^^^^^^^^^^^^^^^^ After the trained model is loaded, it will run on the provided set of query examples and display the classified Intents and Slots. .. code:: ... [NeMo I 2021-01-29 09:52:21 infer:76] The prediction results of some sample queries with the trained model: [NeMo I 2021-01-29 09:52:21 infer:78] Query : set alarm for seven thirty am [NeMo I 2021-01-29 09:52:21 infer:79] Predicted Intent: alarm_set [NeMo I 2021-01-29 09:52:21 infer:80] Predicted Slots: O O O time time time [NeMo I 2021-01-29 09:52:21 infer:78] Query : lower volume by fifty percent [NeMo I 2021-01-29 09:52:21 infer:79] Predicted Intent: audio_volume_up [NeMo I 2021-01-29 09:52:21 infer:80] Predicted Slots: O O O O O [NeMo I 2021-01-29 09:52:21 infer:78] Query : what is my schedule for tomorrow [NeMo I 2021-01-29 09:52:21 infer:79] Predicted Intent: calendar_query [NeMo I 2021-01-29 09:52:21 infer:80] Predicted Slots: O O O O O date Model Export ------------ The following is an example of the spec file for model export: .. code:: # Name of the .tlt EFF archive to be loaded/model to be exported. restore_from: trained-model.tlt # Set export format: ONNX | JARVIS export_format: JARVIS # Output EFF archive containing JARVIS. export_to: exported-model.ejrvs +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ | **Parameter** | **Data Type** | **Default** | **Description** | +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ | restore_from | string | trained-model.tlt | The path to the pre-trained model | +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ | export_format | string | JARVIS | The export format (either "ONNX" or "JARVIS") | +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ | export_to | string | exported-model.ejrvs | The path to the exported model | +-------------------------------------------+-----------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+ To export a pre-trained model, run the following: .. code:: ### For export to Jarvis !tlt intent_slot_classification export \ -e /specs/nlp/intent_slot_classification/export.yaml \ -m finetuned-model.tlt \ -k $KEY Required Arguments for Export ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ * :code:`-e`: The experiment specification file to set up inference. This requires the :code:`input_batch` with the list of examples to run inference on. * :code:`-m`: The path to the pre-trained model checkpoint from which to infer. The file should have a :code:`.tlt` extension. * :code:`-k`: The encryption key Model Deployment ---------------- You can use the Jarvis framework for the deployment of the trained model in the runtime. For more details, refer to the `Jarvis documentation`_ .. _Jarvis documentation: https://docs.nvidia.com/deeplearning/jarvis/user-guide/docs/model-servicemaker.html