Launch UCF Studio¶

This section provides pointers and guides to get started building UCF applications using UCF Studio. It starts with building and deploying simple text based chatbot application using Speech skills and Dialog manager services. Next it explains how upgrade same application to speech based chatbot. This section should give good overview of using UCF Studio.

Run the following command in a terminal:
```
$ ucf_studio
```

Note

If running remotely via SSH, make sure that correct DISPLAY variable is exported before running the command.

UCF Studio can also be run remotely using other methods - Remote Access.

Once Studio starts, you should see the following home screen:

Repo Setup¶

Let’s start with the repository setup. NGC API Key needs to be set in order to get access to the NGC Repositories.

Get the NGC API Key - Refer to Generating Your NGC API Key.
Click on Registry from the Menu Bar at the top and then on Repo Setup.

This will open the Repo Setup dialog popup. Enter your NGC API Key and click on Ok.

It will take a few seconds for the operation to complete. If the entered API Key is valid, list of NGC Repositories will be populated and a success message popup will be shown.

Sync Microservices¶

Next we will synchronize microservices from NGC repository to list available microservices and view their information.

Click on Registry from the Menu Bar at the top and then on Fast Repo Setup.

This will open the Repo Sync popup dialog. Select UCF from the drop down list.

Click on Sync. A progress dialog will pop up. The process may take up to a few seconds. Once the sync is complete, a success confirmation dialog will be displayed and the microservice list on the left will be populated.

Create Application¶

This section demonstrates how to create a Text based Chatbot using UCF Studio with microservices provided by NVIDIA. Some instructions on this page are specific to the applications being created, refer to Creating an Application for more detailed information on creating applications.

Prerequisites¶

Refer to Development System and Deployment System for requirements and instructions to setup development and deployment systems.

Obtain API keys for the following services, as microservices used in this reference application depends on them:

Weather (weatherstack) - https://weatherstack.com/product
Places Of Interest (Yelp) - https://www.yelp.com/fusion
Map (Here) - https://developer.here.com/pricing

Create New Graph¶

We can now start creating the application graph.

Click on Create Graph at the center of the application.

This will display pop-up dialog for Graph Name. Enter a name like ucf-text-bot and click on Ok.

Click on Ok to open a blank canvas on which microservices can be added.

Search for Microservices and their Details¶

You may browse the available microservices using the list on the left

Microservices categorization can be changed for easier browsing

Microservice can be filtered and searched using text

Select a microservice from the list to show its details on the right panel

Add Microservices¶

1. Microservices can be added to the application by selecting them from the list on the left and drag-dropping them on to the canvas.

For this reference application, the following microservices will be required:

Riva Speech Skills

BotMaker Dialog Manager

BotMaker Text Web App

After adding above microservices, the graph now looks like the following:

Connect Microservices¶

Microservice can be connected by connecting their endpoints. Start dragging from the egress endpoint (ports on the right side of a microservice) to the ingress endpoint (ports on the left side of a microservice).

Configure Microservices¶

Parameters can be set by selecting a microservice on the canvas and setting the parameters from the “Properties” window on the right.

For convenience, parameters can also be loaded from a file. Save the following content to a file (e.g. params.yaml)

'Riva Speech Skills':
  imagePullSecrets:
  - name: ngc-docker-reg-secret
  buildModelEngines: "true" # Whether to build engines or use existing in PVC volume
  ngcModelConfigs: # List of NGC models or Bot Config for deployment
    - nvidia/riva/rmir_asr_citrinet_1024_en_us_str:2.6.0
    - nvidia/riva/rmir_tts_fastpitch_hifigan_en_us:2.6.0
    - eevaigoeixww/ucf-11-ea-release/misty_text_classification:1.7.3-ea
    - eevaigoeixww/ucf-11-ea-release/intent_slot_weather:1.7.3-ea
    - eevaigoeixww/ucf-11-ea-release/intent_slot_poi:1.7.3-ea
    - eevaigoeixww/ucf-11-ea-release/intent_slot_smalltalk:1.7.3-ea
    - nvidia/riva/rmir_nlp_question_answering_bert_base:2.6.0
    - nvidia/riva/rmir_nlp_named_entity_recognition_bert_base:2.6.0

'BotMaker Dialog Manager':
  imagePullSecrets:
    - name: ngc-docker-reg-secret
  botNgcPath: eevaigoeixww/ucf-11-ea-release/misty_bot:2.0.0-ea-x86_64 # NGC Path for Bot Config
  botConfigName: "misty_bot_config.yaml" # Only required if multiple botConfig present in bot directory
  # Override bot configs, follow pattern similar to below config
  botConfig:
    fulfillments:
      weather:
        parameters:
          api-key: "<WEATHER-API-KEY>"
      poi:
        parameters:
          api-key: "<POI-API-KEY>"
          here-api-key: "<HERE-MAP-API-KEY>"
      map:
        parameters:
          api-key: "<HERE-MAP-API-KEY>"

'BotMaker Text Web App':
  imagePullSecrets:
    - name: ngc-docker-reg-secret

The API keys can be obtained from:

Weather (weatherstack) - https://weatherstack.com/product

Places Of Interest (Yelp) - https://www.yelp.com/fusion

Map (Here) - https://developer.here.com/pricing

To load the above parameters file, right-click on the canvas to open the context menu and select Load Parameters.

This will open a file browser, navigate to the parameters file and load it.

Set secrets on the microservices. First open the Graph Configuration by right-clicking on the graph window to open the context menu and selecting Configurator.

Next select the Secrets window, click on Add to add a new secret. Enter details as shown below:

Now select Riva Speech Skills microservice on the canvas, this will show the Secrets section in the Property window on the right. For the ngc-api-key-secret, select ngc-api-key-secret from the dropdown.

Do the same for the BotMaker Dialog Manager microservice.

Save the Graph¶

Click on the Save Graph icon from the actions bar

This will open a file browser. Navigate to the directory of choice and save the graph.

Build Application¶

Click on the Build Graph icon from the actions bar.

This will open a file browser. Navigate to the directory of choice and click on Okay.
Building an application may take up to a few minutes. Once it is done, a success pop-up dialog will be shown containing the path where the deployable helm chart for the application has been generated.

Refer to the Deploy Application for instructions on deploying the helm chart.

Update Application¶

This section demonstrates how to create a Speech based Chatbot using UCF Studio with microservices provided by NVIDIA. These are incremental instructions on top to follow after creating the Create Application. This demonstrates how increasingly complex functionality can be added by just changing a few things in the application using UCF Studio.

Some instructions on this page are specific to the applications being created, refer to Creating an Application for more detailed information on creating applications.

Open Existing Graph¶

If the UCF Text Bot Application application graph is not already open, open it by clicking on the Open Graph button at the center of the application or Open Graph button on the action bar.

This will open a file browser, navigate to the UCF Text Bot Application application graph and open it. This will load the application graph in the canvas. If some of the microservices are out of view, right-click on the canvas to open the context menu and click on Focus on All to automatically pan and zoom the canvas so that all microservices are visible.

Add / Replace Microservices¶

For this application:

BotMaker Text Web App should be replaced by Speech Web App
Additional BotMaker Speech Controller microservice will be required

To do this select the BotMaker Text Web App microservice instance on the canvas and then right-click on it to open the context menu and click on Delete Selection to delete the microservice.

Next, add the BotMaker Speech Web App and BotMaker Speech Controller microservices by searching for them in the microservice list on the left and drag-dropping them on to the canvas. This is how the graph should look now:

Connect Microservices¶

speech-controller egress endpoint of BotMaker Speech Web App needs to be connected to grpc-api ingress endpoint of BotMaker Speech Controller
dialog-manager egress endpoint of BotMaker Speech Controller needs to be connected to dialog-manager ingress endpoint of BotMaker Dialog Manager
riva egress endpoint of BotMaker Speech Controller needs to be connected to riva-speech-api ingress endpoint of Riva Speech Skills

Do this by dragging from the egress endpoints to the ingress endpoints as shown in Connect Microservices. The graph should now look as below:

UCF Studio - Connect Speech Microservices

Load Parameters¶

Save the following content to a file (e.g. params.yaml):

'Riva Speech Skills':
  imagePullSecrets:
  - name: ngc-docker-reg-secret
  buildModelEngines: "true" # Whether to build engines or use existing in PVC volume
  ngcModelConfigs: # List of NGC models or Bot Config for deployment
    - nvidia/riva/rmir_asr_citrinet_1024_en_us_str:2.6.0
    - nvidia/riva/rmir_tts_fastpitch_hifigan_en_us:2.6.0
    - eevaigoeixww/ucf-11-ea-release/misty_text_classification:1.7.3-ea
    - eevaigoeixww/ucf-11-ea-release/intent_slot_weather:1.7.3-ea
    - eevaigoeixww/ucf-11-ea-release/intent_slot_poi:1.7.3-ea
    - eevaigoeixww/ucf-11-ea-release/intent_slot_smalltalk:1.7.3-ea
    - nvidia/riva/rmir_nlp_question_answering_bert_base:2.6.0
    - nvidia/riva/rmir_nlp_named_entity_recognition_bert_base:2.6.0

'BotMaker Dialog Manager':
  imagePullSecrets:
    - name: ngc-docker-reg-secret
  botNgcPath: eevaigoeixww/ucf-11-ea-release/misty_bot:2.0.0-ea-x86_64 # NGC Path for Bot Config
  botConfigName: "misty_bot_config.yaml" # Only required if multiple botConfig present in bot directory
  # Override bot configs, follow pattern similar to below config
  botConfig:
    fulfillments:
      weather:
        parameters:
          api-key: "<WEATHER-API-KEY>"
      poi:
        parameters:
          api-key: "<POI-API-KEY>"
          here-api-key: "<HERE-MAP-API-KEY>"
      map:
        parameters:
          api-key: "<HERE-MAP-API-KEY>"

'BotMaker Speech Controller':
  imagePullSecrets:
    - name: ngc-docker-reg-secret
  pipeline: speech_lite
  botNgcPath: eevaigoeixww/ucf-11-ea-release/misty_bot:2.0.0-ea-x86_64
  wordBoostFilePath: ''

'BotMaker Speech Web App':
  imagePullSecrets:
    - name: ngc-docker-reg-secret

Next, follow acquire API keys and load parameters as shown in UCF Text Bot Application section.
Set secrets on BotMaker Speech Controller microservice as shown in UCF Text Bot Application - Set Secrets.

Update Application Information¶

Right click on the canvas to open the context menu and click on Configuration.

This will open the Graph Configuration section at the bottom.

Change the application name to ucf-speech-bot and optionally update the description.

Save and Build the Application¶

From the the File menu at the top, select Save Graph As to the save the graph to a new file.

This will open a file browser, navigate to a directory of choice and save the graph.
Follow instructions in Build Application in the UCF Text Bot Application application section to build the application.

Deploy Application¶

Deploy on local MicroK8S cluster¶

Create Kubernetes secret required to pull the images:

$ microk8s kubectl create secret docker-registry ngc-docker-reg-secret --docker-server=nvcr.io --docker-username='$oauthtoken' --docker-password=<NGC_API_KEY>
If another name is used for the secret instead of ngc-docker-reg-secret, same should be updated in the application and the app should be rebuilt.

Create Kubernetes secret for the NGC API Key:

$ microk8s kubectl create secret generic ngc-api-key-secret --from-literal=NGC_API_KEY=<NGC_API_KEY>
If another secret name is used instead of ngc-api-key-secret or another key is used instead of NGC_API_KEY, same should be updated in the application and the app should be rebuilt.

Install the Local Path Provisioner if not already installed:

$ curl https://raw.githubusercontent.com/rancher/local-path-provisioner/v0.0.23/deploy/local-path-storage.yaml | sed 's/^  name: local-path$/  name: mdx-local-path/g' | microk8s kubectl apply -f -

Deploy the application helm chart:

$ microk8s helm3 install <release-name> <app-output-helm-chart>

Deploy application on AWS¶

Copy helm chart to AWS Instance:

scp -r -i /path/to/<your_key>.pem /path/to/<app-output-helm-chart>/   ubuntu@<aws_instance_ip>:

Install the Local Path Provisioner (on AWS Instance):

$ curl https://raw.githubusercontent.com/rancher/local-path-provisioner/v0.0.23/deploy/local-path-storage.yaml | sed 's/^  name: local-path$/  name: mdx-local-path/g' | kubectl apply -f -

Create Kubernetes secret required to pull the images (on AWS Instance):

$ kubectl create secret docker-registry ngc-docker-reg-secret --docker-server=nvcr.io --docker-username='$oauthtoken' --docker-password=<NGC_API_KEY>
If another name is used for the secret instead of ngc-docker-reg-secret, same should be updated in the application and the app should be rebuilt.

Create Kubernetes secret for the NGC API Key (on AWS Instance):

$ kubectl create secret generic ngc-api-key-secret --from-literal=NGC_API_KEY=<NGC_API_KEY>

If another secret name is used instead of ngc-api-key-secret or another key is used instead of NGC_API_KEY, same should be updated in the application and the app should be rebuilt.
Deploy the application helm chart (on AWS Instance):

$ helm install <release-name> <app-output-helm-chart>

Interact with the application¶

UCF Text Bot Application¶

Watch Pods:

Wait for all the application microservice pods to go into the running state

MicroK8s:

$ microk8s kubectl get pod
NAME                                                  READY   STATUS    RESTARTS     AGE
botmaker-dialog-manager-deployment-6d9d6fd949-88f7f   1/1     Running   0            28m
botmaker-text-web-app-deployment-8499c4b857-xjwcg     1/1     Running   0            28m
riva-speech-skills-deployment-74cc4c9457-mfm98        1/1     Running   0            28m

AWS:

$ kubectl get pods
NAME                                                  READY   STATUS    RESTARTS   AGE
botmaker-dialog-manager-deployment-ffdd98965-49rbk    1/1     Running   0          48m
botmaker-text-web-app-deployment-ddccf454-wsxh6       1/1     Running   0          48m
riva-speech-skills-deployment-7856cc499f-9bnkf        1/1     Running   0          48m

Service List

Get the list of services associated with the application

MicroK8s:

$ microk8s kubectl get svc
NAME                                       TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)              AGE
botmaker-dialog-manager-deployment-svc     ClusterIP   10.152.183.47    <none>        5000/TCP             32m
botmaker-text-web-app-deployment-service   NodePort    10.152.183.132   <none>        5001:31842/TCP       32m
kubernetes                                 ClusterIP   10.152.183.1     <none>        443/TCP              67d
riva-speech-skills-deployment-svc          ClusterIP   10.152.183.21    <none>        50051/TCP,8001/TCP   32m   29m

AWS:

$ kubectl get svc
NAME                                        TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)              AGE
botmaker-dialog-manager-deployment-svc      ClusterIP   10.102.53.150    <none>        5000/TCP             49m
botmaker-text-web-app-deployment-service    NodePort    10.96.23.19      <none>        5001:31666/TCP       49m
kubernetes                                  ClusterIP   10.96.0.1        <none>        443/TCP              151m
riva-speech-skills-deployment-svc           ClusterIP   10.101.176.114   <none>        50051/TCP,8001/TCP   49m

Accessing the webapp

The service of interest is botmaker-text-web-app-deployment-service. We will need the Node Port associated with the service. This port number will change for every deployment.

MicroK8s:
NodePort: 31842

Open http://<MicroK8s-Host-IP>:<Botmaker-Text-Web-App-NodePort>/bot in a browser to start interacting with the bot.

AWS:
NodePort: 31666

We will have to open this port to the internet so that we can access it.

Go to your instance’s Security tab and navigate to the security group. Edit the Inbound Rule to accept connections to the NodePort (in this case 31666) from Anywhere-IPv4 (0.0.0.0/0).

After saving open https://<aws_instance_ip>:<NodePort>/bot in a browser to start interacting with the bot.

Application WebUI

On opening the URL, you should be greeted with the following UI:

You can enter the question/statement in the text box and click on Submit.

The bot answers to questions/statements like “Hello”, “How are you?”, “How is the weather in New York?”

UCF Speech Bot¶

Follow the steps in UCF Text Bot Application to wait till all pods are running and to get the list of services.

Accessing the webapp

The service of interest is botmaker-speech-webapp-deployment-service. We will need the Node Port associated with the service. This port number will change for every deployment.

MicroK8s:
NodePort: 31746

Open https://<MicroK8s-Host-IP>:<Botmaker-Speech-WebApp-NodePort>/bot in a browser to start interacting with the bot.

AWS:

NodePort: 31666

We will have to open this port to the internet so that we can access it.

Go to your instance’s Security tab and navigate to the security group. Edit the Inbound Rule to accept connections to the NodePort (in this case 31666) from Anywhere-IPv4 (0.0.0.0/0).

After saving open https://<aws_instance_ip>:<NodePort>/bot in a browser to start interacting with the bot.

Application WebUI

On opening the URL, you should be greeted with the following UI:

After giving permissions to your microphone, you should be able to provide speech input in the webapp.

Click on the green mic button to talk to the bot.

Click on the yellow “Unmute Speech” button to hear audio replies from the bot.

Alternatively, you can enter the question/statement in the text box and click on Submit and/or view the replies in text format

The bot answers to questions/statements like “Hello”, “How are you?”, “How is the weather in New York?”