Launch UCS Studio
This section provides pointers and guides to get started building UCS Applications using UCS Studio. It starts with building and deploying simple text based chatbot application using Speech skills and Dialog manager services. Next it explains how upgrade same application to speech based chatbot. This section should give good overview of using UCS Studio.
Run the following command in a terminal:
$ ucf_studio
Note
If running remotely via SSH, make sure that correct
DISPLAY
variable is exported before running the command.UCS Studio can also be run remotely using other methods - Remote Access.
Once Studio starts, you should see the following home screen:
Repo Setup
Let’s start with the repository setup. NGC API Key needs to be set in order to get access to the NGC Repositories.
Get the NGC API Key - Refer to Generating Your NGC API Key.
Click on Registry from the Menu Bar at the top and then on Repo Setup.
This will open the Repo Setup dialog popup. Enter your NGC API Key and click on Ok.
It will take a few seconds for the operation to complete. If the entered API Key is valid, list of NGC Repositories will be populated and a success message popup will be shown.
Sync Microservices
Next we will synchronize microservices from NGC repository to list available microservices and view their information.
Click on Registry from the Menu Bar at the top and then on Fast Repo Setup.
This will open the Repo Sync popup dialog. Select
UCS
from the drop down list.
Click on Sync. A progress dialog will pop up. The process may take up to a few seconds. Once the sync is complete, a success confirmation dialog will be displayed and the microservice list on the left will be populated.
Create an Reference Application
This section demonstrates how to create a Text based Chatbot using UCS Studio with microservices provided by NVIDIA. Some instructions on this page are specific to the applications being created, refer to Creating an Application for more detailed information on creating applications.
Prerequisites
Refer to Development System and Deployment System for requirements and instructions to setup development and deployment systems.
Obtain API keys for the following services, as microservices used in this reference application depends on them:
Weather (weatherstack) - https://weatherstack.com/product
Places Of Interest (Yelp) - https://www.yelp.com/fusion
Map (Here) - https://developer.here.com/pricing
Create New Graph
We can now start creating the application graph.
Click on Create Graph at the center of the application.
This will display pop-up dialog for Graph Name. Enter a name like
ucf-text-bot
and click on Ok.
Click on Ok to open a blank canvas on which microservices can be added.
Search for Microservices and their Details
You may browse the available microservices using the list on the left
Microservices categorization can be changed for easier browsing
Microservice can be filtered and searched using text
Select a microservice from the list to show its details on the right panel
Add Microservices
1. Microservices can be added to the application by selecting them from the list on the left and drag-dropping them on to the canvas.
For this reference application, the following microservices will be required:
Riva Speech Skills
BotMaker Dialog Manager
BotMaker NLP Server
BotMaker Fulfillment Server
BotMaker Editor
BotMaker Text Web App
After adding above microservices, the graph now looks like the following:
Connect Microservices
Microservice can be connected by connecting their endpoints. Start dragging from the egress endpoint (ports on the right side of a microservice) to the ingress endpoint (ports on the left side of a microservice).
Configure Microservices
Parameters can be set by selecting a microservice on the canvas and setting the parameters from the “Properties” window on the right.
For convenience, parameters can also be loaded from a file. Save the following content to a file (e.g. params.yaml.)
'BotMaker Dialog Manager': configNgcPath: eevaigoeixww/staging/botmaker-quickstart:23.10.2-ea botConfigName: bots/misty/misty_bot_config.yaml imagePullSecrets: - name: ngc-docker-reg-secret 'Riva Speech Skills': ucfVisibleGpus: [0] modelRepoGenerator: clearAllRMIRSAndModels: false #> description: Flag to clear artifacts and models before downloading and deploying ngcModelConfigs: tritonGroup0: models: #> description: List of NGC models for deployment - nvidia/riva/rmir_nlp_named_entity_recognition_bert_base:2.13.0 - eevaigoeixww/conversational-ai/rmir_nlp_weather_intent_slot_bert_base:2.0.0-ea persistentVolumeClaim: keepPVC: True imagePullSecrets: - name: ngc-docker-reg-secret 'BotMaker NLP Server': ucfVisibleGpus: [0] configNgcPath: "eevaigoeixww/staging/botmaker-quickstart:23.10.2-ea" modelConfigPath: "bots/misty/model_config.yaml" imagePullSecrets: - name: ngc-docker-reg-secret 'BotMaker Fulfillment Server': configNgcPath: "eevaigoeixww/staging/botmaker-quickstart:23.10.2-ea" fmconfigPath: "bots/misty/fulfillment_config.yaml" imagePullSecrets: - name: ngc-docker-reg-secret 'BotMaker Text Web App': imagePullSecrets: - name: ngc-docker-reg-secret 'BotMaker Editor': imagePullSecrets: - name: ngc-docker-reg-secret
The API keys can be obtained from:
Weather (weatherstack) - https://weatherstack.com/product
Places Of Interest (Yelp) - https://www.yelp.com/fusion
Map (Here) - https://developer.here.com/pricing
To load the above parameters file, use Load Parameters button from the actions bar.
This will open a file browser, navigate to the parameters file and load it.
Set secrets on the microservices. First click the Graph Configuration button from the actions bar to open the graph configuration.
Next select the Secrets window, click on Add to add a new secret. Enter details as shown below:
Now select Riva Speech Skills microservice on the canvas, this will show the Secrets section in the Property window on the right. For the ngc-api-key-secret
, select ngc-api-key-secret
from the dropdown.
Do the same for the BotMaker Dialog Manager, BotMaker NLP Server, and BotMaker Fulfillment Server.
Save the Graph
Click on the Save Graph icon from the actions bar
This will open a file browser. Navigate to the directory of choice and save the graph.
Build the Reference Application
Click on the Build the current Graph icon from the actions bar.
This will open a file browser. Navigate to the directory of choice and click on Okay.
Building an application may take up to a few minutes. Once it is done, a success pop-up dialog will be shown containing the path where the deployable helm chart for the application has been generated.
Refer to the Deploy the Reference Application for instructions on deploying the helm chart.
Update the Reference Application
This section demonstrates how to create a Speech based Chatbot using UCS Studio with microservices provided by NVIDIA. These are incremental instructions on top to follow after creating the Create an Reference Application. This demonstrates how increasingly complex functionality can be added by just changing a few things in the application using UCS Studio.
Some instructions on this page are specific to the applications being created, refer to Creating an Application for more detailed information on creating applications.
Open Existing Graph
If the UCS Text Bot Application application graph is not already open, open it by clicking on the Open Graph button at the center of the application or Open Graph button on the action bar.
This will open a file browser, navigate to the UCS Text Bot Application application graph and open it. This will load the application graph in the canvas. If some of the microservices are out of view, right-click on the canvas to open the context menu and click on Focus on All to automatically pan and zoom the canvas so that all microservices are visible.
Add / Replace Microservices
For this application:
BotMaker Text Web App should be replaced by BotMaker Speech Web App
Additional BotMaker Bot Controller microservice will be required
To do this select the
BotMaker Text Web App
microservice instance on the canvas and then right-click on it to open the context menu and click on Delete Selection to delete the microservice.
Next, add the BotMaker Speech Web App and BotMaker Bot Controller microservices by searching for them in the microservice list on the left and drag-dropping them on to the canvas. This is how the graph should look now:
Connect Microservices
bot-controller
egress endpoint ofBotMaker Speech Web App
needs to be connected togrpc-api
ingress endpoint ofBotMaker Bot Controller
bot-editor
egress endpoint ofBotMaker Speech Web App
needs to be connected tohttp-api
ingress endpoint ofBotMaker Editor
dialog-manager
egress endpoint ofBotMaker Bot Controller
needs to be connected todialog-manager
ingress endpoint ofBotMaker Dialog Manager
riva
egress endpoint ofBotMaker Bot Controller
needs to be connected toriva-speech-api
ingress endpoint ofRiva Speech Skills
Do this by dragging from the egress endpoints to the ingress endpoints as shown in Connect Microservices. The graph should now look as below:
Load Parameters
Save the following content to a file (e.g. params.yaml):
'BotMaker Dialog Manager': configNgcPath: eevaigoeixww/staging/botmaker-quickstart:23.10.2-ea botConfigName: bots/misty/misty_bot_config.yaml imagePullSecrets: - name: ngc-docker-reg-secret 'Riva Speech Skills': ucfVisibleGpus: [0] modelRepoGenerator: clearAllRMIRSAndModels: false #> description: Flag to clear artifacts and models before downloading and deploying ngcModelConfigs: tritonGroup0: models: #> description: List of NGC models for deployment - nvidia/riva/rmir_nlp_named_entity_recognition_bert_base:2.13.0 - eevaigoeixww/conversational-ai/rmir_nlp_weather_intent_slot_bert_base:2.0.0-ea - eevaigoeixww/staging/asr_conformer_en_us_streaming_throughput_flashlight_vad:2.13.0-tokkio #english - nvidia/riva/rmir_tts_fastpitch_hifigan_en_us:2.13.0 persistentVolumeClaim: keepPVC: True imagePullSecrets: - name: ngc-docker-reg-secret 'BotMaker NLP Server': ucfVisibleGpus: [0] configNgcPath: "eevaigoeixww/staging/botmaker-quickstart:23.10.2-ea" modelConfigPath: "bots/misty/model_config.yaml" imagePullSecrets: - name: ngc-docker-reg-secret 'BotMaker Fulfillment Server': configNgcPath: "eevaigoeixww/staging/botmaker-quickstart:23.10.2-ea" fmconfigPath: "bots/misty/fulfillment_config.yaml" imagePullSecrets: - name: ngc-docker-reg-secret 'BotMaker Speech Web App': imagePullSecrets: - name: ngc-docker-reg-secret 'BotMaker Editor': imagePullSecrets: - name: ngc-docker-reg-secret 'BotMaker Bot Controller': pipeline: speech_lite configNgcPath: "eevaigoeixww/staging/botmaker-quickstart:23.10.2-ea" speechConfigPath: "bots/misty/speech_config.yaml" imagePullSecrets: - name: ngc-docker-reg-secret
Next, follow acquire API keys and load parameters as shown in UCS Text Bot Application section.
Set secrets on BotMaker Bot Controller microservice as shown in UCS Text Bot Application - Set Secrets.
Update Application Information
click the Graph Configuration button from the actions bar to open the graph configuration.
This will open the Graph Configuration section at the bottom.
Change the application name to
ucf-speech-bot
and optionally update the description.
Save and Build the Application
From the the File menu at the top, select Save Graph As to the save the graph to a new file.
This will open a file browser, navigate to a directory of choice and save the graph.
Follow instructions in Build the Reference Application in the UCS Text Bot Application application section to build the application.
Deploy the Reference Application
Deploy on local MicroK8S cluster
Create Kubernetes secret required to pull the images:
$ microk8s kubectl create secret docker-registry ngc-docker-reg-secret --docker-server=nvcr.io --docker-username='$oauthtoken' --docker-password=<NGC_API_KEY>If another name is used for the secret instead of
ngc-docker-reg-secret
, same should be updated in the application and the app should be rebuilt.
Create Kubernetes secret for the NGC API Key:
$ microk8s kubectl create secret generic ngc-api-key-secret --from-literal=NGC_API_KEY=<NGC_API_KEY>If another secret name is used instead of
ngc-api-key-secret
or another key is used instead ofNGC_API_KEY
, same should be updated in the application and the app should be rebuilt.
Install the Local Path Provisioner if not already installed:
$ curl https://raw.githubusercontent.com/rancher/local-path-provisioner/v0.0.23/deploy/local-path-storage.yaml | sed 's/^ name: local-path$/ name: mdx-local-path/g' | microk8s kubectl apply -f -
Deploy the application helm chart:
$ microk8s helm3 install <release-name> <app-output-helm-chart>
Deploy application on AWS
Copy helm chart to AWS Instance:
scp -r -i /path/to/<your_key>.pem /path/to/<app-output-helm-chart>/ ubuntu@<aws_instance_ip>:
Install the Local Path Provisioner (on AWS Instance):
$ curl https://raw.githubusercontent.com/rancher/local-path-provisioner/v0.0.23/deploy/local-path-storage.yaml | sed 's/^ name: local-path$/ name: mdx-local-path/g' | kubectl apply -f -
Create Kubernetes secret required to pull the images (on AWS Instance):
$ kubectl create secret docker-registry ngc-docker-reg-secret --docker-server=nvcr.io --docker-username='$oauthtoken' --docker-password=<NGC_API_KEY>If another name is used for the secret instead of
ngc-docker-reg-secret
, same should be updated in the application and the app should be rebuilt.
Create Kubernetes secret for the NGC API Key (on AWS Instance):
$ kubectl create secret generic ngc-api-key-secret --from-literal=NGC_API_KEY=<NGC_API_KEY>
If another secret name is used instead of
ngc-api-key-secret
or another key is used instead ofNGC_API_KEY
, same should be updated in the application and the app should be rebuilt.Deploy the application helm chart (on AWS Instance):
$ helm install <release-name> <app-output-helm-chart>
Interact with the Reference Application
UCS Text Bot Application
Watch Pods:
Wait for all the application microservice pods to go into the running state
MicroK8s:
$ microk8s kubectl get pod NAME READY STATUS RESTARTS AGE botmaker-dialog-manager-deployment-6d9d6fd949-88f7f 1/1 Running 0 28m botmaker-text-web-app-deployment-8499c4b857-xjwcg 1/1 Running 0 28m riva-speech-skills-deployment-74cc4c9457-mfm98 1/1 Running 0 28mAWS:
$ kubectl get pods NAME READY STATUS RESTARTS AGE botmaker-dialog-manager-deployment-ffdd98965-49rbk 1/1 Running 0 48m botmaker-text-web-app-deployment-ddccf454-wsxh6 1/1 Running 0 48m riva-speech-skills-deployment-7856cc499f-9bnkf 1/1 Running 0 48m
Service List
Get the list of services associated with the application
MicroK8s:
$ microk8s kubectl get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE botmaker-dialog-manager-deployment-svc ClusterIP 10.152.183.47 <none> 5000/TCP 32m botmaker-text-web-app-deployment-service NodePort 10.152.183.132 <none> 5001:31842/TCP 32m kubernetes ClusterIP 10.152.183.1 <none> 443/TCP 67d riva-speech-skills-deployment-svc ClusterIP 10.152.183.21 <none> 50051/TCP,8001/TCP 32m 29mAWS:
$ kubectl get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE botmaker-dialog-manager-deployment-svc ClusterIP 10.102.53.150 <none> 5000/TCP 49m botmaker-text-web-app-deployment-service NodePort 10.96.23.19 <none> 5001:31666/TCP 49m kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 151m riva-speech-skills-deployment-svc ClusterIP 10.101.176.114 <none> 50051/TCP,8001/TCP 49m
Accessing the WebApp
The service of interest is
botmaker-text-web-app-deployment-service
. We will need the Node Port associated with the service. This port number will change for every deployment.
- MicroK8s:
NodePort:
31842
Open
http://<MicroK8s-Host-IP>:<Botmaker-Text-Web-App-NodePort>/bot
in a browser to start interacting with the bot.- AWS:
NodePort:
31666
We will have to open this port to the internet so that we can access it.
Go to your instance’s Security tab and navigate to the security group. Edit the Inbound Rule to accept connections to the NodePort (in this case
31666
) from Anywhere-IPv4 (0.0.0.0/0).After saving open
https://<aws_instance_ip>:<NodePort>/bot
in a browser to start interacting with the bot.
Application WebUI
On opening the URL, you should be greeted with the following UI:
You can enter the question/statement in the text box and click on Submit.
The bot answers to questions/statements like “Hello”, “How are you?”, “How is the weather in New York?”
UCS Speech Bot
Follow the steps in UCS Text Bot Application to wait till all pods are running and to get the list of services.
Accessing the WebApp
The service of interest is
botmaker-speech-webapp-deployment-service
. We will need the Node Port associated with the service. This port number will change for every deployment.
- MicroK8s:
NodePort:
31746
Open
https://<MicroK8s-Host-IP>:<Botmaker-Speech-WebApp-NodePort>/bot
in a browser to start interacting with the bot.AWS:
NodePort:
31666
We will have to open this port to the internet so that we can access it.
Go to your instance’s Security tab and navigate to the security group. Edit the Inbound Rule to accept connections to the NodePort (in this case
31666
) from Anywhere-IPv4 (0.0.0.0/0).After saving open
https://<aws_instance_ip>:<NodePort>/bot
in a browser to start interacting with the bot.
Application WebUI
On opening the URL, you should be greeted with the following UI:
After giving permissions to your microphone, you should be able to provide speech input in the WebApp.
Click on the green mic button to talk to the bot.
Click on the yellow “Unmute Speech” button to hear audio replies from the bot.
Alternatively, you can enter the question/statement in the text box and click on Submit and/or view the replies in text format
The bot answers to questions/statements like “Hello”, “How are you?”, “How is the weather in New York?”