Sample Clients#

Web Sample Application#

The web interface allows you to interact with the bot in a simple web UI from a web browser. This interface is extremely useful when the bot has reached a certain stage of maturity and is ready to be shared with a wider audience for feedback using a text or speech-based interface.

When you launch the web app, it connects to the ACE Agent server, and exposes a simple web UI at http://<developer-ip-address>:7006. You can share that URL with a wider group for testing and feedback.

Source code of the WebUI is released as part of the quickstart directory at webui directly. Follow instructions in the README.md if you want to customize the WebUI application.

Text-Based Web Sample Application#

This mode allows you to interact with the Chat Engine directly, using text inputs and text outputs.

Launch the bot in the HTTP Interface or the Event Interface by following instructions from the Docker Environment section.

The UI of the web app in text mode can be accessed at http://<developer-ip-address>:7006 and looks something like this:

Text-Based Web Sample Application

Speech-Based Web Sample Application#

ACE Agent uses the Chat Controller to process more complex I/O pipelines such as speech and vision. To interact with a speech-based bot, we can use the speech mode of the web app, which interacts with the Chat Controller.

Launch the bot in the gRPC interface by following instructions from the Docker Environment (Server Interface or Event Interface).

The UI of the web app in speech mode can be accessed at http://<developer-ip-address>:7006 and looks something like this:

Speech-Based Web Sample Application

Note

The UI can take up to 5 minutes to load the first time it is run in speech mode. If you see the message connecting to server…, wait for a few minutes and try again.

To enable audio capture, you will need to update your browser flags to allow insecure origins or convert the endpoints to HTTPS using SSL Certificates.

Adding Browser Flags

  1. Open chrome://flags (or e``dge://flags`` if you are using Microsoft Edge).

  2. Search for the flag #unsafely-treat-insecure-origin-as-secure, enable it, and add http://<developer-ip-address>:7006 to the list.

  3. Click Relaunch to restart the browser with your changes.

  4. Access http://<developer-ip-address>:7006 to interact with the bot.

Using HTTPS Support

  1. Generate an SSL certificate for your domain.

  2. Volume mount the cert.key and cert.pem files by uncommenting in the deploy/docker/docker-compose.yml file for bot-web-ui-client and bot-web-ui-server services.

  3. Redeploy the services with the updated docker-compose.yml file.

gRPC Server Sample Client#

The gRPC Server Sample Client showcases the minimal code needed to send an audio input to Chat Controller via gRPC API and receive an audio response.

  1. Ensure that a bot is deployed with the gRPC interface by following instructions from the Docker Environment (Server Interface or Event Interface).

  2. Create a WAV file with your speech query inside clients/speech_client/. Ensure that it is single channel (mono) audio.

  3. Compile the .proto files into the pb2 files needed for the Sample Client. From clients/speech_client/, run:

    cd clients/speech_client/
pip install grpcio-tools
bash setup.sh

    Ensure that you have grpcio-tools>=1.62.1 installed.

  4. Run the Sample Client.

    python3 speech_client.py --audio_file_path Sample.wav --tts_output_path tts_output.wav

  5. After the script completes, you should see a file called tts_output.wav with the speech response.

Chat Python Sample Client#

ACE Agent includes a sample Python app that showcases how to connect to the ACE Agent HTTP server and place chat requests using the CLI.

  1. Start any bot in server mode using the HTTP Server Interface deployment instructions.

  2. Setup the chat sample client packaged as part of the Quick Start resource at clients/chat_client/. To run the chat client, you need to install a few Python packages. It is recommended that you create a new Python environment to do so.

    pip install -r clients/chat_client/requirements.txt

  3. Run the chat sample client to interact with the deployed bot.

    python3 clients/chat_client/chat_client.py

Event Sample Client#

ACE Agent includes a sample Python app called Event Simulator that showcases how to connect to the ACE Agent Event Interface and handle different types of events asynchronously.

  1. Start any Colang 2.0 bot in event mode using the Event Interface deployment instructions.

  2. Setup the Event Simulator packaged as part of Quick Start resource at clients/event_client/. To run the Event Simulator script, you need to install a few Python packages. It is recommended that you create a new Python environment to do so.

    cd clients/event_client
sudo apt-get install python3-venv
python3 -m venv sim-env
source sim-env/bin/activate
pip install -r requirements.txt

  3. Run the Event Simulator.

    python event_client.py
Event Sample Client

  1. Interact with the bot using the Prompt field in the Sample Client (hit ENTER after you are done with your text), you can see the bot responses in the Chat section on the left and the list of events that the event interfaces processed and produced on the right in the Interaction History section.

The Event Simulator Client provides a simple way in which you can interact and observe an interaction with a bot running behind the event interface. This is a very simple client intended for local testing and to get you started with the event interface.The tool simulates an interactive system compatible with the event interface. It simulates a system in which a user interacts with an avatar character that can talk, make gestures and show information on a UI screen. You can find a brief overview of the main functionality below.

Send and receive messages - You can use the interface to send text messages to the bot and receive multimodal replies. The bot can talk back to you, use gestures, change its posture and show UI overlays. This is all shown on the left-hand side of the terminal window in the corresponding sections (for example, verbal bot responses are shown in the Chat section, whereas gestures and postures are shown in the Motion section.

Note

The multimodal responses of the bot are visualized using text for this sample client. If you connect your bot to the corresponding ACE component (for example, ACE Animation Pipeline to render Avatar Characters in real time) the same actions will lead to actual character animations).

View interaction history - On the right side of the terminal you can see the event history, which shows you all events that have been processed by the event interface as part of the interaction between you (the user) and the bot. You can toggle timestamp information on and off by pressing CTRL-T inside the terminal. The listed events typically are part of actions which means you will see corresponding actions that share the same action ID. An example is the BotUtteranceAction that consists of events like StartUtteranceBotAction, UtteranceBotActionStarted, and UtteranceBotActionFinished. These events mark the lifetime of the action. You can react to any of these events in your Colang scripts (for example using a match … statement).

Interact with a simulated UI - The sample script supports simple interactive UI elements that are shown as an overlay on top of the interaction history. These UI elements are controlled directly from the Colang script (Visual Scene actions). UI elements may include static content (text, lists) as well as interactive components (option groups, buttons). If you follow our tutorial on Building a Bot using Colang 2.0 and Event Interface, you can make use of these actions. Should the UI be shown and you want to go back to the event history you can toggle the UI by pressing CTRL-B inside the terminal.

On this page