Sample Clients

Web Sample Application

The web interface allows you to interact with the bot in a simple web UI from a web browser. This interface is extremely useful when the bot has reached a certain stage of maturity and is ready to be shared with a wider audience for feedback using text-based interface.

When you launch the web app, it connects to the ACE Agent server, and exposes a simple web UI at http://<developer-ip-address>:9001. You can share that URL with a wider group for testing and feedback.

Text-Based Web Sample Application

This mode allows you to interact with the Chat Engine directly, using text inputs and text outputs.

Launch the bot in the HTTP interface by following instructions from Docker Environment.

To launch the web app in text mode, run:

source deploy/docker/docker_init.sh
docker compose -f deploy/docker/docker-compose.yml up frontend

The UI of the web app in text mode can be accessed at https://<developer-ip-address>:9001 and looks something like this:

Text-Based Web Sample Application

Speech-based Web Sample Application

ACE Agent uses the Chat Controller to process more complex I/O pipelines such as speech and vision. To interact with a speech-based bot, we can use the speech mode of the web app, which interacts with the Chat Controller.

Launch the bot in the gRPC interface by following instructions from the Docker Environment.

To launch the web app in speech mode, run:

source deploy/docker/docker_init.sh
docker compose -f deploy/docker/docker-compose.yml up frontend-speech

The UI of the web app in speech mode can be accessed at http://<developer-ip-address>:9001 and looks something like this:

Speech-Based Web Sample Application

Note

To enable audio capture, you will need to update your browser flags to allow insecure origins.

  1. Open chrome://flags (or e``dge://flags`` if you are using Microsoft Edge).

  2. Search for the flag #unsafely-treat-insecure-origin-as-secure and add http://<developer-ip-address>:9001 to the list.

  3. Click Relaunch to restart the browser with your changes.

  4. Access http://<developer-ip-address>:9001 to interact with the bot.

Launching the Web Sample Application with Certificates

By default, the Web Sample Applications launches in http mode. However, you can launch the Web Sample Application with your own certificate files using the following steps:

  1. Ensure that the certificate files are inside the Quick Start Script.

  2. Add the following args to the frontend or frontend-speech command in docker-compose.yaml.

    --ssl_keyfile <path-to-keyfile> --ssl_certfile <path-to-certfile> --ssl_keyfile_password <password-used-to-generate-cert>

    Ensure that you use the relative paths to the working directory.

gRPC Server Sample Client

The gRPC Server Sample Client showcases the minimal code needed to send an audio input to Chat Controller via gRPC API and receive an audio response.

  1. Ensure that a bot is deployed with the gRPC interface by following instructions from the Docker Environment.

  2. Create a WAV file with your speech query inside clients/speech_client/. Ensure that it is single channel (mono) audio.

  3. Compile the .proto files into the pb2 files needed for the Sample Client. From clients/speech_client/, run:

    cd clients/speech_client/
pip install grpcio-tools
bash setup.sh

    Ensure that you have grpcio-tools>=1.62.1 installed.

  4. Run the Sample Client.

    python3 speech_client.py --audio_file_path Sample.wav --tts_output_path tts_output.wav

  5. After the script completes, you should see a file called tts_output.wav with the speech response.

Chat Python Sample Client

ACE Agent includes a sample Python app that showcases how to connect to the ACE Agent HTTP server and place chat requests using the CLI.

  1. Start any bot in server mode using the HTTP Server Interface deployment instructions.

  2. Setup the chat sample client packaged as part of the Quick Start resource at clients/chat_client/. To run the chat client, you need to install a few Python packages. It is recommended that you create a new Python environment to do so.

    pip install -r clients/chat_client/requirements.txt

  3. Run the chat sample client to interact with the deployed bot.

    python3 clients/chat_client/chat_client.py

Event Sample Client

ACE Agent includes a sample Python app called Event Simulator that showcases how to connect to the ACE Agent Event Interface and handle different types of events asynchronously.

  1. Start any Colang 2.0 bot in event mode using the Event Interface deployment instructions.

  2. Setup the Event Simulator packaged as part of Quick Start resource at clients/event_client/. To run the Event Simulator script, you need to install a few Python packages. It is recommended that you create a new Python environment to do so.

    cd clients/event_client
sudo apt-get install python3-venv
python3 -m venv sim-env
source sim-env/bin/activate
pip install -r requirements.txt

  3. Run the Event Simulator.

    python event_client.py
Event Sample Client

  1. Interact with the bot using the Prompt field in the Sample Client (hit ENTER after you are done with your text), you can see the bot responses in the Chat section on the left and the list of events that the event interfaces processed and produced on the right in the Interaction History section.

The Event Simulator Client provides a simple way in which you can interact and observe an interaction with a bot running behind the event interface. This is a very simple client intended for local testing and to get you started with the event interface.The tool simulates an interactive system compatible with the event interface. It simulates a system in which a user interacts with an avatar character that can talk, make gestures and show information on a UI screen. You can find a brief overview of the main functionality below.

Send and receive messages - You can use the interface to send text messages to the bot and receive multimodal replies. The bot can talk back to you, use gestures, change its posture and show UI overlays. This is all shown on the left-hand side of the terminal window in the corresponding sections (for example, verbal bot responses are shown in the Chat section, whereas gestures and postures are shown in the Motion section.

Note

The multimodal responses of the bot are visualized using text for this sample client. If you connect your bot to the corresponding ACE component (for example, ACE Animation Pipeline to render Avatar Characters in real time) the same actions will lead to actual character animations).

View interaction history - On the right side of the terminal you can see the event history, which shows you all events that have been processed by the event interface as part of the interaction between you (the user) and the bot. You can toggle timestamp information on and off by pressing CTRL-T inside the terminal. The listed events typically are part of actions which means you will see corresponding actions that share the same action ID. An example is the BotUtteranceAction that consists of events like StartUtteranceBotAction, UtteranceBotActionStarted, and UtteranceBotActionFinished. These events mark the lifetime of the action. You can react to any of these events in your Colang scripts (for example using a match … statement).

Interact with a simulated UI - The sample script supports simple interactive UI elements that are shown as an overlay on top of the interaction history. These UI elements are controlled directly from the Colang script (Visual Scene actions). UI elements may include static content (text, lists) as well as interactive components (option groups, buttons). If you follow our tutorial on Building a Bot using Colang 2.0, you can make use of these actions. Should the UI be shown and you want to go back to the event history you can toggle the UI by pressing CTRL-B inside the terminal.