UI#
The Tokkio UI is a reference web application that demonstrates the end to end interactions between the frontend and the Tokkio deployment with both speech and touch inputs. It sends events to the UI server by sending HTTP REST API requests through Ingress, and it maintains a WebSocket connection with UI Server for receiving events. The ASR and TTS transcripts, user attention state, user presence state, current UI view, and more are enabled with this WebSocket connection.
Tokkio UI supports an audio & video connection through WebRTC provided by VST. This connection is facilitated over WebSocket through the VST streaming library. Permissions for accessing the camera and/or microphone needs to be granted from the browser before UI can be run.
The Tokkio UI is deployed separately from the rest of the Tokkio application. This allows one to have full control over the deployment, scaling, and security of the UI, and allows developers to take advantage of the many available options for deploying a web application such as AWS, GCP, and more.