Quickstart Guide

UCS Tool

NVIDIA Unified Cloud Services (UCS) tools is a fully accelerated framework for developing cloud-native and real-time multimodal applications with optimized microservices containers, no-code design tools, and sample applications. Adopting a Microservices Architecture approach, Unified Cloud Services enables developers to combine optimized and accelerated microservices into a multimodal application capable of meeting the real-time requirements of interactive AI use cases.

Tokkio uses UCS as a framework to build and deploy the microservices. For more information on UCS, see UCS tool docs.

Building the Tokkio App With UCS Tool

Prerequisites

  1. Download and install the UCS tool.

  2. Make sure the required privileges for pulling from UCS repository are granted - details below.

  3. For deployment and testing - Chrome Browser (version >= 107.0.5304.107), Camera input for video (> 30 fps), Microphone for audio input.

  4. A setup available using a setup guide Tokkio Setup Guides - AWS CSP Automated Setup Guide, Azure CSP Automated Setup Guide, and GCP CSP Automated Setup Guide.

Steps: Sync UCS registry and Download Tokkio App

  1. Follow the instructions to setup the UCS tool.

  2. Ensure to sync the repo for UCS tools, as instructed in the documentation referenced above.

  3. Clone the latest ACE code from the Github repository:

  1. Navigate to the Tokkio reference apps source directory:

$ cd ACE/workflows/tokkio/

Steps: Using UCS Studio

  1. Launch UCS Studio

  2. Click on Open Graph and select tokkio-app.yaml. To open LLM app’s graph, open tokkio-llm-app.yaml instead.

  3. Once the graph is loaded, right click on an empty space on the canvas and click Load parameters. Select tokkio-app-params.yaml (tokkio-llm-app-params.yaml for LLM app) to load the parameters for Tokkio. For more information on importing parameters, refer to UCS Studio Import Parameters

  4. Build the Tokkio app using the Build the current app button on UCS Studio. For more information refer to UCS Studio Building the App

  5. Once the helm chart is built, it can be published to NGC using:

    $ helm package ucs-tokkio-audio-video-llm-app-VERSION
$ ngc registry chart push --org ORG_NAME --team TEAM_NAME ORG_NAME/TEAM_NAME/CHART_NAME:VERSION

  6. Once the application is built successfully, deploy it on cloud using the infrastructure set up with the help of the CSP setup guide (AWS CSP Automated Setup Guide, Azure CSP Automated Setup Guide, and GCP CSP Automated Setup Guide).

Steps: Using UCS CLI

  1. Open the tokkio-app.yaml (tokkio-llm-app.yaml for LLM) or tokkio-app-params.yaml (tokkio-llm-app-params.yaml for LLM) to edit and make all necessary changes.

  2. Build the app with UCS app builder to create a helm chart:

    $ ucf_app_builder_cli app build <app-spec-yaml> <param-yaml>
# <param-yaml> is optional

  3. Upon successful build, the confirmation message below will be shown:

    Syncing any missing service versions to cache...
Building application ucs-tokkio-audio-video-app-<v> ...
Application Helm Chart generated in ucs-tokkio-audio-video-app-<v>
App built

  4. Once the helm chart is built, it can be published to NGC using:

    $ helm package ucs-tokkio-audio-video-llm-app-VERSION
$ ngc registry chart push --org ORG_NAME --team TEAM_NAME ORG_NAME/TEAM_NAME/CHART_NAME:VERSION

  5. Deploy the application on cloud using the infrastructure set up with the help of the CSP setup guide (AWS CSP Automated Setup Guide, Azure CSP Automated Setup Guide, and GCP CSP Automated Setup Guide).

User Guide

Session Management

Tokkio application now supports up to 3 concurrent sessions in a standardized deployment on one of the 3 supported CSPs. (AWS, Azure, GCP). If the capacity is reached, additional client will be presented with error page “All bots busy helping others, please try again later”. You will need to wait for one of the web client disconnects before you can access.

Tokkio uses the number of active WebRTC connections to track usage. This means that a single user can with multiple browser tabs open, with each accessing the same Tokkio application will still be considered as separate Tokkio sessions, as they function individually without any knowledge about the other sessions. Refreshing the page will reset the transaction. Always disconnect the web client after using, otherwise this will block others from accessing the instance.

Audio & Video

Accept the prompt at the browser to grant permission on camera and mic access on launching the web application. Make sure the room is well-lit and your face is not obstructed (It might take a while for the robot to start interacting). When your network condition is unstable or the lighting condition is compromised, the mic icon will turn yellow, which indicates a sub-optimal operation environment. This might cause lags in both video and audio. Leaving the sight of your camera will cause the application to terminate the transaction. The bot will not start a new transaction unless the previous person has left the scene and a second person or the same person has entered the scene.

Bot Interaction

For using the reference application of the Quick Service Restaurant (QSR), ensure that audio and video inputs are available by following the step above. Speak clearly into the microphone while being present in front of the webcam for the video feed in order to interact with the bot. You can order items from the menu, navigate the menu, make small talk with the bot, etc., to try out the reference application.

Customization

Tokkio - Customization offers various ways Tokkio can be leveraged by the users beyond the reference application.

Follow the steps below to build a customized version of Tokkio application

  1. Ensure that the pre-requisites mentioned in section Prerequisites are met.

  2. See the steps to customize a Tokkio microservice Customizing a microservice within Tokkio

  3. Reference the newly updated version of the microservice (if applicable) in tokkio-app.yaml (tokkio-llm-app.yaml for LLM) and update the tokkio-app-params.yaml (tokkio-llm-app-params.yaml for LLM) file if necessary.

  4. Build the app with UCS app builder to create a helm chart:

    $ ucf_app_builder_cli app build <app-spec-yaml> <param-yaml>
# <param-yaml> is optional

  5. Refer to the relevant CSP guide (AWS CSP Automated Setup Guide, Azure CSP Automated Setup Guide, and GCP CSP Automated Setup Guide) to deploy the newly created helm chart.