Tokkio 2.0#

Tokkio 2.0 unlocks various features including scalability, faster deployment times, helm upgrade support, lower latency, improved stability and error handling compared to release 1.5. Main features listed below:

Notable updates#

Reference applications and Customization#

  • Tokkio IR + LLM reference application to showcase how the Large Language Models can be easily used with Tokkio pipeline

  • Customization guidance to use Tokkio pipeline without vision component

  • Addition of recommendations feature for the reference application of Quick Service Restaurant

Scaling#

  • One deployment/pipeline supports up to three active connections on a single 4-T4 GPU instance

BotMaker#

  • OpenAI LLM backend support

  • Bot Controller RTSP Audio input support

  • Barge in support for speech pipeline

  • Information Retrieval support to allow utilizing existing knowledge sources

  • Dedicated helm chart to support bot customization and development seamlessly

  • Non-English language support in BotMaker pipeline with OpenAI backend

  • Automatic Speech Recognition improvements

  • Tokkio Food Ordering Fulfillment scaling support

  • Upgraded to latest Riva Skills 2.11.0 models

MLOps#

  • Customizable metadata collector

  • Tokkio Package Analyzer (TPA) tool to visualize and validate accuracy of various AI subsystems within Tokkio deployments

K8s Cluster Logging#

  • Elasticsearch cluster with Kibana can be enabled to view Tokkio logs on Kibana dashboard

Stream Distribution and Routing#

  • Workload distribution agents created to manage the routing of video streams for various microservices

Easy Deployment#

  • Update to the 1-click deployment scripts to enable fast and easy Tokkio deployment on supported cloud setups

Miscellaneous changes#

  • Removal of Maxine audio and ThinClient microservices as a part of Tokkio deployment

  • The UI Server implements an addition websocket endpoint with VST

Updating to Tokkio 2.0#

  • For any new microservice created to work with Tokkio 1.5, a crucial detail to note before the migration would be the endpoints it uses for connecting to other components within the Tokkio 1.5 deployment.

  • If any parameters values were updated/customized for microservices within Tokkio 1.5 deployment, these would also need to be manually carried forward into the Tokkio 2.0 deployment.

  • You can use UCF studio to set the relevant parameters and ensure that the connection endpoints can be successfully maintained or make minor adjustments, if necessary, to enable interfacing with the updated APIs.

  • Delete the previous (Tokkio 1.5) deployment before installing Tokkio 2.0 using the applicable 1-click scripts.

On this page