Overview

Tokkio is an interactive avatar virtual customer service assistant product SDK. It can be adopted for virtual assistant use case in any domain. The face of the product is avatar. The avatar can see using camera and perceive the situation using vision AI. It can interact intelligently using AI. The SDK includes a reference app, an end-to-end pipeline that integrates the following microservices:

Microservices

Microservice	Functionality imparted in Tokkio	Functional Category
VMS	The Video Management Service (VMS) is an audio-video stream provider	Media Service
Avatar Renderer	The Avatar Renderer is used for rendering bot animation	Graphics and Rendering
Audio2Face	Accepts streaming audio, and injects facial expression into bot animation	Graphics and Rendering
Animation Graph	Animation graph compiles and buffers the animation data provided by all the animation sources and send the current chunk of audio and the current avatar pose/frame to the renderer	Graphics and Rendering
UMIM (Unified multimodal Interaction Management)	UMIM service accepts events from vision, audio, and UI pipeline and instructs the animation pipeline and UI server to perform certain actions	Graphics and Rendering
Chat Engine	Accepts textual query, events, multimodal metadata and generates detailed bot response which can be used to drive TTS, avatar, UI visualization.	Speech AI
Chat Controller	Accepts realtime audio, passes the audio through bot-pipeline and returns synthesized speech to downstream	Speech AI
Riva Speech Skills	Imparts NLP (Natural Language Processing), TTS (Text to Speech) and ASR (Automatic Speech Recognition) related features to Tokkio pipeline	Speech AI
SDR (Stream Distribution & Routing)	Provides a way to distribute media streams to the individual pods and responsible for the routing and stream state management. A separate instance of SDR is created for each stateful microservice	Foundational / Scaling
Tokkio Cart Manager	Part of a reference app for Quick Service Restaurant (QSR). Used for user cart management	App-Specific
Metropolis Occupancy and Alerts	Detection of a user within the Field Of View (FOV)	AI Analytics
Metropolis Occupancy Alerts API	WebAPI for Metropolis Occupancy Alerts	Web API
Tokkio Ingress Controller	A specialized load balancer for Tokkio deployment	App-Specific
DS Vision AI	DeepStream Vision AI component imparts Face Detection and Tracking	Vision AI
Tokkio Menu API	Part of a reference app for Quick Service Restaurant (QSR). Used for menu database management	App-Specific
Tokkio UI Server	Part of a reference app for Quick Service Restaurant (QSR). Web UI Server for Tokkio	App-Specific
Plugin Server	Part of a reference app for Quick Service Restaurant (QSR). Handles food ordering for QSR bots by interacting with Menu database, Cart Manager and UI Server	App-Specific
MongoDB	Data storage used to persist information from various components	Storage
Redis Timeseries	Message bus used to facilitate inter-service communication, and temporarily store timeseries data	Message Bus
Redis	Used as a cache to store session stats and other routing related information.	Cache

For deployment, the SDK includes reference helm chart to deploy in Azure, AWS and GCP. Once you are onboarded with NGC, you will be able to generate API keys, with which grants you access to all the Tokkio software components.

Prerequisites

• To deploy the Tokkio helm chart user needs an NGC account (ngc.nvidia.com)

• Obtain an NGC API key by following the set up instruction provided in ngc.nvidia.com

• NVAIE access to ACE is required. Please contact NVIDIA to request NVAIE access to ACE.