Microservices#
See below for an illustration of how microservices are interconnected, and what data flows through each connection.
See below for a complete list of microservices used in a Tokkio system
Microservice |
Description |
Category |
Stateful/Stateless |
|---|---|---|---|
VST |
Audio-video stream provider |
Media Service |
Stateless |
Avatar Renderer |
The Avatar Renderer is used for rendering bot animation |
Graphics and Rendering |
Stateful |
UE Renderer |
Unreal-based renderer as an alternative to the default Avatar Renderer |
Graphics and Rendering |
Stateful |
LivePortrait |
2D renderer as an alternative to the default Avatar Renderer |
Graphics and Rendering |
Stateful |
Audio2Face |
Accepts streaming audio, and injects facial expression into bot animation |
Graphics and Rendering |
Stateless |
Animation Graph |
Animation graph compiles and buffers the animation data provided by all the animation sources and send the current chunk of audio and the current avatar pose/frame to the renderer |
Graphics and Rendering |
Stateful |
UMIM |
UMIM service accepts events from vision, audio, and UI pipeline and instructs the animation pipeline and UI server to perform certain actions |
Graphics and Rendering |
Stateless |
Chat Engine |
Accepts textual query, events, multimodal metadata and generates detailed bot response which can be used to drive TTS, avatar, UI visualization. |
Speech AI |
Stateless |
Chat Controller |
Accepts realtime audio, passes the audio through bot-pipeline and returns synthesized speech to downstream |
Speech AI |
Stateful |
Riva Speech Skills |
Imparts NLP (Natural Language Processing), TTS (Text to Speech) and ASR (Automatic Speech Recognition) related features to Tokkio pipeline |
Speech AI |
Stateless |
SDR |
Provides a way to distribute media streams to the individual pods and responsible for the routing and stream state management. A separate instance of SDR is created for each stateful microservice |
Foundationa |
Stateless |
Tokkio Cart Manager |
Part of a reference app for Retail. Used for user cart management |
App-Specific |
Stateless |
Metropolis Occupancy and Alerts |
Detection of a user within the Field Of View (FOV) |
AI Analytics |
Stateless |
Metropolis Occupancy Alerts API |
WebAPI for Metropolis Occupancy Alerts |
Web API |
Stateless |
Tokkio Ingress Controller |
A specialized load balancer for Tokkio deployment |
App-Specific |
Stateless |
DS Vision AI |
DeepStream Vision AI component imparts Face Detection and Tracking |
Vision AI |
Stateless |
Tokkio Catalog RAG |
Part of a reference app for Retail. Used for Catalog based RAG pipeline |
App-Specific |
Stateless |
Tokkio UI Server |
Part of a reference app for Retail. Web UI Server for Tokkio |
App-Specific |
Stateless |
Plugin Server |
Part of a reference app for Retail. Handles food ordering for Retail bots by interacting with Catalog RAG, Cart Manager and UI Server |
App-Specific |
Stateless |
Internal Ingress Proxy |
Load balancing and routing gRPC calls from clients to A2F-2D Microserivices |
Foundation |
Stateless |
Signaling Proxy |
Initiate UDP ports negotiation with VST from RTP/UDP streaming |
Foundation |
Stateless |
MongoDB |
Data storage used to persist information from various components |
Foundation |
Stateless |
Redis Timeseries |
Message bus used to facilitate inter-service communication, and temporarily store timeseries data |
Foundation |
Stateless |
Redis |
Used as a cache to store session stats and other routing related information. |
Foundation |
Stateless |