Microservices

See below for an illustration of how microservices are interconnected, and what data flows through each connection.

See below for a complete list of microservices used in a Tokkio workflow.

Microservices

Microservice	Description	Category	Stateful/Stateless
VST	Audio-video stream provider	Streaming Pipeline	Stateless
Tokkio Ingress Controller	A specialized load balancer for Tokkio deployment	Streaming Pipeline	Stateless
Internal Ingress Proxy	Load balancing and routing gRPC calls from clients to A2F-2D Microservices	Streaming Pipeline	Stateless
DS Vision AI	DeepStream Vision AI component imparts Face Detection and Tracking	Vision Pipeline	Stateless
Metropolis Occupancy and Alerts	Detection of a user within the Field Of View (FOV)	Vision Pipeline	Stateless
Metropolis Occupancy Alerts API	WebAPI for Metropolis Occupancy Alerts	Vision Pipeline	Stateless
Chat Controller	Accepts realtime audio, passes the audio through bot-pipeline and returns synthesized speech to downstream	Speech Pipeline	Stateful
Riva Speech Skills	Imparts NLP (Natural Language Processing), TTS (Text to Speech) and ASR (Automatic Speech Recognition) related features to Tokkio pipeline	Speech Pipeline	Stateless
Action Server	The action server manages the flow of the UMIM events in Tokkio.	Interaction Pipeline	Stateless
Chat Engine	Accepts textual query, events, multimodal metadata and generates detailed bot response which can be used to drive TTS, avatar, UI visualization.	Interaction Pipeline	Stateless
Tokkio UI Server	Part of a reference app for Retail. Web UI Server for Tokkio	Fulfillment Pipeline	Stateless
Plugin Server	Part of a reference app for Retail. Handles food ordering for Retail bots by interacting with Catalog RAG, Cart Manager and UI Server	Fulfillment Pipeline	Stateless
Audio2Face-3D	Accepts streaming audio, and injects facial expression into bot animation	Amimation Pipeline	Stateless
Animation Graph	Animation graph compiles and buffers the animation data provided by all the animation sources and send the current chunk of audio and the current avatar pose/frame to the renderer	Amimation Pipeline	Stateful
Avatar Renderer	The Avatar Renderer is used for rendering bot animation	Amimation Pipeline	Stateful
UE Renderer	Unreal-based renderer as an alternative to the default Avatar Renderer	Amimation Pipeline	Stateful
LivePortrait	2D renderer as an alternative to the default Avatar Renderer	Amimation Pipeline	Stateful
Signaling Proxy	Initiate UDP ports negotiation with VST from RTP/UDP streaming	Amimation Pipeline	Stateless
SDR	Provides a way to distribute media streams to the individual pods and responsible for the routing and stream state management. A separate instance of SDR is created for each stateful microservice	Foundationa	Stateless
MongoDB	Data storage used to persist information from various components	Foundation	Stateless
Redis Timeseries	Message bus used to facilitate inter-service communication, and temporarily store timeseries data	Foundation	Stateless
Redis	Used as a cache to store session stats and other routing related information.	Foundation	Stateless
Tokkio Catalog RAG	Part of a reference app for Retail. Used for Catalog based RAG pipeline	For the QSR reference workflow only	Stateless
Tokkio Cart Manager	Part of a reference app for Retail. Used for user cart management	For the QSR reference workflow only	Stateless