Overview

This guide outlines the process for deploying the complete Tokkio Workflow on various target platforms. The Tokkio Workflow setup encompasses multiple components, and a successful deployment requires several configurations to work in coherence. In this section, we will walk you through the components involved in the setup, the building blocks of Deployment scripts, and the workflow of using Deployment scripts to bring up a functional Tokkio environment.

Components

Take a moment to read the diagram below. This diagram details a simplified view of the components that make up a typical Tokkio Workflow deployment.

Below are the core components required for Tokkio to function. However, based on the target environment we choose, such as AWS or BareMetal, we might need some additional components to make the end-to-end pipeline work. For example, when deploying Tokkio on AWS, we must set up an Application Load Balancer to access the Tokkio Back-end in a secured way. This Application Load Balancer is not needed, when we deploy Tokkio with BareMetal machines.

Client

A browser application on an End user’s device such as mobile or laptop where we interact with the application.

Tokkio Front-end

A Web application front-end comprising necessary components to interact with Tokkio Back-end. More details about the Tokkio Front-end are covered in Front-end section.

Tokkio Back-end

Tokkio back-end is deployed as a helm chart on Kubernetes environment. In addition to Kubernetes the Back-end machine consists of the below software stack.

Media Relay Service

This component is responsible for relaying media between Client browser and Tokkio Back-end. Especially when the client browser and Tokkio Back-end cannot connect directly, they send data to an entity such as TURN server, which then relays this data to the other device. This process ensures that communication can occur even if the direct connections are blocked for reasons such as when they are behind NAT or Firewalls. Tokkio deployment supports the options below as of today.

• COTURN service

• ReverseProxy service

• Twilio

LLM RAG / NVIDIA NIM

This component helps Tokkio Back-end get responses for User queries. While NVIDIA NIM comes configured with every default installation, you can easily replace it with your own LLM RAG.

Setting up these components and establishing proper connectivity between them could be very involved and take up some time and expertise. These deployment scripts help simplify the initial setup and let you bring up Tokkio Workflow swiftly to interact with. All the necessary components on the target machine(s) will be set up.

Interacting with Deployment scripts

A typical usage of script would be like any other bash scripts. This follows <command subcommand options> style. We will cover other other sub commands in the subsequent sections.

Example usage for install sub-command:

./envbuild.sh install --component all

Building blocks

Using deployment scripts involves dealing with below components:

• Controller instance

• Config file

• Environment variables file

Controller Instance

Referring to the machine where the Deployment scripts will be launched from. Any machine running Ubuntu OS version 22.04 or 24.04 OS and can be used as Controller Instance. At minimum this machine should have connectivity with the internet and should run bash scripts.

To ensure successful execution, these deployment scripts rely on specific software components. If not already present on the instance, the scripts will attempt to install or upgrade the following:

Required Software

1. Terraform with tfswitch

• Manages multiple Terraform versions

• Automatically switches to the appropriate version for each project

2. Python

• Core programming language used in the scripts

3. Python modules

• Installed within a virtual environment

• Isolates project dependencies from system-wide packages

4. jq

• Command-line JSON processor

• Useful for parsing and manipulating JSON data

5. yq

• YAML counterpart to jq

• Processes YAML files efficiently

The deployment scripts are designed to check for these dependencies and install them if necessary, ensuring a smooth execution environment.

Config file

This file captures all the information about the target environment such as target platform, Helm chart version to be used, UI parameters to be used while setting up Tokkio UI etc. By default script expects a config.yml file in the current directory where the script is launched from. If your config file has a different name, it can be supplied with either -f or --config-file option during install and uninstall sub commands.

While there are a large number of parameters you can tune in this config.yml we recommend you to start with a copy of example config templates we provide along with scripts. Look for the config-template-examples folder for each target environment supported. For example, azure/config-template-examples.

Environment variables file

This file helps abstract out reused parameters and secret like values that are supplied at the time of installation. For this reason, prior to running the envbuild.sh command, you will need to ensure these environment variables are supplied to the current bash shell. This can be done by using source <my-env-file.env> command.

The primary reason for separating these values from the Config file is to allow Version controlling the config file and maintain the secret like values in a different method.

Workflow of Deployment scripts

A typical workflow involves the following phases, as shown in the diagram below:

• Prepare

• Setup

• Maintain

• Teardown

Prepare

Preparation includes identifying a machine to be used for the Controller instance. Based on the target platform of your choice (AWS, BareMetal, etc.,), take a copy sample template and environment file from provided examples and modify them to your needs.

For example if you chose to work with a BareMetal sample, you might end up with below files.

• dev1-baremetal-configs.yml

• dev1-baremetal-env-vars.env

Note

File names are completely arbitrary. But for simplicity, we will use these sample file names while showing examples here in the following sections.

Setup

In this phase you will load the secret values into the bash environment and launch the deployment script with the config file created above.

1. Loading environment variables:

$ source ./dev1-baremetal-env-vars.env

2. Launching deployment script:

$ ./envbuild.sh install --component all -f dev1-baremetal-configs.yml

By the end of this script, you will be presented with a summary of results and key URLs that you will need in future. Here is a sample output:

<..logs trimmed for brevity>
<..logs trimmed for brevity>
<..logs trimmed for brevity>
access_urls:
 app:
   app: http://10.62.172.41:80/
   grafana: http://10.62.172.41:32300/login
   kibana: http://10.62.172.41:31565/app/kibana
   prometheus: http://10.62.172.41:30090/graph
 turn: {}
ssh_command:
 app:
   master: ssh -i /home/metroadmin/.ssh/id_rsa -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null horde@10.62.172.41
 turn:
   master: ssh -i /home/metroadmin/.ssh/id_rsa -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null horde@10.62.172.41

Here is a quick run down of the output and what it means. This will be similar to other target platforms as well.

Output description

access_urls -> app -> app	Tokkio UI can be accessed here.
access_urls -> app -> grafana	Grafana dashboard installed as part of platform services. Useful to watch metrics of GPU usage, etc.,
access_urls -> app -> kibana	Kibana Dashboard installed as part of platform services. Useful to access app component logs from Kubernetes.
access_urls -> app -> prometheus	Prometheus service installed as part of platform services.
ssh_command -> app -> master	Pre-populated SSH command that can be used to access your Application Instance
ssh_command -> turn -> master	Pre-populated SSH command that can be used to access your TURN server Instance. In case of BareMetal setup, TURN server gets installed on the same Application instance so it will be the same.

Maintain

This is the phase where application customizations take place. As you customize the application you can use the Deployment script to update it into the target environment by using appropriate subcommand. Below are the steps you will do in this process

• Update config file

• Apply the changes in target environment

For instance, if your changes to configurations are for Tokkio back-end, you can run the below sub command to get these changes applied.

./envbuild.sh install --component app -f dev1-baremetal-configs.yml

Notice the --component switch now refers to app instead of all. Based on what changes of config template you wish to apply you may choose from different options for --component such as all, infra, platform, app .

Output of this command will be similar to what we saw during the Setup phase.

During the maintenance phase you may want to see the summary of results of a given deployment. In such a case, you can use the info sub command by supplying the Environment variables and the correct config file as shown below.

1. Load the environment variables

$ source ./dev1-baremetal-env-vars.env

2. Launch info sub command

./envbuild.sh info -f dev1-baremetal-configs.yml

This command’s output will be in the same format we discussed during the Setup phase. For example:

access_urls:
 app:
   app: http://10.62.172.41:80/
   grafana: http://10.62.172.41:32300/login
   kibana: http://10.62.172.41:31565/app/kibana
   prometheus: http://10.62.172.41:30090/graph
 turn: {}
ssh_command:
 app:
   master: ssh -i /home/metroadmin/.ssh/id_rsa -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null horde@10.62.172.41
 turn:
   master: ssh -i /home/metroadmin/.ssh/id_rsa -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null horde@10.62.172.41

Teardown

When need arises to uninstall some component from this setup or the entire setup itself, you can use the uninstall subcommand with appropriate options for --component.

For instance if you chose to uninstall only the application component, you may run the below command:

./envbuild.sh uninstall --component app -f dev1-baremetal-configs.yml

Additionally if you choose to uninstall the entire setup, you can run the command with the below combination. Notice the --component option’s value is all and be cautious when using this option since this will basically reverse all the steps you did since start.

./envbuild.sh uninstall --component all -f dev1-baremetal-configs.yml

For more details you can run the command with --help option shown below.

$ ./envbuild.sh --help
Usage: ./envbuild.sh (-v|--version)
  or: ./envbuild.sh (-h|--help)
  or: ./envbuild.sh (install/uninstall) (-c|--component <component>) [options]
  or: ./envbuild.sh (info) [options]

install/uninstall components:
-c, --component        one or more of all/infra/platform/app, pass arg multiple times for more than one

install/uninstall options:
-f, --config-file      path to file containing config overrides, defaults to config.yml
-i, --skip-infra       skip install/uninstall of infra component
-p, --skip-platform    skip install/uninstall of platform component
-a, --skip-app         skip install/uninstall of app component
-d, --dry-run          don't make any changes, instead, try to predict some of the changes that may occur
-h, --help             provide usage information

info options:
-f, --config-file      path to file containing config overrides, defaults to config.yml
-h, --help             provide usage information

Deployment Options

At present we have deployment scripts for below environments:

• BareMetal

• Amazon Web Services (AWS)

• Microsoft Azure

• Google Cloud Platform (GCP)

Note

While these scripts help as reference for initial setup such as for development environments, we recommend that you review these thoroughly before bringing up Production like environments for scaling and security.