Setup Guide#

Tokkio Workflow on a Bare-metal instance can be brought up in different ways. This document covers setting up Tokkio Pipeline on a single Bare-metal machine with necessary GPU hardware using Tokkio deployment scripts. Below diagram represents the layout of components for Tokkio installation on a Bare-metal server.

Bare-metal Installation

Prerequisites#

System#

Controller Instance#

  • Ubuntu 22.04 Operating system.

  • SSH key pair

  • Password less sudo access.

Application Instance#

  • Ubuntu 22.04 Operating system.

  • Passwordless authentication from the controller instance.

  • Passwordless sudo access.

  • Kubernetes is not already installed.

  • Storage minimum 300GB space on /opt folder and 400GB on /var/lib/containerd

  • Necessary GPU devices as per Reference workflows

Note

Instructions to set up some of these prerequisites, such as setting up an SSH key pair and passwordless sudo access, can be found at Common Setup Procedures.

Access#

Access to all the artifacts used during the bringing up of the Tokkio Pipeline Application. For example, the Tokkio Application Helm chart on NGC. For instructions on how to generate a personal API key, refer to Generating a Personal API Key.

Installation Steps#

Note

Before proceeding with the installation steps, ensure you have reviewed the Deployment section to understand the deployment workflow and building blocks, including the Controller Instance, Config File, Environment Variables File, and Application Instance.

  1. Clone the NVIDIA/ACE.git repository and navigate to the Bare-metal deployment scripts.

git clone --single-branch --branch 5.0.0-beta https://github.com/NVIDIA/ACE.git
cd ACE/workflows/tokkio/5.0.0-beta/scripts/one-click/baremetal

  1. Prepare a config file, either by copying the base config-template.yml or by copying one of the example config files available under config-template-examples folder.

#list of example templates
config-template-examples/
└── tokkio-1stream
    ├── config-template.yml
    └── my-config.env

Copy a config template of your choice as the base config template for this installation.

cp config-template-examples/tokkio-1stream/config-template.yml ./my-config.yml

  1. Modify the my-config.yml with your specific settings. For more details about each parameter, refer to Advanced Configuration

vi my-config.yml

  1. Set up environment variables.

  • Create a file my-env-file.env or copy it from the config-template-examples folder.

  • Modify the environment variables file with your specific settings. For more details about each environment variable, refer to Environment Variable Definitions.

  • Source the environment variables file

cp config-template-examples/tokkio-1stream/my-config.env my-env-file.env
vi my-env-file.env

  1. Source the environment variables file

source my-env-file.env

  1. Run the installation command

./envbuild.sh install --tf-binary terraform --component all --config-file ./my-config.yml

  1. Capture installation results at the end of logs. Output will look like the sample below.

access_urls:
  app:
     ace_configurator_endpoint: http://<application-instance-ip>:30180/
     api_endpoint: https://<application-instance-ip>:30888/
     ui_endpoint: https://<application-instance-ip>:30111/
     grafana_endpoint: http://<application-instance-ip>:32300/
  turn: {}
ssh_command:
  app:
    master: ssh -i /home/metroadmin/.ssh/id_rsa.pem -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ProxyCommand="ssh -i /home/metroadmin/.ssh/id_rsa.pem -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -W %h:%p ubuntu@<bastion-host-ip>" ubuntu@<application-instance-ip>
turn:
    master: ssh -i /home/metroadmin/.ssh/id_rsa.pem -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null ubuntu@<turn-server-ip>

  1. Verify installation

  • Login to the application instance using the SSH command from the installation results ssh_command.app.master.

  • Check pod status: kubectl get pods -n <application-namespace>

  • Wait for all pods to reach Ready status (may take up to 60 minutes)

  1. Access the UI

  • Obtain the UI endpoint from the deployment script’s output. It should be at access_urls -> app -> ui_endpoint

  • For Bare-metal deployment, you will need to bypass browser warnings related to self-signed certificates.

  • Access Tokkio pipeline UI at the UI endpoint

  • Accept browser permissions for mic and speaker

  • Avatar should show up based on the UI settings configured

Bare-metal Installation

Note

Follow equivalent steps if you are using any other browser.

Uninstallation Steps#

Uninstalling Just the Application#

  • Source the correct environment variables file

source my-env-file.env

  • Run the commands below to uninstall just the application components.

./envbuild.sh uninstall --tf-binary terraform --component app --config-file ./my-config.yml

Uninstalling the Whole Setup#

  • Source the correct environment variables file

source my-env-file.env

  • Run the below uninstall command.

./envbuild.sh uninstall --tf-binary terraform --component all --config-file ./my-config.yml

Caution

This step uninstalls the entire Kubernetes cluster, so use this with caution.

Additional Considerations#

Security#

The security of Tokkio in production environments is the responsibility of the end users deploying it. When deploying in a production environment, have the security experts review any potential risks and threats; define the trust boundaries, secure the communication channels, integrate AuthN & AuthZ with appropriate access controls, keep the deployment including the containers up to date, and ensure the containers are secure and free of vulnerabilities.

Essential Skills and Background#

While the instructions here can help you get the Tokkio workflow up and running, there are certain skills and background knowledge that would make your deployment successful and that can also be debugged easily in case any issues arise. We mentioned some of these below.

Command-Line-Interface (CLI)#

  • Basic Commands: Users should be comfortable with basic command-line operations, such as navigating directories, executing scripts, and managing files.

  • Environment Configuration: Understanding how the environment variables and how the PATH setup work on Linux will greatly help in operating the OneClick script.

  • Scripting Basics: Basic scripting knowledge (e.g., shell scripting) is beneficial for understanding how the OneClick script operates and for troubleshooting any issues that may arise.

Yaml#

YAML Syntax and Structure: YAML is often used for configuration files in cloud-native applications due to its readability and flexibility. The configuration templates used in OneClick script use YAML format. Users should be familiar with YAML syntax and structure.

Kubernetes Eco System#

Tokkio UI Application is a cloud-native application and uses concepts like containerization, Kubernetes, Helm, etc. You’ll need to be familiar with these to get the best results from using deployment scripts and the app.

  • Kubernetes Basics: Basic understanding of Kubernetes core concepts such as pods, services, and deployments.

  • kubectl: Familiarity with the command line tool used to interact with Kubernetes clusters, including querying the status or logs of running application pods, etc.

  • Helm: Understanding the Helm package manager for Kubernetes that simplifies application deployment by managing charts (collections of pre-configured Kubernetes resource definitions). How to use Helm with override values will help configure the templates appropriately.

On this page