Bare-metal - Manual Setup#

In this section, we will walk through the steps to deploy the Tokkio workflow on a Bare-metal machine with manual steps. We will use COTURN service to relay media between the Client and Tokkio Back-end. We will work directly on the application instance where the application will be deployed.

Prerequisites#

System Requirements#

Application Instance#

  • Ubuntu 22.04 Operating system

  • Passwordless sudo access

  • Storage requirements:

    • 300GB space on /opt

    • 400GB on /var/lib/containerd

  • Required GPU devices as per Reference workflows

Note

Instructions to set up some of these prerequisites, such as setting up an SSH key pair and passwordless sudo access, can be found at Common Setup Procedures.

Note

This is a manual setup process where we will directly log into the application instance via SSH and perform all configuration steps manually, rather than using automated deployment scripts.

Setup Instructions#

Kubernetes Cluster Bootstrapping#

We will be setting up NVIDIA Cloud Native Stack (CNS) 14.0. This brings the following components:

  • Kubernetes 1.31.2

  • NVIDIA GPU Operator 24.9.2

  • NVIDIA GPU Driver 570.86.15

  • Containerd 1.7.23

  • NVIDIA Container Toolkit 1.17.4

For detailed instructions you can refer to NVIDIA Cloud Native Stack (CNS) 14.0 Installation Guide.

Installation Steps#

  1. Clone the repository:

git clone --branch v24.11.2 --single-branch  NVIDIA/cloud-native-stack.git
cd cloud-native-stack/playbooks

Note

Notice the clone command above; we are cloning a specific tag. Change this according to your needs.

  1. Update the hosts file:

This file will contain the IP address of the application instance and the username and password to access it. Here in this case, we provide the details of the same instance we are logged in.

[master]
<application-instance-ip> ansible_ssh_user=<username> ansible_ssh_pass=<password> ansible_ssh_common_args='-o StrictHostKeyChecking=no'
[nodes]

  1. Set CNS version:

echo "14.0" > cns_version.yaml

  1. Update CNS settings:

Edit the cns_values_14.0.yaml file to set the following parameters.

cns_nvidia_driver: yes

vi cns_values_14.0.yaml

  1. Install CNS:

bash setup.sh install

Note

This step will take a while to complete. However if you see the terminal output hanging at the below steup, you may have to kill the process and restart or start a new terminal session and run the bash setup.sh install command again.

Scanning processes….
Scanning candidates.
Scanning
processor microcode….
Scanning
linux images...
Running kernel seems to be up-to-date.
The processor microcode seems to be up-to-date.
Restarting services..

TURN Server Setup#

The TURN server setup consists of installing and configuring the COTURN service. COTURN is an open-source implementation of TURN and STUN server used for NAT traversal in WebRTC applications. The setup involves:

  1. Installing the COTURN package

  2. Configuring network settings and credentials

  3. Setting up the service to run automatically

Installation#

Create the setup script:

cd $HOME
cat << 'EOF' > setup-coturn.sh
#!/bin/bash
if [ $1 == 'install' ]; then
  echo "Setup coturn server -- Start"
  DEBIAN_FRONTEND=noninteractive
  apt-get update -y
  apt-get install -y coturn

  if [ -n "$TURNSERVER_PUBLIC_IP" ] && [ -n "${TURNSERVER_PRIVATE_IP}" ]; then
    EXTERNAL_IP="${TURNSERVER_PUBLIC_IP}/${TURNSERVER_PRIVATE_IP}"
  elif [ -z "$TURNSERVER_PUBLIC_IP" ] && [ -n "${TURNSERVER_PRIVATE_IP}" ]; then
    EXTERNAL_IP="${TURNSERVER_PRIVATE_IP}"
  else
    echo "TURNSERVER_PRIVATE_IP variable ${TURNSERVER_PRIVATE_IP} is not set"
  fi

  sed \
    -e "s|^#TURNSERVER_ENABLED=1|TURNSERVER_ENABLED=1|g" \
    -i /etc/default/coturn

  sed \
    -e "/^#\?listening-port=/d" \
    -e "/^#\?listening-ip=/d" \
    -e "/^#\?external-ip=/d" \
    -e "/^#\?relay-ip=/d" \
    -e "/^#\?min-port=/d" \
    -e "/^#\?max-port=/d" \
    -e "/^#\?fingerprint/d" \
    -e "/^#\?realm=/d" \
    -e "/^#\?user=/d" \
    -e "/^#\?log-file=/d" /etc/turnserver.conf \
    -i /etc/turnserver.conf

  cat <<EOT >> /etc/turnserver.conf
listening-port=3478
listening-ip=${TURNSERVER_PRIVATE_IP}
external-ip=${EXTERNAL_IP}
relay-ip=${TURNSERVER_PRIVATE_IP}
min-port=49152
max-port=65535
fingerprint
realm=${TURNSERVER_REALM}
user=${TURNSERVER_USERNAME}:${TURNSERVER_PASSWORD}
log-file=/var/tmp/turn.log
EOT

  systemctl restart coturn

  echo "Setup coturn server -- End"

else
  systemctl stop coturn
  DEBIAN_FRONTEND=noninteractive
  apt-get purge coturn -y
  rm -rf /etc/turnserver.conf
fi
EOF

Configuration#

  1. Set required environment variables:

export TURNSERVER_REALM=<mydummy.org>  # Realm name
export TURNSERVER_USERNAME=<foo>  # Username
export TURNSERVER_PASSWORD=<bar>  # Password

Note

Carefully check if your Application instance’s IP is behind NAT or not. Set the environment variables accordingly.

  1. Determine whether your application instance is behind NAT or not:

  • Not Behind NAT

    • Only set TURNSERVER_PRIVATE_IP to the IP address used to login.

export TURNSERVER_PRIVATE_IP=<IPV4 ADDRESS>      # IP Address of the application instance used to login

  • Behind NAT

    • Set the internal IP as TURNSERVER_PRIVATE_IP.

    • Set the external (public) IP as TURNSERVER_PUBLIC_IP.

export TURNSERVER_PRIVATE_IP=<IPV4 ADDRESS>      # IP Address of the application instance after login
export TURNSERVER_PUBLIC_IP=<IPV4 ADDRESS>      # IP Address of the application instance used to login

  1. Execute the setup script:

chmod u+x ./setup-coturn.sh
sudo -E ./setup-coturn.sh install

  1. Verify the service:

sudo systemctl status coturn.service

Foundational Charts Setup#

In this section, we will be setting up the Foundational Charts. This will include the local path provisioner and the observability stack.

Local Path Provisioner Chart Setup#

  1. Create values file for local-path-provisioner:

echo 'nodePathMap:
  - node: DEFAULT_PATH_FOR_NON_LISTED_NODES
    paths:
      - /opt/local-path-provisioner
storageClass:
  name: mdx-local-path
'| envsubst > /tmp/local-path-provisioner-values.yml

  1. Add and update helm repository:

helm repo add containeroo https://containeroo.github.io/helm-charts
helm repo update

  1. Install the helm chart:

helm upgrade --install local-path-provisioner containeroo/local-path-provisioner \
  --namespace platform \
  --create-namespace \
  --version 0.0.32 \
  -f /tmp/local-path-provisioner-values.yml

Observability Chart Setup#

  1. Set NGC API key:

In order to access NGC, we need to set the NGC API key created on NGC website. For instructions on how to generate a personal API key, refer to Generating a Personal API Key.

export NGC_CLI_API_KEY=<YOUR NGC PERSONAL KEY>

  1. Add and update helm repository:

helm repo add nvidia-ace https://helm.ngc.nvidia.com/nvidia/ace --username '$oauthtoken' --password $NGC_CLI_API_KEY
helm repo update

  1. Install observability stack:

helm upgrade --install obs \
  nvidia-ace/ucf-foundational-chart-observability-stack \
  --version 0.0.5-beta \
  --namespace platform \
  --create-namespace

Application Chart Setup#

In this section, we will be setting up the Tokkio Helm Chart. This will include the creation of the application namespace and the secrets required for the application.

Kubernetes Secrets Configuration#

  1. Create application namespace:

kubectl create namespace app

  1. Set environment variables:

export NGC_CLI_API_KEY="<replace-with-ngc-cli-api-key>"
export OPENAI_API_KEY="<replace-with-openai-api-key>"
export NVIDIA_API_KEY="<replace-with-nvidia-api-key>"
export ELEVENLABS_API_KEY="<replace-with-elevenlabs-api-key>"

  1. Create required secrets:

# NGC API Key Secret
kubectl create secret generic ngc-api-key-secret \
  --from-literal=NGC_CLI_API_KEY="${NGC_CLI_API_KEY}" -n app

# OpenAI Key Secret
kubectl create secret generic openai-key-secret \
  --from-literal=OPENAI_API_KEY="${OPENAI_API_KEY}" -n app

# NVIDIA API Key Secret
kubectl create secret generic nvidia-api-key-secret \
  --from-literal=NVIDIA_API_KEY="${NVIDIA_API_KEY}" -n app

# NGC Docker Registry Secret
kubectl create secret docker-registry ngc-docker-reg-secret \
  --docker-server=nvcr.io \
  --docker-username='$oauthtoken' \
  --docker-password="${NGC_CLI_API_KEY}" -n app

# ElevenLabs API Key Secret
kubectl create secret generic elevenlabs-api-key-secret \
  --from-literal=ELEVENLABS_API_KEY="${ELEVENLABS_API_KEY}" -n app

Tokkio Helm Chart Installation#

  1. Environment Setup:

Here we will set up the required environment variables. Refer to Configuration section under TURN Server Setup for more details.

export TURNSERVER_REALM="<replace-with-turn-server-realm>"  # same as coturn setup
export TURNSERVER_USERNAME="<replace-with-turn-server-username>"  # same as coturn setup
export TURNSERVER_PASSWORD="<replace-with-turn-server-password>"  # same as coturn setup

export TURN_SERVER_IP="<replace-with-turn-server-ip>"  # Instance IP used to ssh

  1. Create Override Values:

Here we will generate the override values file. This file will contain the configuration for the Tokkio Helm Chart, and will be used in helm install command.

cat <<EOF | envsubst > /tmp/ucf-tokkio-audio-video-app.yml
tokkio-app:
  vms:
    configs:
      vst_config.json:
        network:
          static_turnurl_list:
          - "${TURNSERVER_USERNAME}:${TURNSERVER_PASSWORD}@${TURN_SERVER_IP}:3478"
  ia-unreal-renderer-microservice:
    configs:
      IAUEMS_SIGNALLING_SERVER_PEER_CONNECTION_OPTIONS: |
        {
          "iceServers": [
            {
              "urls": ["turn:${TURN_SERVER_IP}:3478"],
              "username": "${TURNSERVER_USERNAME}",
              "credential": "${TURNSERVER_PASSWORD}"
            }
          ],
          "iceTransportPolicy": "relay"
        }
EOF

  1. Install Tokkio Helm Chart:

Now, we will install the Tokkio helm chart using the override values file and the NGC CLI API key.

helm upgrade \
  --install \
  --reset-values \
  --create-namespace \
  --namespace app \
  --username '$oauthtoken' \
  --password "${NGC_CLI_API_KEY}" \
  --values /tmp/ucf-tokkio-audio-video-app.yml \
  tokkio-app \
  https://helm.ngc.nvidia.com/nvidia/ace/charts/tokkio-1stream-with-ui-5.0.0-beta.tgz

  1. Additional Configuration:

At times we will have additional override values for example, to customize application. You can include them during installation in a separate file. For example, if you want to use a custom asset for the unreal renderer, you can include the following overrides file (/tmp/additional-overrides.yml):

tokkio-app:
  ia-unreal-renderer-microservice:
    applicationSpecs:
      deployment:
        initContainers:
        - env:
          - name: REMOTE_RESOURCE_PATH
            value: "{'metahumanPak': '', 'scenePak': '', 'unrealEngineProject': 'my-org/my-team/unreal-renderer-custom-asset:0.1.0'}"
          - name: DESTINATION_RESOURCE_PATH
            value: /home/interactive-avatar
          envFrom:
          - secretRef:
              name: ngc-api-key-secret
          image: nvcr.io/my-org/my-team/ngc-resource-downloader:1.2.0
          imagePullPolicy: IfNotPresent
          name: init
          volumeMounts:
          - mountPath: /home/interactive-avatar
            name: asset-volume

  1. To apply additional overrides:

helm upgrade \
  --install \
  --reset-values \
  --create-namespace \
  --namespace app \
  --username '$oauthtoken' \
  --password "${NGC_CLI_API_KEY}" \
  --values /tmp/ucf-tokkio-audio-video-app.yml \
  --values /tmp/additional-overrides.yml \
  tokkio-app \
  https://helm.ngc.nvidia.com/nvidia/ace/charts/tokkio-1stream-with-ui-5.0.0-beta.tgz

Note

Similar to the above, you can use helm upgrade --install to install the helm chart with additional overrides, or even change the helm chart with a different version.

Accessing the UI#

  1. Launch the UI in your browser:

    https://<app-instance-ip>:30111

  2. Click “Advanced” when prompted about the unsigned certificate.

  3. Review the security message and click “Proceed to…” to continue

  4. Allow audio permissions when prompted by the browser

Note

The unsigned certificate warning is expected for development environments. Production deployments should use proper SSL certificates.

Bare-metal Installation

Accessing Monitoring and Configuration#

  1. Grafana Dashboard:

    Access Grafana to view system metrics and logs: http://<application-instance-ip>:32300/

  2. ACE Configurator:

    By Default, the ACE Configurator interface is available at: http://<application-instance-ip>:30180

On this page