Customize Reference Workflows#

The reference workflows described in section Reference Workflows represent high-level starting points for Tokkio. This section lists the common customizations that can be performed for these workflows.

Some of these customizations require rebuilding the application using UCS tools in order to create an updated helm chart. Each customization section uses an indicator is_rebuild_needed: Yes/No to indicate that.

Note

This section assumes that you have one of the reference workflows already successfully deployed.

RAG Endpoint Customization#

is_rebuild_needed: No

This is one of the most common customizations that can be performed easily with single line changes. The LLM RAG workflow uses a NIM endpoint by default. One can follow the steps below to tweak the options to cater to the use cases described below:

Use cases :

  1. Use OpenAI as the LLM endpoint (instead of the default NIM endpoint)

  2. Use RAG pipeline and a custom RAG deployment URL

  3. Change the NIM or OpenAI model used as the LLM endpoint.

Steps listed below are valid for any rendering variation of the LLM RAG bot, but are listed with the single stream OV rendering helm chart as an example:

  1. Download the helm chart of the sample LLM RAG workflow from NGC. As an example, here is a link to the single stream OV rendering variation of the helm chart https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ace/helm-charts/ucs-tokkio-app-base-1-stream-llm-rag-3d-ov.

  2. Update the values.yaml from the downloaded helm chart folder as shown below.

    ace-agent-plugin-server:
  pluginConfig:
    plugins:
      rag:
        parameters:
            USE_RAG: false
            RAG_SERVER_URL: http://IP:PORT
            NIM_MODEL: "meta/llama3-8b-instruct" #as defined at https://build.nvidia.com/meta/llama-3_1-8b-instruct
            USE_OPENAI: false
            OPENAI_MODEL: "gpt-4"

  3. To use OpenAI as the LLM endpoint (instead of the default NIM endpoint), set USE_OPENAI: true

  4. To use RAG pipeline and a custom RAG deployment URL, update USE_RAG: true and RAG_SERVER_URL: http://IP:PORT

  5. To change the NIM or OpenAI model used as the LLM endpoint, please update the selection by specifying the value for NIM_MODEL or the OPENAI_MODEL accordingly.

    Note

    Ensure that the response format of the choosen LLM model or the RAG endpoint is compatible with the out of the box implementation for the LLM RAG bot. If not, update would be needed in rag.py from the downloaded source to adapt to the needed request/response format. Source and implementation details for the LLM RAG resource provided here.

  6. Follow the steps listed in Integrating Customization changes without rebuild to reflect the changes in your already deployed Tokkio application.

Avatar and Scene Customization#

is_rebuild_needed: No

Use case:

  1. Custom Avatar/scene displayed in the UI for Tokkio deployment

Tokkio allows you customize your avatar with the Avatar Configurator or alternatively you can import a third-party avatar by following the custom avatar guide. More information can be found in the Avatar Customization section.

Please note that there are various alternative methods to upload avatar scenes.

This document describes how to upload an ACE avatar scene to NGC using the NGC Resource Downloader and how to update the UCS configuration to reference these new scene files. These instructions apply for custom avatar scenes as well as for scenes created with the Avatar Configurator.

The steps listed below are valid for any rendering variation of the LLM RAG bot, but are listed with the single stream OV rendering helm chart as an example:

  1. Download the helm chart of the sample LLM RAG workflow from NGC. As an example, here is a link to the single stream OV rendering variation of the helm chart https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ace/helm-charts/ucs-tokkio-app-base-1-stream-llm-rag-3d-ov.

  2. Update the values.yaml from the downloaded helm chart folder as shown below. Note that only the relevant sections from values.yaml are shown below. Replace PATH_TO_REMOTE_RESOURCE value with the path to your resource location.

    ia-animation-graph-microservice:
  initContainers:
  - command:
    - /bin/bash
    - download_resource.sh
    env:
    - name: REMOTE_RESOURCE_PATH
      value: "<PATH_TO_REMOTE_RESOURCE>"

  resourceDownload:
    remoteResourcePath: "<PATH_TO_REMOTE_RESOURCE>"

ia-omniverse-renderer-microservice:
  initContainers:
  - command:
    - /bin/bash
    - download_resource.sh
    env:
    - name: REMOTE_RESOURCE_PATH
      value: "<PATH_TO_REMOTE_RESOURCE>"
  resourceDownload:
      remoteResourcePath: "<PATH_TO_REMOTE_RESOURCE>"

  3. Follow the steps listed in Integrating Customization changes without rebuild to reflect the changes in your already deployed Tokkio application.

Avatar Voice Customization (Nvidia Riva TTS)#

is_rebuild_needed: No

Use case:

  1. Select a voice option for the custom avatar, when using Riva TTS (default configuration with Tokkio)

Tokkio allows the users to customize the avatar voice easily with Riva TTS configuration. Checkout the selection availability on Riva TTS for more information.

The steps listed below are valid for any rendering variation of the LLM RAG bot, but are listed with the single stream OV rendering helm chart as an example:

  1. Download the helm chart of the sample LLM RAG workflow from NGC. As an example, here is a link to the single stream OV rendering variation of the helm chart https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ace/helm-charts/ucs-tokkio-app-base-1-stream-llm-rag-3d-ov.

  2. Update the values.yaml from the downloaded helm chart folder as shown below. Note that only the relevant sections from values.yaml are shown below. Replace voice_name value with the selection of your choice from Riva TTS compatible with the speech model selection. The speech model for TTS is also listed below for convenience.

    riva:
  ngcModelConfigs:
    triton0:
      models:
      - nvidia/ace/rmir_asr_parakeet_1-1b_en_us_str_vad:2.17.0
      - nvidia/riva/rmir_tts_fastpitch_hifigan_en_us_ipa:2.17.0
  .
  .
riva_tts:
  RivaTTS:
    voice_name: English-US.Male-1 # Replace with a voice selection of choice

  3. Follow the steps listed in Integrating Customization changes without rebuild to reflect the changes in your already deployed Tokkio application.

Plugin Resource Customization#

is_rebuild_needed: No

Use cases:

  1. Change the greeting at the start/stop of the conversation

  2. Update the request or response schema to connect to the custom RAG pipeline

  3. Add filler words to mask the latency of fetching the response from the pipeline

  4. Change avatar name

  5. Update bot gestures

Refer to Tokkio LLM RAG for implementing various customizations as described in the use cases above and publish the bot. Publish the bot

The steps listed below are valid for any rendering variation of the LLM RAG bot, but are listed with the single stream OV rendering helm chart as an example:

  1. Download the helm chart of the sample LLM RAG workflow from NGC. As an example, here is a link to the single stream OV rendering variation of the helm chart https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ace/helm-charts/ucs-tokkio-app-base-1-stream-llm-rag-3d-ov.

  2. Update the values.yaml from the downloaded helm chart folder as shown below:

    ace-agent-chat-controller:
    configNgcPath: "<path to the custom ACE bot resource on ngc>"
    .
    .
ace-agent-chat-engine:
    configNgcPath: <path to the custom ACE bot resource on ngc>
    .
    .
ace-agent-plugin-server:
    configNgcPath: "<path to the custom ACE bot resource on ngc>"
    .
    .
# Optional edit. Only applicable is using NLP server
ace-agent-nlp-server:
    configNgcPath: "<path to the custom ACE bot resource on ngc>"

  3. Follow the steps listed in Integrating Customization changes without rebuild to reflect the changes in your already deployed Tokkio application.

Disable User Attention#

is_rebuild_needed: No

Use case:

  1. Disable the bot behavior of responding to the user queries only when user is directly looking into the camera.

The steps listed below are valid for any rendering variation of the LLM RAG bot, but are listed with the single stream OV rendering helm chart as an example:

  1. Download the helm chart of the sample LLM RAG workflow from NGC. As an example, here is a link to the single stream OV rendering variation of the helm chart https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ace/helm-charts/ucs-tokkio-app-base-1-stream-llm-rag-3d-ov.

  2. To disable the user attention, set the enableUserAttention flag to false in the values.yaml as shown below.

    ace-agent-chat-engine:
    enableUserAttention: "false"

  3. Follow the steps listed in Integrating Customization changes without rebuild to reflect the changes in your already deployed Tokkio application.

Vision AI customizations#

is_rebuild_needed: Yes

Use cases:

  1. Use Tokkio without the Vision AI component.

  2. Apply customizations for model and parameter configurations for Vision AI microservice

Disable Tokkio vision section provides information regarding using Tokkio without vision AI

Vision and AI Analytics section provides information regarding the customizations for model and parameter configurations.

Follow the steps listed in Integrating Customization changes with rebuild to reflect the changes by re-deploying your Tokkio application.

Third party TTS Voice Customization#

is_rebuild_needed: Yes

Use case:

  1. Use Eleven Labs or other third party TTS solution with Tokkio

Please refer to Using 3rd Party Text-to-Speech (TTS) Solutions for using 3rd party TTS solutions. For more information on customizing TTS pronounciation, please refer to Customizing TTS Pronunciation using IPA.

Follow the steps listed in Integrating Customization changes with rebuild to reflect the changes by re-deploying your Tokkio application.

UI Customization#

is_rebuild_needed: No

Use case:

  1. Use a custom UI layout for Tokkio

The UI is not currently a part of Tokkio helm chart. So, UI changes can be made independently of the rest of the Tokkio microservices. Please refer to the UI Customization Guide for the different UI customization options.

Audio2Face Microservice Customization#

is_rebuild_needed: No

Use case:

  1. Tune avatar’s facial animation and lip motion

Tokkio uses the Audio2Face microservice to drive the avatar’s facial animation including the lip motion. The Audio2Face microservice takes avatar speech as input and generates a facial animation stream. The quality of the facial animation depends on the 3D avatar model and its blendshape setup. In addition, the generation also depends on the voice that drives that animation. To account for the different avatar assets and the different voice inputs the Audio2Face microservice exposes a handful of parameters that can be tuned to improve the visual quality of the facial animation.

More information about the various parameters can be found in the Audio2Face parameter tuning guide.

The steps listed below are valid for any rendering variation of the LLM RAG bot, but are listed with the single stream OV rendering helm chart as an example:

  1. Download the helm chart of the sample LLM RAG workflow from NGC. Here is a link to the single stream OV rendering variation of the helm chart https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ace/helm-charts/ucs-tokkio-app-base-1-stream-llm-rag-3d-ov

  2. Update the values.yaml from the downloaded helm chart folder.The following Audio2Face example configuration has been optimized for the ElevenLab’s Jessica voice and the Claire avatar model.

    .. # bsWeightOffsets: [
.. #     0.0
.. #     0.0
.. #     0.0
.. #     0.0
.. #     0.0
.. #     0.0
.. #     0.0
.. #     0.0
.. #     0.0
.. #     0.0
.. #     0.0
.. #     0.0
.. #     0.0
.. #     0.0
.. #     0.0
.. #     0.0
.. #     0.0
.. #     0.0
.. #     0.0
.. #     0.0
.. #     0.0
.. #     0.0
.. #     0.0
.. #     0.0
.. #     0.0
.. #     0.0
.. #     0.0
.. #     0.0
.. #     0.0
.. #     0.0
.. #     0.0
.. #     0.0
.. #     0.0
.. #     0.0
.. #     0.0
.. #     0.0
.. #     0.0
.. #     0.0
.. #     0.0
.. #     0.0
.. #     0.0
.. #     0.0
.. #     0.0
.. #     0.0
.. #     0.0
.. #     0.0
.. #     0.0
.. #     0.0
.. #     0.0
.. #     0.0
.. #     0.0
.. #     0.0
.. # ]
.. # <-----------
.. # Not supported: bsWeightOffsets
.. # TODO: This should be a chat controller config
.. # "preferred_emotion_names": [
.. #     "amazement",
.. #     "anger",
.. #     "cheekiness",
.. #     "disgust",
.. #     "fear",
.. #     "grief",
.. #     "joy",
.. #     "outofbreath",
.. #     "pain",
.. #     "sadness"
.. # ],
.. # "preferred_emotion": [
.. # 0.30820000171661377,
.. # 0.0,
.. # 0.4726000130176544,
.. # 0.0,
.. # 0.0,
.. # 0.0,
.. # 0.30000001192092896,
.. # 0.0,
.. # 0.0,
.. # 0.0
.. # ],

  3. Follow the steps listed in Integrating Customization changes without rebuild to reflect the changes in your already deployed Tokkio application.

Renderer specific customization#

Please refer to Reference Workflows for renderer based options - Omniverse Renderer, Live Portrait or Unreal Engine and their related customizations.

Adding a new microservice#

If required, you can also add a new microservice to Tokkio in order to customize your use case. Refer to UCS tool for creating and building a new UCS microservice.

Note

The addition of a microservice to Tokkio pipeline can be done with careful consideration of the the endpoints that the microservice would interface with. One can use UCS Studio for visualizing and connecting microservice endpoints for the additional microservice.

Once the microservice is added to the Tokkio graph in UCS Studio, follow the steps listed in Integrating Customization changes with rebuild to reflect the changes by re-deploying your Tokkio application.

On this page