Tokkio LLM-RAG - A2F-2D#
Note
This workflow leverages Audio2Face-2D. To access this resource, apply for ACE Early Access. Once approved, verify you have access to this resource, or some of the hyperlinks might not work.
Introduction#
Tokkio-A2F-2D, empowered by Audio2Face-2D / Speech Live Portrait, allows users to setup their avatar by simply providing a png portrait image file. There are various settings that users can configure based on their needs between performance and quality. Some facial features (e.g. eye focus / eye blink / etc.) can also be customized. More information on the available configurations on the A2F-2D microservice are explained in detail from its A2F-2D user guide.
This reference app is a variation of Tokkio LLM-RAG using A2F-2D as its renderer. Other workflows such as Tokkio Retail can also be used with this particular rendering option.
Minimum GPU Requirements#
L4 |
A10 |
|
|---|---|---|
1 stream |
2x |
2x |
3 streams |
4x |
4x |
Prerequisite#
A frontal face portrait png image file
Access to ACE Early Access assets in NGC
Architecture#
Below diagram illustrates the overview of the components that comprise the Tokkio-A2F-2D app as well as their relationship and interaction.
Deployment#
Helm Chart Deployment#
To deploy the Tokkio-A2F-2D app with default settings, use the sample A2F-2D helm chart to deploy. Refer to Tokkio deployment for details on how to install the chart with One-Click scripts.
Running the App#
Run the Front-end to test out your deployment.
Note
The steps below are optional.
Customizations#
Additional steps specific to the Tokkio A2F-2D app customization are explained below.
For a complete guide on the available configuration options and general tuning guideline, refer to the A2F-2D user guide.
Portrait picture customization#
1. Create a frontal face png file#
A png portrait image file that clearly shows the frontal face of the person being animated is required. A sample portrait image is shown below:
2. Replace the default png file from the POD#
Note
The updated png will be replaced by the default portrait png once the chat-controller POD is restarted.
$ kubectl cp <portrait-png> <chat_controller_pod_name>:/workspace/riva/lp_portrait.png
Replace the above placeholder(s) as follow:
<portrait-png> == path of your portrait png file
<chat_controller_pod_name> == POD name of the chat-controller
Animation Property Tuning#
Refer to A2F-2D Customization for instructions.