Kubernetes Deployment

To start with a simple kubernetes deployment of an Audio2Face cluster, you can use our quick deployment script.

It assumes the following dependencies:

• microk8s

• microk8s GPU addon running the NVIDIA GPU operator.

• A valid NGC token to connect to NVIDIA’s image repository as well as a configured NGC CLI.

Make sure you have your Personal Key.

Note

NVIDIA Fabric Manager Requirement for Multi-GPU Systems: If your Kubernetes host uses multiple GPUs connected via NVLink or NVSwitch (e.g., DGX, HGX, or 8x B200 systems), NVIDIA Fabric Manager must be installed and running on the host before proceeding. Without it, CUDA operations will fail with error 802 even though nvidia-smi shows all GPUs. See CUDA Error 802 “system not yet initialized” on B200 Multi-GPU Systems in the Troubleshooting guide for installation instructions.

Note

If this is the first time you are deploying A2F and your host machine already have a kubernetes setup, please remove ~/.kube/ folder. You may make a backup of this folder if required:

$ mv ~/.kube ~/.kube.bk

1. Install microk8s, GPU addon, set local path provisioner and set secrets

Run the following command to install microk8s with the GPU addon with snap.

$ if [ -d $HOME/.kube ]; then
          echo "kubernetes already setup; skipping microk8s installation; please rm -rf $HOME/.kube to force installation"
  else
          sudo snap install microk8s --revision=5891 --classic
          sudo microk8s enable gpu --version v23.6.1
          sudo snap install --classic kubectl
          sudo snap install helm --classic
          sudo usermod -a -G microk8s ${USER}
          mkdir -p $HOME/.kube
          sudo microk8s config > $HOME/.kube/config
          sudo chown $(id -u):$(id -g) $HOME/.kube/config
          sudo microk8s stop
          sudo microk8s refresh-certs --cert ca.crt
          sudo microk8s start
          sudo microk8s status --wait-ready
  fi

Warning

The --revision=5891 flag pins MicroK8s to a tested snap revision. Installing without this flag (e.g., using --channel=1.31/stable) may pull a newer revision where the bundled containerd config template is incompatible with the containerd binary, causing MicroK8s to fail to start. See MicroK8s Fails to Start (Containerd Config Version Mismatch) in Troubleshooting if you encounter this issue.

Then to avoid using sudo run:

$ newgrp microk8s

Note

Throughout this guide, commands use sudo microk8s kubectl (not standalone kubectl). The standalone kubectl installed via snap requires a separate kubeconfig and will fail with connection to the server localhost:8080 was refused if not configured. See kubectl Connection Refused on localhost:8080 in Troubleshooting if you encounter this.

Note

If you see Sorry, home directories outside of /home needs configuration, your home directory is outside the standard /home path (e.g., /localhome/username). This is a snap confinement limitation that affects all snap-installed tools including microk8s, helm, and kubectl.

Workarounds:

• Continue using sudo with all microk8s commands.

• For helm fetch / helm pull, use curl instead (see Snap Tools Fail with “Home Directories Outside of /home” in Troubleshooting).

• Or configure snap to allow non-standard home directories:

  $ sudo snap set system homedirs=/your/actual/home/path
  

  See https://snapcraft.io/docs/home-outside-home for details.

Verify the GPU Operator pods are healthy and GPUs are detected:

$ sudo microk8s kubectl get pods -n gpu-operator-resources
$ sudo microk8s kubectl describe node | grep nvidia.com/gpu

You should see nvidia.com/gpu in the node’s allocatable resources before proceeding.

Warning

If nvidia-container-toolkit-daemonset is in CrashLoopBackOff with the error failed to create device node nvidiactl, apply the symlink fix described in Kubernetes Pod Stuck in Pending Status. This is a known GPU Operator issue observed on L40, L40S, B200, RTX 5090, and others. Do not apply this fix preemptively — disabling symlink creation on systems that don’t have the crash can cause containers to lose GPU access after a systemctl daemon-reload on hosts using systemd cgroup management.

Set local path provisioner by running:

$ curl https://raw.githubusercontent.com/rancher/local-path-provisioner/v0.0.23/deploy/local-path-storage.yaml | sed 's/^  name: local-path$/  name: mdx-local-path/g' | sudo microk8s kubectl apply -f -

Export the NGC API KEY in an environment variable:

$ export NGC_API_KEY=<value>

Delete any pre-existing secrets and set the required secrets with your valid NGC_API_KEY:

$ sudo microk8s kubectl delete secret --ignore-not-found ngc-docker-reg-secret
$ sudo microk8s kubectl delete secret --ignore-not-found ngc-api-key-secret
$ sudo microk8s kubectl create secret docker-registry ngc-docker-reg-secret --docker-username='$oauthtoken' --docker-password=$NGC_API_KEY --docker-server=nvcr.io
$ sudo microk8s kubectl create secret generic ngc-api-key-secret --from-literal=NGC_API_KEY=$NGC_API_KEY

2. Download and install the helm chart

Use the following command to create a new directory and download the Helm chart into it:

$ export CHART_VERSION=2.0
$ mkdir a2f-3d-nim/ && cd a2f-3d-nim/
$ ngc registry chart pull nim/nvidia/audio2face-3d:${CHART_VERSION}
$ tar xvf audio2face-3d-${CHART_VERSION}.tgz

Note

If ngc or helm fetch fails with the snap home directory error, download the chart with curl instead:

$ curl -o audio2face-3d-${CHART_VERSION}.tgz \
    -u "\$oauthtoken:${NGC_API_KEY}" \
    https://helm.ngc.nvidia.com/nim/nvidia/charts/audio2face-3d-${CHART_VERSION}.tgz
$ tar xvf audio2face-3d-${CHART_VERSION}.tgz

The Helm chart includes observability features for the A2F-3D service. For additional details about observability, refer to this page: Observability.

More information about updating the configuration

The config map can be found in audio2face-3d/charts/a2f/values.yaml file starting in line 140. It is a concatenation of the same 3 YAML files used in the Audio2Face-3D NIM Container Deployment and Configuration Guide, and appears as follows:

configs:
  advanced_config.yaml
    ...
  deployment_config.yaml
    ...
  stylization_config.yaml
    ...

You can update the values directly in that file. For different models, use the content of the following three stylization files:

claire_stylization_config.yaml

# These are the default emotions applied at the beginning of any audio clip.
# Their values range from 0.0 to 1.0
default_beginning_emotions:
  amazement: 0.0
  anger: 0.0
  cheekiness: 0.0
  disgust: 0.0
  fear: 0.0
  grief: 0.0
  joy: 0.0
  outofbreath: 0.0
  pain: 0.0
  sadness: 0.0

a2e:
  enabled: true # Enable audio2emotion, ai-generated audio-driven emotion
  live_transition_time: 0.5 # Controls the smoothness of the output transition toward the target value across frames; higher values result in smoother transitions. Each frame updates at a rate of <frame time length> / <live transition time> (capped at 1.0) toward the raw result.
  post_processing_params:
    emotion_contrast: 1.0 # Increases the spread between emotion values by pushing them higher or lower
    emotion_strength: 0.6 # Sets the strength of generated emotions relative to neutral emotion
    enable_preferred_emotion: true # Activate blending preferred emotion with auto-emotion
    live_blend_coef: 0.7 # Coefficient for exponential smoothing of emotion
    max_emotions: 3 # Sets a firm limit on the quantity of emotion sliders engaged by A2E - emotions with the highest weight will be prioritized
    preferred_emotion_strength: 0.5 # Sets the strength of the preferred emotion (if is loaded) relative to generated emotions

a2f:
  # regression / diffusion
  inference_type: regression

  regression_model:
    inference_model_id: claire_v2.3.1

  diffusion_model:
    inference_model_id: multi_v3.2
    identity: claire
    # If true, use deterministic noise for diffusion inference (more stable/repeatable results).
    # If false, use non-deterministic noise (more variation between runs).
    constant_noise: true

  # Enable or disable tongue blendshapes output
  enable_tongue_blendshapes: false

  face_params:
    eyelid_offset: 0.0 # Adjusts the default pose of eyelid open-close
    face_mask_level: 0.6 # Determines the boundary between the upper and lower regions of the face
    face_mask_softness: 0.0085 # Determines how smoothly the upper and lower face regions blend on the boundary
    input_strength: 1.0 # Controls the magnitude of the input audio
    lip_close_offset: 0.0 # Adjusts the default pose of lip close-open
    lower_face_smoothing: 0.006 # Applies temporal smoothing to the lower face motion
    lower_face_strength: 1.25 # Controls the range of motion on the lower regions of the face
    skin_strength: 1.0 # Controls the range of motion of the skin
    upper_face_smoothing: 0.001 # Applies temporal smoothing to the upper face motion
    upper_face_strength: 1.0 # Controls the range of motion on the upper regions of the face
    tongue_strength: 1.3 # Controls the range of motion of the tongue
    tongue_height_offset: 0.0 # Controls the height of the tongue
    tongue_depth_offset: 0.0 # Controls the depth of the tongue

  blendshape_params: # Modulates the effect of each blendshapes. Gain * w + offset
    # Clamps blendshape weights to [0.0, 1.0] range. Recommended for production to ensure compatibility with renderers.
    enable_clamping_bs_weight: true
    weight_multipliers: # Modulates the effect of each blendshapes. blendshape_values * weight_multipliers + weight_offsets (for more details, see the documentation for blendshape_params) 
      EyeBlinkLeft: 1.0
      EyeLookDownLeft: 1.0
      EyeLookInLeft: 1.0
      EyeLookOutLeft: 1.0
      EyeLookUpLeft: 1.0
      EyeSquintLeft: 1.0
      EyeWideLeft: 1.0
      EyeBlinkRight: 1.0
      EyeLookDownRight: 1.0
      EyeLookInRight: 1.0
      EyeLookOutRight: 1.0
      EyeLookUpRight: 1.0
      EyeSquintRight: 1.0
      EyeWideRight: 1.0
      JawForward: 1.0
      JawLeft: 1.0
      JawRight: 1.0
      JawOpen: 1.0
      MouthClose: 1.0
      MouthFunnel: 1.0
      MouthPucker: 1.0
      MouthLeft: 1.0
      MouthRight: 1.0
      MouthSmileLeft: 1.0
      MouthSmileRight: 1.0
      MouthFrownLeft: 1.0
      MouthFrownRight: 1.0
      MouthDimpleLeft: 1.0
      MouthDimpleRight: 1.0
      MouthStretchLeft: 1.0
      MouthStretchRight: 1.0
      MouthRollLower: 1.0
      MouthRollUpper: 1.0
      MouthShrugLower: 1.0
      MouthShrugUpper: 1.0
      MouthPressLeft: 1.0
      MouthPressRight: 1.0
      MouthLowerDownLeft: 1.0
      MouthLowerDownRight: 1.0
      MouthUpperUpLeft: 1.0
      MouthUpperUpRight: 1.0
      BrowDownLeft: 1.0
      BrowDownRight: 1.0
      BrowInnerUp: 1.0
      BrowOuterUpLeft: 1.0
      BrowOuterUpRight: 1.0
      CheekPuff: 1.0
      CheekSquintLeft: 1.0
      CheekSquintRight: 1.0
      NoseSneerLeft: 1.0
      NoseSneerRight: 1.0
      TongueOut: 1.0
      TongueTipUp: 1.0
      TongueTipDown: 1.0
      TongueTipLeft: 1.0
      TongueTipRight: 1.0
      TongueRollUp: 1.0
      TongueRollDown: 1.0
      TongueRollLeft: 1.0
      TongueRollRight: 1.0
      TongueUp: 1.0
      TongueDown: 1.0
      TongueLeft: 1.0
      TongueRight: 1.0
      TongueIn: 1.0
      TongueStretch: 1.0
      TongueWide: 1.0
      TongueNarrow: 1.0
    weight_offsets: # Modulates the effect of each blendshapes. blendshape_values * weight_multipliers + weight_offsets (for more details, see the documentation for blendshape_params)
      EyeBlinkLeft: 0.0
      EyeLookDownLeft: 0.0
      EyeLookInLeft: 0.0
      EyeLookOutLeft: 0.0
      EyeLookUpLeft: 0.0
      EyeSquintLeft: 0.0
      EyeWideLeft: 0.0
      EyeBlinkRight: 0.0
      EyeLookDownRight: 0.0
      EyeLookInRight: 0.0
      EyeLookOutRight: 0.0
      EyeLookUpRight: 0.0
      EyeSquintRight: 0.0
      EyeWideRight: 0.0
      JawForward: 0.0
      JawLeft: 0.0
      JawRight: 0.0
      JawOpen: 0.0
      MouthClose: 0.0
      MouthFunnel: 0.0
      MouthPucker: 0.0
      MouthLeft: 0.0
      MouthRight: 0.0
      MouthSmileLeft: 0.0
      MouthSmileRight: 0.0
      MouthFrownLeft: 0.0
      MouthFrownRight: 0.0
      MouthDimpleLeft: 0.0
      MouthDimpleRight: 0.0
      MouthStretchLeft: 0.0
      MouthStretchRight: 0.0
      MouthRollLower: 0.0
      MouthRollUpper: 0.0
      MouthShrugLower: 0.0
      MouthShrugUpper: 0.0
      MouthPressLeft: 0.0
      MouthPressRight: 0.0
      MouthLowerDownLeft: 0.0
      MouthLowerDownRight: 0.0
      MouthUpperUpLeft: 0.0
      MouthUpperUpRight: 0.0
      BrowDownLeft: 0.0
      BrowDownRight: 0.0
      BrowInnerUp: 0.0
      BrowOuterUpLeft: 0.0
      BrowOuterUpRight: 0.0
      CheekPuff: 0.0
      CheekSquintLeft: 0.0
      CheekSquintRight: 0.0
      NoseSneerLeft: 0.0
      NoseSneerRight: 0.0
      TongueOut: 0.0
      TongueTipUp: 0.0
      TongueTipDown: 0.0
      TongueTipLeft: 0.0
      TongueTipRight: 0.0
      TongueRollUp: 0.0
      TongueRollDown: 0.0
      TongueRollLeft: 0.0
      TongueRollRight: 0.0
      TongueUp: 0.0
      TongueDown: 0.0
      TongueLeft: 0.0
      TongueRight: 0.0
      TongueIn: 0.0
      TongueStretch: 0.0
      TongueWide: 0.0
      TongueNarrow: 0.0

    active_poses: # Specifies which blendshapes are active for each pose (for more details, see the documentation for blendshape_params)
      EyeBlinkLeft: 1
      EyeLookDownLeft: 0
      EyeLookInLeft: 0
      EyeLookOutLeft: 0
      EyeLookUpLeft: 0
      EyeSquintLeft: 1
      EyeWideLeft: 1
      EyeBlinkRight: 1
      EyeLookDownRight: 0
      EyeLookInRight: 0
      EyeLookOutRight: 0
      EyeLookUpRight: 0
      EyeSquintRight: 1
      EyeWideRight: 1
      JawForward: 1
      JawLeft: 1
      JawRight: 1
      JawOpen: 1
      MouthClose: 1
      MouthFunnel: 1
      MouthPucker: 1
      MouthLeft: 1
      MouthRight: 1
      MouthSmileLeft: 1
      MouthSmileRight: 1
      MouthFrownLeft: 1
      MouthFrownRight: 1
      MouthDimpleLeft: 1
      MouthDimpleRight: 1
      MouthStretchLeft: 1
      MouthStretchRight: 1
      MouthRollLower: 1
      MouthRollUpper: 1
      MouthShrugLower: 1
      MouthShrugUpper: 1
      MouthPressLeft: 1
      MouthPressRight: 1
      MouthLowerDownLeft: 1
      MouthLowerDownRight: 1
      MouthUpperUpLeft: 1
      MouthUpperUpRight: 1
      BrowDownLeft: 1
      BrowDownRight: 1
      BrowInnerUp: 1
      BrowOuterUpLeft: 1
      BrowOuterUpRight: 1
      CheekPuff: 1
      CheekSquintLeft: 1
      CheekSquintRight: 1
      NoseSneerLeft: 1
      NoseSneerRight: 1
      TongueOut: 0
      TongueTipUp: 1
      TongueTipDown: 1
      TongueTipLeft: 1
      TongueTipRight: 1
      TongueRollUp: 1
      TongueRollDown: 1
      TongueRollLeft: 1
      TongueRollRight: 1
      TongueUp: 1
      TongueDown: 1
      TongueLeft: 1
      TongueRight: 1
      TongueIn: 1
      TongueStretch: 1
      TongueWide: 1
      TongueNarrow: 1

    cancel_poses: # Specifies which blendshapes are cancelled for each pose (for more details, see the documentation for blendshape_params)
      EyeBlinkLeft: -1
      EyeLookDownLeft: -1
      EyeLookInLeft: -1
      EyeLookOutLeft: -1
      EyeLookUpLeft: -1
      EyeSquintLeft: -1
      EyeWideLeft: -1
      EyeBlinkRight: -1
      EyeLookDownRight: -1
      EyeLookInRight: -1
      EyeLookOutRight: -1
      EyeLookUpRight: -1
      EyeSquintRight: -1
      EyeWideRight: -1
      JawForward: -1
      JawLeft: -1
      JawRight: -1
      JawOpen: -1
      MouthClose: -1
      MouthFunnel: -1
      MouthPucker: -1
      MouthLeft: -1
      MouthRight: -1
      MouthSmileLeft: -1
      MouthSmileRight: -1
      MouthFrownLeft: -1
      MouthFrownRight: -1
      MouthDimpleLeft: -1
      MouthDimpleRight: -1
      MouthStretchLeft: -1
      MouthStretchRight: -1
      MouthRollLower: -1
      MouthRollUpper: -1
      MouthShrugLower: -1
      MouthShrugUpper: -1
      MouthPressLeft: -1
      MouthPressRight: -1
      MouthLowerDownLeft: -1
      MouthLowerDownRight: -1
      MouthUpperUpLeft: -1
      MouthUpperUpRight: -1
      BrowDownLeft: -1
      BrowDownRight: -1
      BrowInnerUp: -1
      BrowOuterUpLeft: -1
      BrowOuterUpRight: -1
      CheekPuff: -1
      CheekSquintLeft: -1
      CheekSquintRight: -1
      NoseSneerLeft: -1
      NoseSneerRight: -1
      TongueOut: -1
      TongueTipUp: -1
      TongueTipDown: -1
      TongueTipLeft: -1
      TongueTipRight: -1
      TongueRollUp: -1
      TongueRollDown: -1
      TongueRollLeft: -1
      TongueRollRight: -1
      TongueUp: -1
      TongueDown: -1
      TongueLeft: -1
      TongueRight: -1
      TongueIn: -1
      TongueStretch: -1
      TongueWide: -1
      TongueNarrow: -1

    symmetry_poses: # Specifies which blendshapes are symmetrical for each pose (for more details, see the documentation for blendshape_params)
      EyeBlinkLeft: 0
      EyeLookDownLeft: -1
      EyeLookInLeft: -1
      EyeLookOutLeft: -1
      EyeLookUpLeft: -1
      EyeSquintLeft: -1
      EyeWideLeft: 1
      EyeBlinkRight: 0
      EyeLookDownRight: -1
      EyeLookInRight: -1
      EyeLookOutRight: -1
      EyeLookUpRight: -1
      EyeSquintRight: -1
      EyeWideRight: 1
      JawForward: -1
      JawLeft: -1
      JawRight: -1
      JawOpen: -1
      MouthClose: -1
      MouthFunnel: -1
      MouthPucker: -1
      MouthLeft: -1
      MouthRight: -1
      MouthSmileLeft: 2
      MouthSmileRight: 2
      MouthFrownLeft: 3
      MouthFrownRight: 3
      MouthDimpleLeft: 4
      MouthDimpleRight: 4
      MouthStretchLeft: 5
      MouthStretchRight: 5
      MouthRollLower: -1
      MouthRollUpper: -1
      MouthShrugLower: -1
      MouthShrugUpper: -1
      MouthPressLeft: 6
      MouthPressRight: 6
      MouthLowerDownLeft: 7
      MouthLowerDownRight: 7
      MouthUpperUpLeft: 8
      MouthUpperUpRight: 8
      BrowDownLeft: 10
      BrowDownRight: 10
      BrowInnerUp: -1
      BrowOuterUpLeft: 9
      BrowOuterUpRight: 9
      CheekPuff: -1
      CheekSquintLeft: 11
      CheekSquintRight: 11
      NoseSneerLeft: 12
      NoseSneerRight: 12
      TongueOut: -1
      TongueTipUp: -1
      TongueTipDown: -1
      TongueTipLeft: -1
      TongueTipRight: -1
      TongueRollUp: -1
      TongueRollDown: -1
      TongueRollLeft: -1
      TongueRollRight: -1
      TongueUp: -1
      TongueDown: -1
      TongueLeft: -1
      TongueRight: -1
      TongueIn: -1
      TongueStretch: -1
      TongueWide: -1
      TongueNarrow: -1

james_stylization_config.yaml

# These are the default emotions applied at the beginning of any audio clip.
# Their values range from 0.0 to 1.0
default_beginning_emotions:
  amazement: 0.0
  anger: 0.0
  cheekiness: 0.0
  disgust: 0.0
  fear: 0.0
  grief: 0.0
  joy: 0.0
  outofbreath: 0.0
  pain: 0.0
  sadness: 0.0

a2e:
  enabled: true # Enable audio2emotion, ai-generated audio-driven emotion
  live_transition_time: 0.5 # Controls the smoothness of the output transition toward the target value across frames; higher values result in smoother transitions. Each frame updates at a rate of <frame time length> / <live transition time> (capped at 1.0) toward the raw result.
  post_processing_params:
    emotion_contrast: 1.0 # Increases the spread between emotion values by pushing them higher or lower
    emotion_strength: 0.6 # Sets the strength of generated emotions relative to neutral emotion
    enable_preferred_emotion: true # Activate blending preferred emotion with auto-emotion
    live_blend_coef: 0.7 # Coefficient for exponential smoothing of emotion
    max_emotions: 3 # Sets a firm limit on the quantity of emotion sliders engaged by A2E - emotions with the highest weight will be prioritized
    preferred_emotion_strength: 0.5 # Sets the strength of the preferred emotion (if is loaded) relative to generated emotions

a2f:
  # regression / diffusion
  inference_type: regression

  regression_model:
    inference_model_id: james_v2.3.1

  diffusion_model:
    inference_model_id: multi_v3.2
    identity: james
    # If true, use deterministic noise for diffusion inference (more stable/repeatable results).
    # If false, use non-deterministic noise (more variation between runs).
    constant_noise: true

  # Enable or disable tongue blendshapes output
  enable_tongue_blendshapes: false

  face_params:
    eyelid_offset: 0.06 # Adjusts the default pose of eyelid open-close
    face_mask_level: 0.6 # Determines the boundary between the upper and lower regions of the face
    face_mask_softness: 0.0085 # Determines how smoothly the upper and lower face regions blend on the boundary
    input_strength: 1.0 # Controls the magnitude of the input audio
    lip_close_offset: -0.02 # Adjusts the default pose of lip close-open
    lower_face_smoothing: 0.006 # Applies temporal smoothing to the lower face motion
    lower_face_strength: 1.2 # Controls the range of motion on the lower regions of the face
    skin_strength: 1.0 # Controls the range of motion of the skin
    upper_face_smoothing: 0.001 # Applies temporal smoothing to the upper face motion
    upper_face_strength: 1.0 # Controls the range of motion on the upper regions of the face
    tongue_strength: 1.3
    tongue_height_offset: 0.0
    tongue_depth_offset: 0.0

  blendshape_params: # Modulates the effect of each blendshapes. Gain * w + offset
    # Clamps blendshape weights to [0.0, 1.0] range. Recommended for production to ensure compatibility with renderers.
    enable_clamping_bs_weight: true
    
    weight_multipliers: # Modulates the effect of each blendshapes. blendshape_values * weight_multipliers + weight_offsets (for more details, see the documentation for blendshape_params) 
      EyeBlinkLeft: 1.0
      EyeLookDownLeft: 1.0
      EyeLookInLeft: 1.0
      EyeLookOutLeft: 1.0
      EyeLookUpLeft: 1.0
      EyeSquintLeft: 1.0
      EyeWideLeft: 1.0
      EyeBlinkRight: 1.0
      EyeLookDownRight: 1.0
      EyeLookInRight: 1.0
      EyeLookOutRight: 1.0
      EyeLookUpRight: 1.0
      EyeSquintRight: 1.0
      EyeWideRight: 1.0
      JawForward: 1.0
      JawLeft: 1.0
      JawRight: 1.0
      JawOpen: 1.0
      MouthClose: 1.0
      MouthFunnel: 1.0
      MouthPucker: 1.0
      MouthLeft: 1.0
      MouthRight: 1.0
      MouthSmileLeft: 1.0
      MouthSmileRight: 1.0
      MouthFrownLeft: 1.0
      MouthFrownRight: 1.0
      MouthDimpleLeft: 1.0
      MouthDimpleRight: 1.0
      MouthStretchLeft: 1.0
      MouthStretchRight: 1.0
      MouthRollLower: 1.0
      MouthRollUpper: 1.0
      MouthShrugLower: 1.0
      MouthShrugUpper: 1.0
      MouthPressLeft: 1.0
      MouthPressRight: 1.0
      MouthLowerDownLeft: 1.0
      MouthLowerDownRight: 1.0
      MouthUpperUpLeft: 1.0
      MouthUpperUpRight: 1.0
      BrowDownLeft: 1.0
      BrowDownRight: 1.0
      BrowInnerUp: 1.0
      BrowOuterUpLeft: 1.0
      BrowOuterUpRight: 1.0
      CheekPuff: 1.0
      CheekSquintLeft: 1.0
      CheekSquintRight: 1.0
      NoseSneerLeft: 1.0
      NoseSneerRight: 1.0
      TongueOut: 1.0
      TongueTipUp: 1.0
      TongueTipDown: 1.0
      TongueTipLeft: 1.0
      TongueTipRight: 1.0
      TongueRollUp: 1.0
      TongueRollDown: 1.0
      TongueRollLeft: 1.0
      TongueRollRight: 1.0
      TongueUp: 1.0
      TongueDown: 1.0
      TongueLeft: 1.0
      TongueRight: 1.0
      TongueIn: 1.0
      TongueStretch: 1.0
      TongueWide: 1.0
      TongueNarrow: 1.0

    weight_offsets: # Modulates the effect of each blendshapes. blendshape_values * weight_multipliers + weight_offsets (for more details, see the documentation for blendshape_params) 
      EyeBlinkLeft: 0.0
      EyeLookDownLeft: 0.0
      EyeLookInLeft: 0.0
      EyeLookOutLeft: 0.0
      EyeLookUpLeft: 0.0
      EyeSquintLeft: 0.0
      EyeWideLeft: 0.0
      EyeBlinkRight: 0.0
      EyeLookDownRight: 0.0
      EyeLookInRight: 0.0
      EyeLookOutRight: 0.0
      EyeLookUpRight: 0.0
      EyeSquintRight: 0.0
      EyeWideRight: 0.0
      JawForward: 0.0
      JawLeft: 0.0
      JawRight: 0.0
      JawOpen: 0.0
      MouthClose: 0.0
      MouthFunnel: 0.0
      MouthPucker: 0.0
      MouthLeft: 0.0
      MouthRight: 0.0
      MouthSmileLeft: 0.0
      MouthSmileRight: 0.0
      MouthFrownLeft: 0.0
      MouthFrownRight: 0.0
      MouthDimpleLeft: 0.0
      MouthDimpleRight: 0.0
      MouthStretchLeft: 0.0
      MouthStretchRight: 0.0
      MouthRollLower: 0.0
      MouthRollUpper: 0.0
      MouthShrugLower: 0.0
      MouthShrugUpper: 0.0
      MouthPressLeft: 0.0
      MouthPressRight: 0.0
      MouthLowerDownLeft: 0.0
      MouthLowerDownRight: 0.0
      MouthUpperUpLeft: 0.0
      MouthUpperUpRight: 0.0
      BrowDownLeft: 0.0
      BrowDownRight: 0.0
      BrowInnerUp: 0.0
      BrowOuterUpLeft: 0.0
      BrowOuterUpRight: 0.0
      CheekPuff: 0.0
      CheekSquintLeft: 0.0
      CheekSquintRight: 0.0
      NoseSneerLeft: 0.0
      NoseSneerRight: 0.0
      TongueOut: 0.0
      TongueTipUp: 0.0
      TongueTipDown: 0.0
      TongueTipLeft: 0.0
      TongueTipRight: 0.0
      TongueRollUp: 0.0
      TongueRollDown: 0.0
      TongueRollLeft: 0.0
      TongueRollRight: 0.0
      TongueUp: 0.0
      TongueDown: 0.0
      TongueLeft: 0.0
      TongueRight: 0.0
      TongueIn: 0.0
      TongueStretch: 0.0
      TongueWide: 0.0
      TongueNarrow: 0.0

    active_poses: # Specifies which blendshapes are active for each pose (for more details, see the documentation for blendshape_params)
      EyeBlinkLeft: 1
      EyeLookDownLeft: 0
      EyeLookInLeft: 0
      EyeLookOutLeft: 0
      EyeLookUpLeft: 0
      EyeSquintLeft: 1
      EyeWideLeft: 1
      EyeBlinkRight: 1
      EyeLookDownRight: 0
      EyeLookInRight: 0
      EyeLookOutRight: 0
      EyeLookUpRight: 0
      EyeSquintRight: 1
      EyeWideRight: 1
      JawForward: 1
      JawLeft: 1
      JawRight: 1
      JawOpen: 1
      MouthClose: 1
      MouthFunnel: 1
      MouthPucker: 1
      MouthLeft: 1
      MouthRight: 1
      MouthSmileLeft: 1
      MouthSmileRight: 1
      MouthFrownLeft: 1
      MouthFrownRight: 1
      MouthDimpleLeft: 1
      MouthDimpleRight: 1
      MouthStretchLeft: 1
      MouthStretchRight: 1
      MouthRollLower: 1
      MouthRollUpper: 1
      MouthShrugLower: 1
      MouthShrugUpper: 1
      MouthPressLeft: 1
      MouthPressRight: 1
      MouthLowerDownLeft: 1
      MouthLowerDownRight: 1
      MouthUpperUpLeft: 1
      MouthUpperUpRight: 1
      BrowDownLeft: 1
      BrowDownRight: 1
      BrowInnerUp: 1
      BrowOuterUpLeft: 1
      BrowOuterUpRight: 1
      CheekPuff: 1
      CheekSquintLeft: 1
      CheekSquintRight: 1
      NoseSneerLeft: 1
      NoseSneerRight: 1
      TongueOut: 0
      TongueTipUp: 1
      TongueTipDown: 1
      TongueTipLeft: 1
      TongueTipRight: 1
      TongueRollUp: 1
      TongueRollDown: 1
      TongueRollLeft: 1
      TongueRollRight: 1
      TongueUp: 1
      TongueDown: 1
      TongueLeft: 1
      TongueRight: 1
      TongueIn: 1
      TongueStretch: 1
      TongueWide: 1
      TongueNarrow: 1

    cancel_poses: # Specifies which blendshapes are cancelled for each pose (for more details, see the documentation for blendshape_params)
      EyeBlinkLeft: -1
      EyeLookDownLeft: -1
      EyeLookInLeft: -1
      EyeLookOutLeft: -1
      EyeLookUpLeft: -1
      EyeSquintLeft: -1
      EyeWideLeft: -1
      EyeBlinkRight: -1
      EyeLookDownRight: -1
      EyeLookInRight: -1
      EyeLookOutRight: -1
      EyeLookUpRight: -1
      EyeSquintRight: -1
      EyeWideRight: -1
      JawForward: -1
      JawLeft: -1
      JawRight: -1
      JawOpen: -1
      MouthClose: -1
      MouthFunnel: -1
      MouthPucker: -1
      MouthLeft: -1
      MouthRight: -1
      MouthSmileLeft: -1
      MouthSmileRight: -1
      MouthFrownLeft: -1
      MouthFrownRight: -1
      MouthDimpleLeft: -1
      MouthDimpleRight: -1
      MouthStretchLeft: -1
      MouthStretchRight: -1
      MouthRollLower: -1
      MouthRollUpper: -1
      MouthShrugLower: -1
      MouthShrugUpper: -1
      MouthPressLeft: -1
      MouthPressRight: -1
      MouthLowerDownLeft: -1
      MouthLowerDownRight: -1
      MouthUpperUpLeft: -1
      MouthUpperUpRight: -1
      BrowDownLeft: -1
      BrowDownRight: -1
      BrowInnerUp: -1
      BrowOuterUpLeft: -1
      BrowOuterUpRight: -1
      CheekPuff: -1
      CheekSquintLeft: -1
      CheekSquintRight: -1
      NoseSneerLeft: -1
      NoseSneerRight: -1
      TongueOut: -1
      TongueTipUp: -1
      TongueTipDown: -1
      TongueTipLeft: -1
      TongueTipRight: -1
      TongueRollUp: -1
      TongueRollDown: -1
      TongueRollLeft: -1
      TongueRollRight: -1
      TongueUp: -1
      TongueDown: -1
      TongueLeft: -1
      TongueRight: -1
      TongueIn: -1
      TongueStretch: -1
      TongueWide: -1
      TongueNarrow: -1

    symmetry_poses: # Specifies which blendshapes are symmetrical for each pose (for more details, see the documentation for blendshape_params)
      EyeBlinkLeft: 0
      EyeLookDownLeft: -1
      EyeLookInLeft: -1
      EyeLookOutLeft: -1
      EyeLookUpLeft: -1
      EyeSquintLeft: -1
      EyeWideLeft: 1
      EyeBlinkRight: 0
      EyeLookDownRight: -1
      EyeLookInRight: -1
      EyeLookOutRight: -1
      EyeLookUpRight: -1
      EyeSquintRight: -1
      EyeWideRight: 1
      JawForward: -1
      JawLeft: -1
      JawRight: -1
      JawOpen: -1
      MouthClose: -1
      MouthFunnel: -1
      MouthPucker: -1
      MouthLeft: -1
      MouthRight: -1
      MouthSmileLeft: 2
      MouthSmileRight: 2
      MouthFrownLeft: 3
      MouthFrownRight: 3
      MouthDimpleLeft: 4
      MouthDimpleRight: 4
      MouthStretchLeft: 5
      MouthStretchRight: 5
      MouthRollLower: -1
      MouthRollUpper: -1
      MouthShrugLower: -1
      MouthShrugUpper: -1
      MouthPressLeft: 6
      MouthPressRight: 6
      MouthLowerDownLeft: 7
      MouthLowerDownRight: 7
      MouthUpperUpLeft: 8
      MouthUpperUpRight: 8
      BrowDownLeft: 10
      BrowDownRight: 10
      BrowInnerUp: -1
      BrowOuterUpLeft: 9
      BrowOuterUpRight: 9
      CheekPuff: -1
      CheekSquintLeft: 11
      CheekSquintRight: 11
      NoseSneerLeft: 12
      NoseSneerRight: 12
      TongueOut: -1
      TongueTipUp: -1
      TongueTipDown: -1
      TongueTipLeft: -1
      TongueTipRight: -1
      TongueRollUp: -1
      TongueRollDown: -1
      TongueRollLeft: -1
      TongueRollRight: -1
      TongueUp: -1
      TongueDown: -1
      TongueLeft: -1
      TongueRight: -1
      TongueIn: -1
      TongueStretch: -1
      TongueWide: -1
      TongueNarrow: -1

mark_stylization_config.yaml

# These are the default emotions applied at the beginning of any audio clip.
# Their values range from 0.0 to 1.0
default_beginning_emotions:
  amazement: 0.0
  anger: 0.0
  cheekiness: 0.0
  disgust: 0.0
  fear: 0.0
  grief: 0.0
  joy: 0.0
  outofbreath: 0.0
  pain: 0.0
  sadness: 0.0

a2e:
  enabled: true # Enable audio2emotion, ai-generated audio-driven emotion
  live_transition_time: 0.5 # Controls the smoothness of the output transition toward the target value across frames; higher values result in smoother transitions. Each frame updates at a rate of <frame time length> / <live transition time> (capped at 1.0) toward the raw result.
  post_processing_params:
    emotion_contrast: 1.0 # Increases the spread between emotion values by pushing them higher or lower
    emotion_strength: 0.6 # Sets the strength of generated emotions relative to neutral emotion
    enable_preferred_emotion: true # Activate blending preferred emotion with auto-emotion
    live_blend_coef: 0.7 # Coefficient for exponential smoothing of emotion
    max_emotions: 3 # Sets a firm limit on the quantity of emotion sliders engaged by A2E - emotions with the highest weight will be prioritized
    preferred_emotion_strength: 0.5 # Sets the strength of the preferred emotion (if is loaded) relative to generated emotions

a2f:
  # regression / diffusion
  inference_type: regression

  regression_model:
    inference_model_id: mark_v2.3

  diffusion_model:
    inference_model_id: multi_v3.2
    identity: mark
    # If true, use deterministic noise for diffusion inference (more stable/repeatable results).
    # If false, use non-deterministic noise (more variation between runs).
    constant_noise: true

  # Enable or disable tongue blendshapes output
  enable_tongue_blendshapes: false

  face_params:
    eyelid_offset: 0.06 # Adjusts the default pose of eyelid open-close
    face_mask_level: 0.6 # Determines the boundary between the upper and lower regions of the face
    face_mask_softness: 0.0085 # Determines how smoothly the upper and lower face regions blend on the boundary
    input_strength: 1.3 # Controls the magnitude of the input audio
    lip_close_offset: -0.03 # Adjusts the default pose of lip close-open
    lower_face_smoothing: 0.0023 # Applies temporal smoothing to the lower face motion
    lower_face_strength: 1.4 # Controls the range of motion on the lower regions of the face
    skin_strength: 1.1 # Controls the range of motion of the skin
    upper_face_smoothing: 0.001 # Applies temporal smoothing to the upper face motion
    upper_face_strength: 1.0 # Controls the range of motion on the upper regions of the face
    tongue_strength: 1.3 # Controls the range of motion of the tongue
    tongue_height_offset: 0.0 # Controls the height of the tongue
    tongue_depth_offset: 0.0 # Controls the depth of the tongue

  blendshape_params: # Modulates the effect of each blendshapes. Gain * w + offset
    # Clamps blendshape weights to [0.0, 1.0] range. Recommended for production to ensure compatibility with renderers.
    enable_clamping_bs_weight: true
    
    weight_multipliers: # Modulates the effect of each blendshapes. blendshape_values * weight_multipliers + weight_offsets (for more details, see the documentation for blendshape_params) 
      EyeBlinkLeft: 1.0
      EyeLookDownLeft: 1.0
      EyeLookInLeft: 1.0
      EyeLookOutLeft: 1.0
      EyeLookUpLeft: 1.0
      EyeSquintLeft: 1.0
      EyeWideLeft: 1.0
      EyeBlinkRight: 1.0
      EyeLookDownRight: 1.0
      EyeLookInRight: 1.0
      EyeLookOutRight: 1.0
      EyeLookUpRight: 1.0
      EyeSquintRight: 1.0
      EyeWideRight: 1.0
      JawForward: 1.0
      JawLeft: 1.0
      JawRight: 1.0
      JawOpen: 1.0
      MouthClose: 1.0
      MouthFunnel: 1.0
      MouthPucker: 1.0
      MouthLeft: 1.0
      MouthRight: 1.0
      MouthSmileLeft: 1.0
      MouthSmileRight: 1.0
      MouthFrownLeft: 1.0
      MouthFrownRight: 1.0
      MouthDimpleLeft: 1.0
      MouthDimpleRight: 1.0
      MouthStretchLeft: 1.0
      MouthStretchRight: 1.0
      MouthRollLower: 1.0
      MouthRollUpper: 1.0
      MouthShrugLower: 1.0
      MouthShrugUpper: 1.0
      MouthPressLeft: 1.0
      MouthPressRight: 1.0
      MouthLowerDownLeft: 1.0
      MouthLowerDownRight: 1.0
      MouthUpperUpLeft: 1.0
      MouthUpperUpRight: 1.0
      BrowDownLeft: 1.0
      BrowDownRight: 1.0
      BrowInnerUp: 1.0
      BrowOuterUpLeft: 1.0
      BrowOuterUpRight: 1.0
      CheekPuff: 1.0
      CheekSquintLeft: 1.0
      CheekSquintRight: 1.0
      NoseSneerLeft: 1.0
      NoseSneerRight: 1.0
      TongueOut: 1.0
      TongueTipUp: 1.0
      TongueTipDown: 1.0
      TongueTipLeft: 1.0
      TongueTipRight: 1.0
      TongueRollUp: 1.0
      TongueRollDown: 1.0
      TongueRollLeft: 1.0
      TongueRollRight: 1.0
      TongueUp: 1.0
      TongueDown: 1.0
      TongueLeft: 1.0
      TongueRight: 1.0
      TongueIn: 1.0
      TongueStretch: 1.0
      TongueWide: 1.0
      TongueNarrow: 1.0

    weight_offsets: # Modulates the effect of each blendshapes. blendshape_values * weight_multipliers + weight_offsets (for more details, see the documentation for blendshape_params) 
      EyeBlinkLeft: 0.0
      EyeLookDownLeft: 0.0
      EyeLookInLeft: 0.0
      EyeLookOutLeft: 0.0
      EyeLookUpLeft: 0.0
      EyeSquintLeft: 0.0
      EyeWideLeft: 0.0
      EyeBlinkRight: 0.0
      EyeLookDownRight: 0.0
      EyeLookInRight: 0.0
      EyeLookOutRight: 0.0
      EyeLookUpRight: 0.0
      EyeSquintRight: 0.0
      EyeWideRight: 0.0
      JawForward: 0.0
      JawLeft: 0.0
      JawRight: 0.0
      JawOpen: 0.0
      MouthClose: 0.0
      MouthFunnel: 0.0
      MouthPucker: 0.0
      MouthLeft: 0.0
      MouthRight: 0.0
      MouthSmileLeft: 0.0
      MouthSmileRight: 0.0
      MouthFrownLeft: 0.0
      MouthFrownRight: 0.0
      MouthDimpleLeft: 0.0
      MouthDimpleRight: 0.0
      MouthStretchLeft: 0.0
      MouthStretchRight: 0.0
      MouthRollLower: 0.0
      MouthRollUpper: 0.0
      MouthShrugLower: 0.0
      MouthShrugUpper: 0.0
      MouthPressLeft: 0.0
      MouthPressRight: 0.0
      MouthLowerDownLeft: 0.0
      MouthLowerDownRight: 0.0
      MouthUpperUpLeft: 0.0
      MouthUpperUpRight: 0.0
      BrowDownLeft: 0.0
      BrowDownRight: 0.0
      BrowInnerUp: 0.0
      BrowOuterUpLeft: 0.0
      BrowOuterUpRight: 0.0
      CheekPuff: 0.0
      CheekSquintLeft: 0.0
      CheekSquintRight: 0.0
      NoseSneerLeft: 0.0
      NoseSneerRight: 0.0
      TongueOut: 0.0
      TongueTipUp: 0.0
      TongueTipDown: 0.0
      TongueTipLeft: 0.0
      TongueTipRight: 0.0
      TongueRollUp: 0.0
      TongueRollDown: 0.0
      TongueRollLeft: 0.0
      TongueRollRight: 0.0
      TongueUp: 0.0
      TongueDown: 0.0
      TongueLeft: 0.0
      TongueRight: 0.0
      TongueIn: 0.0
      TongueStretch: 0.0
      TongueWide: 0.0
      TongueNarrow: 0.0

    active_poses: # Specifies which blendshapes are active for each pose (for more details, see the documentation for blendshape_params)
      EyeBlinkLeft: 1
      EyeLookDownLeft: 0
      EyeLookInLeft: 0
      EyeLookOutLeft: 0
      EyeLookUpLeft: 0
      EyeSquintLeft: 1
      EyeWideLeft: 1
      EyeBlinkRight: 1
      EyeLookDownRight: 0
      EyeLookInRight: 0
      EyeLookOutRight: 0
      EyeLookUpRight: 0
      EyeSquintRight: 1
      EyeWideRight: 1
      JawForward: 1
      JawLeft: 1
      JawRight: 1
      JawOpen: 1
      MouthClose: 1
      MouthFunnel: 1
      MouthPucker: 1
      MouthLeft: 1
      MouthRight: 1
      MouthSmileLeft: 1
      MouthSmileRight: 1
      MouthFrownLeft: 1
      MouthFrownRight: 1
      MouthDimpleLeft: 1
      MouthDimpleRight: 1
      MouthStretchLeft: 1
      MouthStretchRight: 1
      MouthRollLower: 1
      MouthRollUpper: 1
      MouthShrugLower: 1
      MouthShrugUpper: 1
      MouthPressLeft: 1
      MouthPressRight: 1
      MouthLowerDownLeft: 1
      MouthLowerDownRight: 1
      MouthUpperUpLeft: 1
      MouthUpperUpRight: 1
      BrowDownLeft: 1
      BrowDownRight: 1
      BrowInnerUp: 1
      BrowOuterUpLeft: 1
      BrowOuterUpRight: 1
      CheekPuff: 1
      CheekSquintLeft: 1
      CheekSquintRight: 1
      NoseSneerLeft: 1
      NoseSneerRight: 1
      TongueOut: 0
      TongueTipUp: 1
      TongueTipDown: 1
      TongueTipLeft: 1
      TongueTipRight: 1
      TongueRollUp: 1
      TongueRollDown: 1
      TongueRollLeft: 1
      TongueRollRight: 1
      TongueUp: 1
      TongueDown: 1
      TongueLeft: 1
      TongueRight: 1
      TongueIn: 1
      TongueStretch: 1
      TongueWide: 1
      TongueNarrow: 1

    cancel_poses: # Specifies which blendshapes are cancelled for each pose (for more details, see the documentation for blendshape_params)
      EyeBlinkLeft: -1
      EyeLookDownLeft: -1
      EyeLookInLeft: -1
      EyeLookOutLeft: -1
      EyeLookUpLeft: -1
      EyeSquintLeft: -1
      EyeWideLeft: -1
      EyeBlinkRight: -1
      EyeLookDownRight: -1
      EyeLookInRight: -1
      EyeLookOutRight: -1
      EyeLookUpRight: -1
      EyeSquintRight: -1
      EyeWideRight: -1
      JawForward: -1
      JawLeft: -1
      JawRight: -1
      JawOpen: -1
      MouthClose: -1
      MouthFunnel: -1
      MouthPucker: -1
      MouthLeft: -1
      MouthRight: -1
      MouthSmileLeft: -1
      MouthSmileRight: -1
      MouthFrownLeft: -1
      MouthFrownRight: -1
      MouthDimpleLeft: -1
      MouthDimpleRight: -1
      MouthStretchLeft: -1
      MouthStretchRight: -1
      MouthRollLower: -1
      MouthRollUpper: -1
      MouthShrugLower: -1
      MouthShrugUpper: -1
      MouthPressLeft: -1
      MouthPressRight: -1
      MouthLowerDownLeft: -1
      MouthLowerDownRight: -1
      MouthUpperUpLeft: -1
      MouthUpperUpRight: -1
      BrowDownLeft: -1
      BrowDownRight: -1
      BrowInnerUp: -1
      BrowOuterUpLeft: -1
      BrowOuterUpRight: -1
      CheekPuff: -1
      CheekSquintLeft: -1
      CheekSquintRight: -1
      NoseSneerLeft: -1
      NoseSneerRight: -1
      TongueOut: -1
      TongueTipUp: -1
      TongueTipDown: -1
      TongueTipLeft: -1
      TongueTipRight: -1
      TongueRollUp: -1
      TongueRollDown: -1
      TongueRollLeft: -1
      TongueRollRight: -1
      TongueUp: -1
      TongueDown: -1
      TongueLeft: -1
      TongueRight: -1
      TongueIn: -1
      TongueStretch: -1
      TongueWide: -1
      TongueNarrow: -1

    symmetry_poses: # Specifies which blendshapes are symmetrical for each pose (for more details, see the documentation for blendshape_params)
      EyeBlinkLeft: 0
      EyeLookDownLeft: -1
      EyeLookInLeft: -1
      EyeLookOutLeft: -1
      EyeLookUpLeft: -1
      EyeSquintLeft: -1
      EyeWideLeft: 1
      EyeBlinkRight: 0
      EyeLookDownRight: -1
      EyeLookInRight: -1
      EyeLookOutRight: -1
      EyeLookUpRight: -1
      EyeSquintRight: -1
      EyeWideRight: 1
      JawForward: -1
      JawLeft: -1
      JawRight: -1
      JawOpen: -1
      MouthClose: -1
      MouthFunnel: -1
      MouthPucker: -1
      MouthLeft: -1
      MouthRight: -1
      MouthSmileLeft: 2
      MouthSmileRight: 2
      MouthFrownLeft: 3
      MouthFrownRight: 3
      MouthDimpleLeft: 4
      MouthDimpleRight: 4
      MouthStretchLeft: 5
      MouthStretchRight: 5
      MouthRollLower: -1
      MouthRollUpper: -1
      MouthShrugLower: -1
      MouthShrugUpper: -1
      MouthPressLeft: 6
      MouthPressRight: 6
      MouthLowerDownLeft: 7
      MouthLowerDownRight: 7
      MouthUpperUpLeft: 8
      MouthUpperUpRight: 8
      BrowDownLeft: 10
      BrowDownRight: 10
      BrowInnerUp: -1
      BrowOuterUpLeft: 9
      BrowOuterUpRight: 9
      CheekPuff: -1
      CheekSquintLeft: 11
      CheekSquintRight: 11
      NoseSneerLeft: 12
      NoseSneerRight: 12
      TongueOut: -1
      TongueTipUp: -1
      TongueTipDown: -1
      TongueTipLeft: -1
      TongueTipRight: -1
      TongueRollUp: -1
      TongueRollDown: -1
      TongueRollLeft: -1
      TongueRollRight: -1
      TongueUp: -1
      TongueDown: -1
      TongueLeft: -1
      TongueRight: -1
      TongueIn: -1
      TongueStretch: -1
      TongueWide: -1
      TongueNarrow: -1

If the A2F-3D service is already running and you need to update the config map you can:

1. Export the config map to a file, for example configmap.yaml.

$ sudo microk8s kubectl get configmap a2f-configs-cm -o yaml > configmap.yaml

2. Edit the file using your preferred text editor.

$ nano configmap.yaml

3. Apply the updated config map.

$ sudo microk8s kubectl apply -f configmap.yaml

4. Delete Audio2Face-3D pod to restart it

$ sudo microk8s kubectl delete pods <audio2face_pod_name>

The <audio2face_pod_name> can be found by running:

$ sudo microk8s kubectl get pods
NAME                                  READY   STATUS             RESTARTS       AGE
a2f-a2f-deployment-xxx-xxx            1/1     Running            0              1h

Here it is: a2f-a2f-deployment-xxx-xxx

You can then install the Helm chart by executing the following command:

$ sudo microk8s helm install a2f-3d-nim audio2face-3d/

Check the pod status:

$ sudo microk8s kubectl get pods

Note

Pod stuck in Pending? If the pod is stuck with Insufficient nvidia.com/gpu, the GPU Operator may not be detecting your GPUs. Check the GPU Operator pod status from Step 1 and see Kubernetes Pod Stuck in Pending Status for solutions.

3. Optional - Expose the pod IP publicly

You need to expose the pod ID publicly in order to access the A2F-3D NIM from another machine. If this is not your usecase, you can skip this step.

The following yaml file defines the set of ports for communication between external systems and the a2f-a2f-deployment-a2f-service pod in the Kubernetes cluster. Read the comments to understand the file structure.

service_expose.yaml

apiVersion: v1
kind: Service
metadata:
  name: a2f-a2f-deployment-a2f-service
spec:
  type: NodePort
  ports:
    # Port 50010 serves A2F-3D gRPC service.
    - port: 50010 # The port used internally by the Kubernetes cluster for communication.
      targetPort: 50010 # The port on the application (pod) that the service forwards traffic to.
      nodePort: 30010 # The externally accessible port on the node that maps to the service. External traffic to NodeIP:30010 is routed to PodIP:50010.
    # Port 8000 serves A2F-3D NIM Http service.
    - port: 8000
      targetPort: 8000
      nodePort: 30020
    # Port 9464 serves A2F-3D Prometheus Endpoint.
    - port: 9464
      targetPort: 9464
      nodePort: 30030

You can adjust the external ports as needed, ensuring they fall within the 30000–32767 range. To apply the YAML file and expose the specified ports, run the following command:

$ sudo microk8s kubectl apply -f service_expose.yaml

Then you need to delete Audio2Face-3D pod to restart it.

$ sudo microk8s kubectl delete pods <audio2face_pod_name>

The <audio2face_pod_name> can be found by running:

$ sudo microk8s kubectl get pods
NAME                                  READY   STATUS             RESTARTS       AGE
a2f-a2f-deployment-xxx-xxx            1/1     Running            0              1h

Here it is: a2f-a2f-deployment-xxx-xxx

Verify the service is exposed with the expected NodePort mappings:

$ sudo microk8s kubectl get svc
NAME                             TYPE       CLUSTER-IP   EXTERNAL-IP   PORT(S)                                         AGE
a2f-a2f-deployment-a2f-service   NodePort   <ip>         <none>        50010:30010/TCP,8000:30020/TCP,9464:30030/TCP   12m

4. Check it is up and running

Execute the following command to check if the app was deployed successfully. Please note that the command output may vary.

$ sudo microk8s kubectl get pods
  NAME                                 READY   STATUS    RESTARTS   AGE
  a2f-a2f-deployment-94d89f979-v6n24   0/1     Init:0/1  0          53s

Note

If you see the status being ImagePullBackOff, then reconfigure the secrets. Make sure your NGC_API_KEY is valid.

Note

If you see the status being Pending, first diagnose the cause:

$ sudo microk8s kubectl describe pod <pod-name>

Look at the Events section for scheduling failure reasons. Common causes include:

• GPU not detected: Verify the GPU operator is running and detecting your GPU. Check sudo microk8s kubectl describe node | grep -A10 Allocatable for nvidia.com/gpu.

• Local path provisioner missing: Set it up with:

  $ curl https://raw.githubusercontent.com/rancher/local-path-provisioner/v0.0.23/deploy/local-path-storage.yaml | sed 's/^  name: local-path$/  name: mdx-local-path/g' | sudo microk8s kubectl apply -f -
  

• GPU Operator container-toolkit crash: The nvidia-container-toolkit-daemonset may be in CrashLoopBackOff due to a symlink creation error (affects various GPUs including L40, B200, RTX 5090). Apply the DISABLE_DEV_CHAR_SYMLINK_CREATION patch – see the warning in Step 1 or Kubernetes Pod Stuck in Pending Status.

See Kubernetes Pod Stuck in Pending Status in the troubleshooting guide for detailed solutions.

Wait some time until STATUS changes to Running.

To run a client application, you will need the IP and port to connect to. You can obtain this information by executing the following command:

$ sudo microk8s kubectl get svc
NAME                             TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                       AGE
a2f-a2f-deployment-a2f-service   ClusterIP   <ip>            <none>        50010/TCP,9464/TCP,8000/TCP   56m

If you exposed the pod publicly (Step 3), the service type will be NodePort and you can connect using the node’s IP address with port 30010 (e.g., <node-ip>:30010).

You can then try to interact with the Audio2Face-3D using for example the sample app provided.

Uninstall the app

To uninstall the app, run:

$ sudo microk8s helm uninstall a2f-3d-nim