Sample applications connecting to A2F Authoring Microservice#

Note

This sample application section’s purpose is only to check the microservice endpoint.

Please use Maya-ACE to connect with the A2F Authoring Microservice for interactive avatar tuning.

Follow this section to setup the connection.

Two sample applications are provided to demonstrate how to communicate with the Audio2Face Authoring microservice.

Setup#

Download the NGC resource and unzip:

$ ngc registry resource download-version "eevaigoeixww/ace-ea/a2f-authoring-sample-code:0.2.2"
$ cd a2f-authoring-sample-code_v0.2.2
$ unzip authoring_script.zip
$ cd authoring_script

Inside you will find multiple files, most importat ones being:

  • README.md - Explaining how to setup and use the sample applications.

  • client_local_deploy.py - Sample application to interact and do latency check with a local deployment of Audio2Face Authoring Microservice. See client_local_deploy.py.

  • client_nvcf_deploy.py - Sample application to interact with Audio2Face Authoring Microservice hosted on NVCF. See client_nvcf_deploy.py.

Note

To find the checksum of authoring_script.zip, run:

$ sha256sum authoring_script.zip

You should get:

5ad1e0e3b43f287fc0de316648b4f833ab38b09c90debe94cc875410961c804d  authoring_script.zip

Follow the instructions in the downloaded README.md, the Requirements section.

Using a local deployment#

Follow the instructions in Container Deployment to have a local deployment of the Authoring Audio2Face Microservice.

The client_local_deploy.py python application interacts with a local deployment of Audio2Face Authoring service.

This application has 3 modes:

  • health_check: checks if the service is reachable.

  • data_capture: capture data from a full audio clip.

  • latency: record latency data of the Authoring service.

The application can be used as follows:

$ python3 client_local_deploy.py --help
$ python3 client_local_deploy.py health_check <url>
$ python3 client_local_deploy.py data_capture <url> <audio_file.wav>
$ python3 client_local_deploy.py latency <url> <audio_file.wav> <number_requests> <number_parallel_connections>

  • <url> is the url of a running instance of Audio2Face Authoring microservice. For example 0.0.0.0:50051.

  • <audio_file.wav> is an audio file in PCM 16-bit format.

  • <number_requests> and <number_parallel_connections> are 2 parameters which can be tweaked. Too high concurrency will increase latencies. Too low concurrency will increase processing time.

Health checking

Assuming that Authoring microservice runs on 0.0.0.0:50051:

$ python3 client_local_deploy.py health_check 0.0.0.0:50051
  Service 0.0.0.0:50051 is ONLINE

If using an invalid url:

$ python3 client_local_deploy.py health_check 0.0.0.0:50052
  Error checking health: <_InactiveRpcError of RPC that terminated with:
    status = StatusCode.UNAVAILABLE
    details = "failed to connect to all addresses; last error: UNKNOWN: ipv4:0.0.0.0:50052: Failed to connect to remote host: Connection refused"
    debug_error_string = "UNKNOWN:Error received from peer  {grpc_message:"failed to connect to all addresses; last error: UNKNOWN: ipv4:0.0.0.0:50052: Failed to connect to remote host: Connection refused", grpc_status:14, created_time:"2024-09-06T17:58:55.301073978+02:00"}"
  >
  Service 0.0.0.0:50052 is OFFLINE

Process Audio Clip

This call will save blendshapes and emotions to a csv file similarly as A2F sample app for an example Female_Neutral_16khz.wav audio file. You can provide any audio file in 16bits PCM wav format.

$ python3 client_local_deploy.py data_capture 0.0.0.0:50051 Female_Neutral_16khz.wav
  audio_clip_hash=d6f8005c54cd651dbe2ea041605d5ea37447dc4fe555a56ded01b12e70313f36
  Perform sequential requests for the full audio clip...
  3.73 seconds processed
  Saved results to output_blendshape.csv
  Saved results to output_emotions.csv

The results are saved in 2 files:

  • output_blendshape.csv: contains the blendshapes with their name, value and time codes.

  • output_emotions.csv: contains the emotions with their name, value and time codes.

The timecodes are relative to the beginning of the audio file.

Latency test

The following call will perform latency performance recording, 2 parameters can be tweaked, the number of requests and the number of parallel connections. Too high concurrency will increase latencies. Too low concurrency will increase processing time.

$ python3 client_local_deploy.py latency 0.0.0.0:50051 Female_Neutral_16khz.wav 1000 10
  Computing data for 1000 requests with concurrency of 10
  audio_clip_hash=d6f8005c54cd651dbe2ea041605d5ea37447dc4fe555a56ded01b12e70313f36
  .......................................................................................
  .......................................................................................
  .......................................................................................
  Saved latency data in output_perf.txt
  Plotted latency data in output_latency.png

The results are saved in 2 files:

  • output_perf.txt: Contains statistics about the latency.

  • output_latency.png: Plots the distribution of latency of all requests as a histogram and marks the different percentiles.

Using NVCF#

The client_nvcf_deploy.py python application interacts with Audio2Face Authoring service deployed on NVCF.

As in the local execution case, this application has 3 modes:

  • health_check: checks if the service is reachable.

  • data_capture: capture data from a full audio clip.

  • latency: record latency data of the Authoring service.

The application can be used as follows:

$ python3 client_nvcf_deploy.py --help
$ python3 client_nvcf_deploy.py health_check --function-id <function_id> --version-id <version_id> --apikey <api_key>
$ python3 client_nvcf_deploy.py data_capture --function-id <function_id> --version-id <version_id> --apikey <api_key> <audio_file.wav>
$ python3 client_nvcf_deploy.py latency --function-id <function_id> --version-id <version_id> --apikey <api_key> <audio_file.wav> <number_requests> <number_parallel_connections>

  • <api_key> is the run key provided to you by your NVIDIA account manager.

  • <function_id> and <version_id> are used to access the API function

  • <audio_file.wav> is an audio file in PCM 16-bit format.

  • <number_requests> and <number_parallel_connections> are 2 parameters which can be tweaked. Too high concurrency will increase latencies. Too low concurrency will increase processing time.

Health checking

$ python3 client_nvcf_deploy.py health_check --function-id {FUNCTION_ID} --version-id {VERSION_ID} --apikey {API_KEY}
  NVCF Service is ONLINE

Process Audio Clip

This call will save blendshapes and emotions to a csv file similarly as A2F sample app for an example Female_Neutral_16khz.wav audio file. You can provide any audio file in 16bits PCM wav format.

$ python3 client_nvcf_deploy.py data_capture --function-id {FUNCTION_ID} --version-id {VERSION_ID} --apikey {API_KEY} Female_Neutral_16khz.wav

The results are saved in 2 files:

  • output_blendshape.csv: contains the blendshapes with their name, value and time codes.

  • output_emotions.csv: contains the emotions with their name, value and time codes.

The timecodes are relative to the beginning of the audio file.

Latency test

The following call will perform latency performance recording, 2 parameters can be tweaked, the number of requests and the number of parallel connections. Too high concurrency will increase latencies. Too low concurrency will increase processing time.

$ python3 client_nvcf_deploy.py latency --function-id {FUNCTION_ID} --version-id {VERSION_ID} --apikey {API_KEY} Female_Neutral_16khz.wav 1000 10
  Computing data for 1000 requests with concurrency of 10
  audio_clip_hash=d6f8005c54cd651dbe2ea041605d5ea37447dc4fe555a56ded01b12e70313f36
  .......................................................................................
  .......................................................................................
  .......................................................................................
  Saved latency data in output_perf.txt
  Plotted latency data in output_latency.png

The results are saved in 2 files:

  • output_perf.txt: Contains statistics about the latency.

  • output_latency.png: Plots the distribution of latency of all requests as a histogram and marks the different percentiles.

On this page