Sample applications connecting to A2F-3D Authoring Microservice#

Note

This sample application section’s purpose is only to check the microservice endpoint.

Please use Maya-ACE to connect with the A2F-3D Authoring Microservice for interactive avatar tuning.

Follow this section to setup the connection.

Two sample applications are provided to demonstrate how to communicate with the Audio2Face-3D Authoring microservice.

Setup#

Clone the repository: NVIDIA/Audio2Face-3D-Samples

Go to early_access/a2f-3d-authoring-sample-app subfolder.

Inside you will find multiple files, most important ones being:

  • README.md - Explaining how to setup and use the sample applications.

  • client_local_deploy.py - Sample application to interact and do latency check with a local deployment of Audio2Face-3D Authoring Microservice. See client_local_deploy.py.

  • client_nvcf_deploy.py - Sample application to interact with Audio2Face-3D Authoring Microservice hosted on NVCF. See client_nvcf_deploy.py.

Follow the instructions in the downloaded README.md, the Requirements section.

Using a local deployment#

Follow the instructions in Container Deployment to have a local deployment of the Authoring Audio2Face-3D Microservice.

The client_local_deploy.py python application interacts with a local deployment of Audio2Face-3D Authoring service.

This application has 3 modes:

  • health_check: checks if the service is reachable.

  • data_capture: capture data from a full audio clip.

  • latency: record latency data of the Authoring service.

The application can be used as follows:

$ python3 client_local_deploy.py --help
$ python3 client_local_deploy.py health_check --url <url>
$ python3 client_local_deploy.py data_capture --url <url> --audio-clip <audio_file.wav> [--print-bs-names]
$ python3 client_local_deploy.py latency --url <url> --audio-clip <audio_file.wav> --number-requests <number_requests> --concurrent <number_parallel_connections>  [--print-bs-names]

  • <url> is the url of a running instance of Audio2Face-3D Authoring microservice. For example 0.0.0.0:50051.

  • <audio_file.wav> is an audio file in PCM 16-bit format.

  • <number_requests> and <number_parallel_connections> are 2 parameters which can be tweaked. Too high concurrency will increase latencies. Too low concurrency will increase processing time.

  • –print-bs-names: Optional. If enabled, the command will print out the names of the returned blendshapes.

Health checking

Assuming that Authoring microservice runs on 0.0.0.0:50051:

$ python3 client_local_deploy.py health_check --url 0.0.0.0:50051
  Service 0.0.0.0:50051 is ONLINE

If using an invalid url:

$ python3 client_local_deploy.py health_check --url 0.0.0.0:50052
  Error checking health: <_InactiveRpcError of RPC that terminated with:
    status = StatusCode.UNAVAILABLE
    details = "failed to connect to all addresses; last error: UNKNOWN: ipv4:0.0.0.0:50052: Failed to connect to remote host: Connection refused"
    debug_error_string = "UNKNOWN:Error received from peer  {grpc_message:"failed to connect to all addresses; last error: UNKNOWN: ipv4:0.0.0.0:50052: Failed to connect to remote host: Connection refused", grpc_status:14, created_time:"2024-09-06T17:58:55.301073978+02:00"}"
  >
  Service 0.0.0.0:50052 is OFFLINE

Process Audio Clip

This call will save blendshapes and emotions to a csv file similarly as A2F-3D sample app for an example Claire_neutral.wav audio file. You can provide any audio file in 16bits PCM wav format.

$ python3 client_local_deploy.py data_capture --url 0.0.0.0:50051 --audio-clip ../../example_audio/Claire_neutral.wav
  audio_clip_hash=5c8de20d275e53fcfceba0f39505f8b2b1fd960294518fbadf210d51dcc3a2e4
  Perform sequential requests for the full audio clip...
  3.73 seconds processed
  Saved results to output_blendshape.csv
  Saved results to output_emotions.csv

The results are saved in 2 files:

  • output_blendshape.csv: contains the blendshapes with their name, value and time codes.

  • output_emotions.csv: contains the emotions with their name, value and time codes.

The timecodes are relative to the beginning of the audio file.

Latency test

The following call will perform latency performance recording, 2 parameters can be tweaked, the number of requests and the number of parallel connections. Too high concurrency will increase latencies. Too low concurrency will increase processing time.

$ python3 client_local_deploy.py latency --url 0.0.0.0:50051 --audio-clip ../../example_audio/Claire_neutral.wav --number-requests 1000 --concurrent 10
  Computing data for 1000 requests with concurrency of 10
  audio_clip_hash=5c8de20d275e53fcfceba0f39505f8b2b1fd960294518fbadf210d51dcc3a2e4
  .......................................................................................
  .......................................................................................
  .......................................................................................
  Saved latency data in output_perf.txt
  Plotted latency data in output_latency.png

The results are saved in 2 files:

  • output_perf.txt: Contains statistics about the latency.

  • output_latency.png: Plots the distribution of latency of all requests as a histogram and marks the different percentiles.

Using NVCF#

The client_nvcf_deploy.py python application interacts with Audio2Face-3D Authoring service deployed on NVCF.

As in the local execution case, this application has 3 modes:

  • health_check: checks if the service is reachable.

  • data_capture: capture data from a full audio clip.

  • latency: record latency data of the Authoring service.

The application can be used as follows:

$ python3 client_nvcf_deploy.py --help
$ python3 client_nvcf_deploy.py health_check --function-id <function_id> --version-id <version_id> --apikey <api_key>
$ python3 client_nvcf_deploy.py data_capture --function-id <function_id> --version-id <version_id> --apikey <api_key> --audio-clip <audio_file.wav> [--print-bs-names]
$ python3 client_nvcf_deploy.py latency --function-id <function_id> --version-id <version_id> --apikey <api_key> --audio-clip <audio_file.wav> --number-requests <number_requests> --concurrent <number_parallel_connections> [--print-bs-names]

  • <api_key> is the run key provided to you by your NVIDIA account manager.

  • <function_id> and <version_id> are used to access the API function

  • <audio_file.wav> is an audio file in PCM 16-bit format.

  • <number_requests> and <number_parallel_connections> are 2 parameters which can be tweaked. Too high concurrency will increase latencies. Too low concurrency will increase processing time.

  • –print-bs-names: Optional. If enabled, the command will print out the names of the returned blendshapes.

Health checking

$ python3 client_nvcf_deploy.py health_check --function-id {FUNCTION_ID} --version-id {VERSION_ID} --apikey {API_KEY}
  NVCF Service is ONLINE

Process Audio Clip

This call will save blendshapes and emotions to a csv file similarly as A2F-3D sample app for an example Claire_neutral.wav audio file. You can provide any audio file in 16bits PCM wav format.

$ python3 client_nvcf_deploy.py data_capture --function-id {FUNCTION_ID} --version-id {VERSION_ID} --apikey {API_KEY} --audio-clip ../../example_audio/Claire_neutral.wav

The results are saved in 2 files:

  • output_blendshape.csv: contains the blendshapes with their name, value and time codes.

  • output_emotions.csv: contains the emotions with their name, value and time codes.

The timecodes are relative to the beginning of the audio file.

Latency test

The following call will perform latency performance recording, 2 parameters can be tweaked, the number of requests and the number of parallel connections. Too high concurrency will increase latencies. Too low concurrency will increase processing time.

$ python3 client_nvcf_deploy.py latency --function-id {FUNCTION_ID} --version-id {VERSION_ID} --apikey {API_KEY} --audio-clip ../../example_audio/Claire_neutral.wav --number-requests 1000 --concurrent 10
  Computing data for 1000 requests with concurrency of 10
  audio_clip_hash=5c8de20d275e53fcfceba0f39505f8b2b1fd960294518fbadf210d51dcc3a2e4
  .......................................................................................
  .......................................................................................
  .......................................................................................
  Saved latency data in output_perf.txt
  Plotted latency data in output_latency.png

The results are saved in 2 files:

  • output_perf.txt: Contains statistics about the latency.

  • output_latency.png: Plots the distribution of latency of all requests as a histogram and marks the different percentiles.

On this page