Troubleshooting#

Known issues#

  • Non-verbal human sounds (E.g. “hmmmm…”) and non-human audio do not translate well into facial expressions, resulting in random lip motions. This is an area identified for future improvement.

  • With certain driver versions running on H100 HBM3, the following error is observed for Audio2Emotion inference. This is under investigation. Meanwhile please be sure to use recommended driver and CUDA versions.

    [A2E SDK] [ERROR] [A2E SDK] IExecutionContext::setInputShape: Error Code 3: API Usage Error (Parameter check failed, condition: satisfyProfile. Set dimension [14,30000] for tensor input_values does not satisfy any optimization profiles. Valid range for profile 0: [1,30000]..[10,30000].)
[A2E SDK] [ERROR] Set binding dimensions failed.

Q&A#

The Avatar doesn’t close its mouth at the end of the audio clip. How can I fix it?

There are multiple requirements for the mouth to close, we suggest you to try in this order:

  • The audio clip need to end with some silence, you need to append some silence at the end if that’s not the case.

  • The emotions must be reset to neutral, you need to send a neutral emotion (E.g. send an empty map or joy=0) and some silent audio if that’s not the case.

Note

If you set the emotion to neutral at the very end of the audio clip, you will not see any change because some smoothing applies in Audio2Face-3D to make the emotion change less abrupt. So you will need to push some silence audio at the end.

E.g.: In the Audio2Face-3D NIM at the end of an audio clip, we set all emotions to 0 and send 1.5 seconds of silence.

If preferred_emotion_strength from EmotionPostProcessingParameters is close to zero then the neutral emotion will not be taken into account. You might need to increase that value to allow the mouth to close.

  • The face parameter selected might prevent the mouth from closing, you can try updating them to make sure the mouth closes.

  • The blend shape multipliers and offset might prevent the mouth from closing, you can try updating them to make sure the mouth closes.

gRPC Response Status Code#

Users will receive a gRPC error message with the code nvidia_ace::status::v1::Status_Code_ERROR in the following cases, among others:

  • The gRPC request is missing a data field or audio header.

  • The audio header is invalid (E.g. unsupported audio channels, sample rate, bit depth).

  • The audio buffer is empty in the data field.

  • The id (request_id, stream_id, target_object_id) exceeds maxLenUUID or contains unsupported characters.

  • The number of concurrent streams has reached streamNumber limit.

  • The FPS is too low.

Otherwise, the gRPC request will return with the code nvidia_ace::status::v1::Status_Code_SUCCESS.

Audio2Face-3D NIM Public API#

Here are some common errors you may encounter when accessing the Audio2Face-3D NIM Public API

grpc error messages

Potential Reasons

Possible Solutions

grpc_message:’failed to establish link to worker’, grpc_status:13

No server/instance available at the moment of requesting

The current deployment has reached it’s capacity with instance_count * max_occurrence settings.

grpc_message:’failed to open stateful work request: rpc error: code = Unauthenticated desc = invalid response from UAM’, grpc_status:13

API key is invalid

Check the API Key, and generate a new one if this continue to happen.

grpc_message:’failed to open stateful work request: rpc error: code = Internal desc = There was a server error trying to handle an exception’, grpc_status:13

API key does not have enough credit / API key doesn’t have access to this function

Check the function is a public function, or if your API key is related to an account with access to the function.

GStreamer-WARNING#

When you start the service, you might encounter warnings labeled as GStreamer-WARNING. These warnings occur because some libraries are missing from the container. However, they are safe to ignore, as these libraries are not used by Audio2Face-3D.

On this page