Advanced Usage
Model Caching
When the container launches for the first time, it downloads the required models from NGC. To avoid downloading the models on subsequent runs, you can cache them locally by using a cache directory:
For more information about MANIFEST_PROFILE_ID, refer to Model Manifest Profiles.
SSL Enablement
Active Speaker Detection NIM provides an SSL mode to ensure secure communication between clients and the server by encrypting data in transit. To enable SSL, you must provide the path to the SSL certificate and key files in the container. The following example shows how to do this:
NIM_SSL_MODE can be set to mtls, tls, or disabled. If set to mtls, the container uses mutual TLS authentication. If set to tls, the container uses TLS authentication.
For more information, refer to Environment Variables.
Be sure to verify the permissions of the SSL certificate and key files on the host machine. The container cannot access the files if they are not readable by the user running the container.
Multiple Concurrent Inputs
To run the server in multi-input concurrent mode, set the environment variable NV_AI4M_MAX_CONCURRENCY_PER_GPU to an integer greater than 1 in the server container. The server then accepts as many concurrent inputs per GPU as specified by the NV_AI4M_MAX_CONCURRENCY_PER_GPU variable.
Because Triton distributes the workload equally across all GPUs, the total number of concurrent inputs supported by the server is the number of GPUs multiplied by NV_AI4M_MAX_CONCURRENCY_PER_GPU.
The Active Speaker Detection NIM uses NVENC/NVDEC hardware acceleration for video decoding.
- GPUs without NVDEC hardware support might not be supported.
- Some GPUs support only a limited number of concurrent NVDEC sessions, which means the NIM can process only that same number of concurrent inputs on those GPUs.
For details, refer to the Video Encode and Decode Support Matrix.
If incoming requests to the NIM exceed the GPU’s maximum concurrent decode limit, the processing might fail.
Maximum Speakers Configuration
The NV_AI4M_ASD_MAX_SPEAKERS environment variable controls the maximum number of audio speaker streams that the Active Speaker Detection NIM can handle simultaneously. This setting affects both the gRPC service and the Triton backend.
- Default: 5
- Range: 1 to 5
To configure the maximum number of speakers:
The NV_AI4M_ASD_MAX_SPEAKERS value must match the number of speaker streams expected in your use case.
Audio Source Configuration
The Active Speaker Detection NIM supports two modes for audio input feed, configured by the client at request time:
- Separate Stream (
AUDIO_SOURCE_CONFIG_SEPARATE_STREAM): The client provides a separate audio file alongside the video. This is the default behavior. Use the--audio-inputargument in the client to specify the audio file path. - Embedded in Video (
AUDIO_SOURCE_CONFIG_EMBEDDED_IN_VIDEO): Audio is demuxed from the supplied video container. No separate audio file is needed. Use the--skip-audioflag in the client.
Separate Audio Stream
Embedded Audio
Speaker Detection Threshold
The speaker_detection_threshold field in the ActiveSpeakerDetectionConfig proto controls the minimum confidence score required to classify a detected face as actively speaking.
- Valid range:
(0, 1)open interval (exclusive of both endpoints). - Proto field:
optional float speaker_detection_thresholdinActiveSpeakerDetectionConfig.
A lower threshold is more permissive, resulting in more faces being classified as speaking. A higher threshold is more strict, resulting in fewer faces classified as speaking. The face-detection bounding boxes themselves are not affected by this threshold—only the is_speaking classification changes.
To set the threshold, include the speaker_detection_threshold field in the ActiveSpeakerDetectionConfig message sent as the first request in the gRPC stream:
If the field is omitted or set to a value less than or equal to zero, the server uses the default threshold value configured. Values outside the valid range (such as 1.5 or -0.5) are rejected by the server with an INVALID_ARGUMENT gRPC error.
For more information about Maxine NIM clients, refer to the GitHub repository NVIDIA-Maxine/nim-clients.