Advanced Usage
This section provides a detailed breakdown of the inferencing script for more advanced users.
Ensure that the provided client software is in PYTHONPATH and run the following command to set up the client.
The NIM invocation uses bi-directional gRPC streaming. To generate the request data stream, define a Python generator function. This is also known as a Python iterator of form, a simple function that yields after a call. The yield returns a chunk to be streamed. The first item in the stream is used for configuration objects that sets the NVIDIA Maxine Eye Contact feature parameters.
The following parameters are available for use in this NIM:
temporal- (UINT32) Flag to control temporal filtering (default0xffffffff). When set to true, the landmark computation for eye contact is temporally optimized.detect_closure- (UINT32) Flag to toggle detection of eye closure and occlusion on/off. Value is either 0 or 1. (default 0).eye_size_sensitivity- (UINT32) Eye size sensitivity parameter, an integer value between 2 to 6 (default 3)enable_lookaway- (UINT32) Flag to toggle look away on/off. If set to on, the eyes are redirected to look away for a random period occasionally to avoid staring. Value is either 0 or 1.(default 0)lookaway_max_offset- (UINT32) Maximum value of gaze offset angle (degrees) during a random look away, an integer value between 1 to 10 (default 5)lookaway_interval_min- (UINT32) Minimum limit for the number of frames at which random look away occurs, an integer value between 1 to 600 (default 100)lookaway_interval_range- (UINT32) Range for picking the number of frames at which random look away occurs, an integer value between 1 to 600 (default 250)gaze_pitch_threshold_low- (FP32) Gaze pitch threshold (degrees) at which the redirection starts transitioning, away from camera towards estimated gaze, float between 10 to 35 (default 20)gaze_pitch_threshold_high- (FP32) Gaze pitch threshold (degrees) at which the redirection is equal to estimated gaze, float between 10 to 35 (default 30)gaze_yaw_threshold_low- (FP32) Gaze yaw threshold (degrees) at which the redirection starts transitioning, away from camera towards estimated gaze, float between 10 to 35 (default 20)gaze_yaw_threshold_high- (FP32) Gaze yaw threshold (degrees) at which the redirection the redirection is equal to estimated gaze, float between 10 to 35 (default 30)head_pitch_threshold_low- (FP32) Head pose pitch yaw threshold (degrees) at which the redirection start transitioning away from camera towards estimated gaze, float between 10 to 35 (default 15)head_pitch_threshold_high- (FP32) Head pose pitch yaw threshold (degrees) at which the redirection is equal to estimated gaze, float between 10 to 35 (default 25)head_yaw_threshold_low- (FP32) Head pose yaw threshold (degrees) at which the redirection starts transitioning, away from camera towards estimated gaze, float between 10 to 35 (default 25)head_yaw_threshold_high- (FP32) Head pose yaw threshold (degrees) at which the redirection is equal to estimated gaze, float between 10 to 35 (default 30)
Before invoking the NIM, define a function that handles the incoming stream and writes it to an output file. For more details on the technical aspects of this algorithm, please refer to the technical blog.
Now that we have the request generator and output iterator setup, connect to the NIM and invoke it.
The input file path is in the variable input_filepath and the output file is written at the location in the variable output_filepath.
params is a Python dictionary that holds the feature parameter name and value pairs.
If params is empty, default values are used.
Wait for a message confirming that the function invocation has completed before checking the output file. Fill in the correct host and port for your target in the code snippet below:
Multiple Concurrent Inputs
To run the server in multi-input concurrent mode, set the environment variable MAXINE_MAX_CONCURRENCY_PER_GPU to an integer greater than 1 in the server container. The server will then accept as many concurrent inputs per GPU as specified by the MAXINE_MAX_CONCURRENCY_PER_GPU variable.
Since Triton distributes the workload equally across all GPUs, if there are NUM_GPUS GPUs, the total number of concurrent inputs supported by the server will be NUM_GPUS * MAXINE_MAX_CONCURRENCY_PER_GPU.
Model Caching
When the container starts for the first time, it will download the required models from NGC. To avoid downloading the models on subsequent runs, you can cache them locally by using a cache directory: