Basic Inference
-
Perform a health check on the gRPC endpoint.
-
Install
grpcurlfrom github.com/fullstorydev/grpcurl/releases.Example commands to run on Ubuntu:
-
Download the health checking proto:
-
Run the health check:
If the service is ready, you get a response similar to the following:
For using grpcurl with an SSL enabled server, avoid using the
--plaintextargument, and use--cacertwith a CA certificate,--keywith a private key, or--certwith a certificate file. For more details, refer togrpcurl --help. -
-
Download the Eye Contact Python client code by cloning the Clients repository (NVIDIA-Maxine/nim-clients):
-
Install the required dependencies:
Compile the Protos (Optional)
If you want to use the client code provided in the Clients repository (NVIDIA-Maxine/nim-clients), you can skip this step.
The proto files are available in the eye-contact/protos folder. You can compile them to generate client interfaces in your preferred programming language. For more details, refer to Supported languages in the gRPC documentation.
The following is an example of how to compile the protos for Python on Linux and Windows.
The grpcio version needed for compilation can be referred from requirements.txt
To compile protos on Linux:
To compile protos on Windows:
Input and Output
The input and output of the NVIDIA Eye Contact NIM are mp4 files. The input file must use the H.264 codec for video and can include audio. The output video file also uses the H.264 codec for video and contains the same audio as the input video file.
Videos with Variable Frame Rate (VFR) are not supported.
Input Modes
The NVIDIA Eye Contact NIM supports two distinct input processing modes based on the input file capabilities:
Transactional Mode (Default)
In transactional mode, the entire input video file must be received and processed as a complete unit by the NIM before returning results.
This mode is suitable for the following use cases:
- Processing of small video files, as these files are copied to NIM in entirety before inference can begin.
- Applications that can wait for complete processing before receiving output.
- Videos that are not optimized for streaming.
To run the NVIDIA Eye Contact NIM in transactional mode, simply run the NVIDIA Eye Contact sample client without any additional flags:
Go to the scripts directory:
Send a gRPC request:
Streaming Mode
In streaming mode, the NIM can process incoming video frames incrementally as soon as frame information is available. The output frames are streamed back to the client immediately after processing.
This is the preferred mode for the following use cases:
- Streamable video inputs.
- Applications that do not want to wait until the entire file is completely uploaded.
- Large video files that can benefit from incremental processing and thereby not constrained by the disk space on the server.
The --streaming option works with streamable videos where metadata is positioned at the beginning of the file. Videos that are not streamable can be easily converted to a streamable format.
To make any video streamable, use FFmpeg with the following command:
You can then specify the streamable video as input to the NIM by using the --input parameter and enabling the --streaming option.
The first inference is not indicative of the model’s actual performance because it includes the time taken by the Triton Inference Server to load the models in addition to the time required to process the inference request.
Usage for Preview API Request
To view details of command-line arguments, run this command:
You get a response similar to the following. All parameters are optional.
Example Commands
Basic inference with default settings:
Using streaming mode for streamable videos:
Advanced configuration example:
Python Notebook
For an interactive experience and to explore all feature parameters, we provide a comprehensive Python notebook that demonstrates the Eye Contact service capabilities.
The Python notebook is located at examples/maxine-eye-contact.ipynb in the eye-contact client folder.