Basic Inference#

  1. Perform a health check on the gRPC endpoint.

    • Install grpcurl from github.com/fullstorydev/grpcurl/releases.

      Example commands to run on Ubuntu:

      $ wget https://github.com/fullstorydev/grpcurl/releases/download/v1.9.1/grpcurl_1.9.1_linux_amd64.deb
$ sudo dpkg -i grpcurl_1.9.1_linux_amd64.deb

    • Download the health checking proto:

      $ wget https://raw.githubusercontent.com/grpc/grpc/master/src/proto/grpc/health/v1/health.proto

    • Run the health check:

      $ grpcurl --plaintext --proto health.proto localhost:8001 grpc.health.v1.Health/Check

    • If the service is ready, you get a response similar to the following:

      { "status": "SERVING" }

    Note

    For using grpcurl with an SSL enabled server, avoid using the --plaintext argument, and use --cacert with a CA certificate, --key with a private key, or --cert with a certificate file. For more details, refer to grpcurl --help.

  2. Download the Eye Contact Python client code by cloning the gRPC Client repository:

    git clone https://github.com/NVIDIA-Maxine/nim-clients.git

// Go to the 'eye-contact' folder

cd nim-clients/eye-contact/

  3. Install the required dependencies:

    sudo apt-get install python3-pip
pip install -r requirements.txt

Compile the Protos (Optional)#

If you want to use the client code provided in the github Client repository, you can skip this step.

The proto files are available in the eye-contact/protos folder. You can compile them to generate client interfaces in your preferred programming language. For more details, refer to Supported languages in the gRPC documentation.

Here is an example of how to compile the protos for Python on Linux and Windows. The grpcio version needed for compilation can be referred from requirements.txt

To compile protos on Linux, run:

// Go to eye-contact/protos/linux folder
cd eye-contact/protos/linux

chmod +x compile_protos.sh
./compile_protos.sh

To compile protos on Windows, run:

// Go to eye-contact/protos/windows folder
cd eye-contact/protos/windows

./compile_protos.bat

Running Inference via Script#

You can use the sample client script in the Eye Contact GitHub repo to send a gRPC request to the hosted NIM server:

Go to the scripts directory

cd scripts

Run the command to send gRPC request

   python eye-contact.py --target <server_ip:port> \
      --input <input file path> \
      --output <output file path along with file name> \
      --ssl-mode <ssl mode value> \
      --ssl-key <ssl key file path> \
      --ssl-cert <ssl cert filepath> \
      --ssl-root-cert <ssl root cert filepath>

For more details on the command-line arguments, refer to the Feature Parameters section.

Note

The first inference is not indicative of the model’s actual performance because it includes the time taken by the Triton Inference Server to load the models in addition to the time required to process the inference request.

Usage for Preview API Request#

   python eye-contact.py --preview-mode \
      --target grpc.nvcf.nvidia.com:443 \
      --function-id <function_id> \
      --api-key $API_KEY_REQUIRED_IF_EXECUTING_OUTSIDE_NGC \
      --input <input_file_path> \
      --output <output_file_path>

To view details of command-line arguments, run this command:

python eye-contact.py -h

You will get a response similar to the following.

options:
-h, --help                        show this help message and exit
--preview-mode                    Flag to send request to preview NVCF server on https://build.nvidia.com/nvidia/eyecontact/api.
--ssl-mode {DISABLED,MTLS,TLS}    Flag to set SSL mode, default is DISABLED
--ssl-key SSL_KEY                 The path to ssl private key.
--ssl-cert SSL_CERT               The path to ssl certificate chain.
--ssl-root-cert SSL_ROOT_CERT     The path to ssl root certificate.
--target TARGET                   IP:port of gRPC service, when hosted locally. Use grpc.nvcf.nvidia.com:443 when hosted on NVCF.
--input INPUT                     The path to the input video file.
--output OUTPUT                   The path for the output video file.
--api-key API_KEY                 NGC API key required for authentication, utilized when using TRY API ignored otherwise
--function-id FUNCTION_ID         NVCF function ID for the service, utilized when using TRY API ignored otherwise

If the command-line arguments are not passed, the script takes the following default values:

  • target is 127.0.0.1:8001.

  • input is ../assets/sample_input.mp4.

  • output is output.mp4 in the current directory.

  • ssl-mode is DISABLED (no SSL).

  • ssl-key is ../ssl_key/ssl_key_client.pem. Used only if ssl-mode is MTLS.

  • ssl-cert is ../ssl_key/ssl_cert_client.pem. Used only if ssl-mode is MTLS.

  • ssl-root-cert is ../ssl_key/ssl_ca_cert.pem. Used only if ssl-mode is MTLS or TLS.

On this page