Experimental Client

A beta version of the Python client library is available that uses the new GRPC protocol based on the community standard inference protocols that have been proposed by the KFServing project. This Python library does not yet expose all capabilities of Triton Server but will be enhanced over time. A version of the library that uses the HTTP/REST protocol will also be provided in the future.

To try the new client, first get the Python client library that uses the new protocol following directions in Getting the Client Libraries.

The Python GRPC client interface documentation is available at src/clients/python/experimental_api_v2/library/grpcclient.py and in the API Reference.

Examples are available in src/clients/python/experimental_api_v2/examples. The examples that start with grpc_ use the protoc compiler to generate the GRPC client stub. The examples that start with simple_ use the Python GRPC client library.

For Triton Server to support the new GRPC protocol it must be run with the --api-version=2 flag.