HTTP/REST and GRPC Protocol

This directory contains documents related to the HTTP/REST and GRPC protocols used by Triton. Triton uses the KServe community standard inference protocols plus several extensions that are defined in the following documents:

• Binary tensor data extension

• Classification extension

• Schedule policy extension

• Sequence extension

• Shared-memory extension

• Model configuration extension

• Model repository extension

• Statistics extension

• Trace extension

• Logging extension

• Parameters extension

Note that some extensions introduce new fields onto the inference protocols, and the other extensions define new protocols that Triton follows, please refer to the extension documents for detail.

For the GRPC protocol, the protobuf specification is also available. In addition, you can find the GRPC health checking protocol protobuf specification here.

You can configure the Triton endpoints, which implement the protocols, to restrict access to some protocols and to control network settings, please refer to protocol customization guide for detail.