Struct InferStat

Struct Documentation

struct InferStat

Cumulative inference statistics.

Note

For GRPC protocol, ‘cumulative_send_time_ns’ represents the time for marshaling infer request. ‘cumulative_receive_time_ns’ represents the time for unmarshaling infer response.

Public Functions

InferStat()

Create a new InferStat object with zero-ed statistics.

Public Members

size_t completed_request_count

Total number of requests completed.

uint64_t cumulative_total_request_time_ns

Time from the request start until the response is completely received.

uint64_t cumulative_send_time_ns

Time from the request start until the last byte is sent.

uint64_t cumulative_receive_time_ns

Time from receiving first byte of the response until the response is completely received.