Struct InferContext::Stat

Nested Relationships

This struct is a nested type of Class InferContext.

Struct Documentation

struct Stat

Cumulative statistic of the InferContext.

Note
For GRPC protocol, ‘cumulative_send_time_ns’ represents the time for marshaling infer request. ‘cumulative_receive_time_ns’ represents the time for unmarshaling infer response.

Public Functions

Stat()

Create a new Stat object with zero-ed statistics.

Public Members

size_t completed_request_count

Total number of requests completed.

uint64_t cumulative_total_request_time_ns

Time from the request start until the response is completely received.

uint64_t cumulative_send_time_ns

Time from the request start until the last byte is sent.

uint64_t cumulative_receive_time_ns

Time from receiving first byte of the response until the response is completely received.