Class InferenceServerClient

• Defined in File common.h

Inheritance Relationships

Derived Types

• public nvidia::inferenceserver::client::InferenceServerGrpcClient (Class InferenceServerGrpcClient)

• public nvidia::inferenceserver::client::InferenceServerHttpClient (Class InferenceServerHttpClient)

Class Documentation

class InferenceServerClient

The base class for InferenceServerClients.

Subclassed by nvidia::inferenceserver::client::InferenceServerGrpcClient, nvidia::inferenceserver::client::InferenceServerHttpClient

Public Types

using OnCompleteFn = std::function<void(InferResult *)>

Public Functions

InferenceServerClient(bool verbose)

virtual ~InferenceServerClient()

Error ClientInferStat(InferStat *infer_stat) const

Obtain the cumulative inference statistics of the client.

Return

Error object indicating success or failure.

Parameters

• Returns: the InferStat object holding current statistics.

Protected Functions

Error UpdateInferStat(const RequestTimers &timer)

Protected Attributes

bool verbose_

std::thread worker_

std::mutex mutex_

std::condition_variable cv_

bool exiting_

InferStat infer_stat_