Class InferenceServerClient

Inheritance Relationships

Derived Types

Class Documentation

class InferenceServerClient

The base class for InferenceServerClients.

Subclassed by nvidia::inferenceserver::client::InferenceServerGrpcClient, nvidia::inferenceserver::client::InferenceServerHttpClient

Public Types

using OnCompleteFn = std::function<void(InferResult *)>

Public Functions

InferenceServerClient(bool verbose)
virtual ~InferenceServerClient()
Error ClientInferStat(InferStat *infer_stat) const

Obtain the cumulative inference statistics of the client.

Return

Error object indicating success or failure.

Parameters
  • Returns: the InferStat object holding current statistics.

Protected Functions

Error UpdateInferStat(const RequestTimers &timer)

Protected Attributes

bool verbose_
std::thread worker_
std::mutex mutex_
std::condition_variable cv_
bool exiting_
InferStat infer_stat_