Class ModelControlContext¶
Defined in File request.h
Class Documentation¶
-
class
ModelControlContext¶ A ModelControlContext object is used to control the model loading / unloading on the inference server.
Once created a ModelControlContext object can be used repeatedly.
A ModelControlContext object can use either HTTP protocol or GRPC protocol depending on the Create function (ModelControlHttpContext::Create or ModelControlGrpcContext::Create). For example:
std::unique_ptr<ModelControlContext> ctx; ModelControlGrpcContext::Create(&ctx, "localhost:8000"); std::string model_name = "model"; ctx->Load(model_name); ... ctx->Unload(model_name); ...
Public Functions
-
virtual
~ModelControlContext() = 0¶
-
virtual Error
Load(const std::string &model_name) = 0¶ Load a model on the inference server.
If the model is already loaded, it will be reloaded to use the latest configuration.
- Return
Error object indicating success or failure.
- Parameters
model_name: The name of the model to be loaded.
-
virtual Error
Unload(const std::string &model_name) = 0¶ Unload a model from the inference server.
Unloading a model that is not loaded on server has no affect and success code will be returned.
- Return
Error object indicating success or failure.
- Parameters
model_name: The name of the model to be unloaded.
-
virtual