Wrapper class for creating Triton Inference Server instance.
Definition at line 533 of file infer_trtis_server.h.
◆ TrtISServer()
Constructor.
Save the model repository configuration settings.
- Parameters
-
◆ ~TrtISServer()
nvdsinferserver::TrtISServer::~TrtISServer |
( |
| ) |
|
Destructor.
Stops the Triton server if the server handle is valid.
◆ createRequest()
Create and initializes a new inference request.
- Parameters
-
[in] | model | Name of the model. |
[in] | version | Version of the model. |
[in] | inputs | Array of input batch buffers. |
[in] | outputs | List of requested output names. |
[in] | clasList | Triton Classification parameters, if any. |
- Returns
- Pointer to the request instance upon success, null pointer on failure.
◆ getInstance()
Get a new or existing instance of the Triton Inference Server.
This function checks if an instance of Triton is present. If it doesn't exist, a new one is instantiated using the provided model repository configuration. If a Triton instance is already running and repository configuration is provided, the existing and new configuration is checked to be same otherwise a null pointer is returned.
- Parameters
-
[in] | repo | Model repository configuration, can be null when the Triton server is already instantiated. |
- Returns
- Pointer to the Triton server instance.
◆ getModelConfig()
NvDsInferStatus nvdsinferserver::TrtISServer::getModelConfig |
( |
const std::string & |
model, |
|
|
int64_t |
version, |
|
|
ni::ModelConfig & |
config |
|
) |
| |
Get the model configuration for the specified model.
- Parameters
-
[in] | model | Name of the model. |
[in] | version | Version of the model. |
[out] | config | ModelConfig protobuf message to be populated. |
- Returns
- NVDSINFER_SUCCESS or NVDSINFER_TRITON_ERROR.
◆ getRepoSettings()
◆ inferAsync()
Submit a request for asynchronous inference.
This functions sets the release and response callback functions for the request and then triggers the asynchronous inference.
- Parameters
-
[in] | request | Pointer to the request object. |
[in] | allocator | Pointer to the response allocator. |
[in] | done | Pointer to the function to be called after inference is done. |
- Returns
- NVDSINFER_SUCCESS or NVDSINFER_TRITON_ERROR.
◆ initialize()
Create a new instance of the Triton Inference Server.
This functions creates a TRITONSERVER_ServerOptions instance and updates it as per the repository configuration. These options are then used to create a new Triton instance and save the handle in m_Impl.
- Returns
- Error status.
◆ isModelReady()
bool nvdsinferserver::TrtISServer::isModelReady |
( |
const std::string & |
model, |
|
|
int64_t |
version |
|
) |
| |
Check if the server is ready for inference using specified model.
- Parameters
-
[in] | model | Name of the model. |
[in] | version | Version of the model. |
- Returns
- Boolean indicating readiness.
◆ isServerLive()
bool nvdsinferserver::TrtISServer::isServerLive |
( |
| ) |
|
Check if the server is live.
◆ isServerReady()
bool nvdsinferserver::TrtISServer::isServerReady |
( |
| ) |
|
Check if the server is ready.
◆ loadModel()
NvDsInferStatus nvdsinferserver::TrtISServer::loadModel |
( |
const std::string & |
modelName | ) |
|
Load or reload the specified model.
- Parameters
-
[in] | modelName | Name of the model |
- Returns
- NVDSINFER_SUCCESS or NVDSINFER_TRITON_ERROR.
◆ unloadModel()
NvDsInferStatus nvdsinferserver::TrtISServer::unloadModel |
( |
const std::string & |
modelName | ) |
|
Unload the specified model.
- Parameters
-
[in] | modelName | Name of the model. |
- Returns
- NVDSINFER_SUCCESS or NVDSINFER_TRITON_ERROR.
◆ TrtServerRequest
◆ TrtServerResponse
The documentation for this class was generated from the following file: