Class CustomInstance

Class Documentation

class CustomInstance

Base class for custom backend instances.

CustomInstance is responsible for the state of the instance, provide a C++ wrapper around the C-API, and provide common helper functions that can be used for initialization and execution.

Public Functions

virtual ~CustomInstance()
virtual int Execute(const uint32_t payload_cnt, CustomPayload *payloads, CustomGetNextInputFn_t input_fn, CustomGetOutputFn_t output_fn)

Execute the custom instance.

User should override this function if version 1 of the custom interface is used or the version is not specified.

Return

Error code indicating success or the type of failure

Parameters

  • payload_cnt: The number of payloads to execute.

  • payloads: The payloads to execute.

  • input_fn: The callback function to get tensor input (see CustomGetNextInputFn_t).

  • output_fn: The callback function to get buffer for tensor output (see CustomGetOutputFn_t).

virtual int Execute(const uint32_t payload_cnt, CustomPayload *payloads, CustomGetNextInputV2Fn_t input_fn, CustomGetOutputV2Fn_t output_fn)

Execute the custom instance.

User should override this function if version 2 of the custom interface is used.

Return

Error code indicating success or the type of failure

Parameters

  • payload_cnt: The number of payloads to execute.

  • payloads: The payloads to execute.

  • input_fn: The callback function to get tensor input (see CustomGetNextInputV2Fn_t).

  • output_fn: The callback function to get buffer for tensor output (see CustomGetOutputV2Fn_t).

const char *ErrorString(uint32_t error) const

Get the string for an error code.

/param error Error code returned by a CustomInstance function /return Descriptive error message for a specific error code.

Public Static Functions

static int Create(CustomInstance **instance, const std::string &name, const ModelConfig &model_config, int gpu_device, const CustomInitializeData *data)

Create a custom backend.

This static method is declared here, but must be defined in the child custom class. This create method is used CustomInitialization C-API method.

Return

Error code indicating success or the type of failure

Parameters

  • instance: A return pointer for the custom instance object created by this method. Note that the custom object pointer should be returned even in the case of failure, so that the error string associated to the custom backend can be retrieved.

  • name: The name of the custom instance

  • model_config: The model configuration

  • gpu_device: The GPU device ID

Protected Functions

CustomInstance(const std::string &instance_name, const ModelConfig &model_config, int gpu_device)

Base constructor for CustomInstance.

Return

Error code indicating success or the type of failure

Parameters

  • name: The name of the custom instance

  • model_config: The model configuration

  • gpu_device: The GPU device ID

int RegisterError(const std::string &error_message)

Register a custom error and error message.

Return

The unique error code registered to this error message

Parameters

  • error_message: A descriptive error message string

Protected Attributes

const std::string instance_name_

The name of this backend instance.

const ModelConfig model_config_

The model configuration.

const int gpu_device_

The GPU device ID to execute on or CUSTOM_NO_GPU_DEVICE if should execute on CPU.