Class CustomInstance

Class Documentation

class CustomInstance

Base class for custom backend instances.

CustomInstance is responsible for the state of the instance, provide a C++ wrapper around the C-API, and provide common helper functions that can be used for initialization and execution.

Public Functions

virtual ~CustomInstance()
virtual int Execute(const uint32_t payload_cnt, CustomPayload *payloads, CustomGetNextInputFn_t input_fn, CustomGetOutputFn_t output_fn) = 0

Execute the custom instance.

Parameters

  • payload_cnt: The number of payloads to execute.

  • payloads: The payloads to execute.

  • input_fn: The callback function to get tensor input (see CustomGetNextInputFn_t).

  • output_fn: The callback function to get buffer for tensor output (see CustomGetOutputFn_t).

const char *ErrorString(uint32_t error) const

Get the string for an error code.

/param error Error code returned by a CustomInstance function /return Descriptive error message for a specific error code.

Public Static Functions

static int Create(CustomInstance **instance, const std::string &name, const ModelConfig &model_config, int gpu_device, const CustomInitializeData *data)

Create a custom backend.

This static method is declared here, but must be defined in the child custom class. This create method is used CustomInitialization C-API method.

Return

Error code indicating success or the type of failure

Parameters

  • instance: A return pointer for the custom instance object created by this method

  • name: The name of the custom instance

  • model_config: The model configuration

  • gpu_device: The GPU device ID

Protected Functions

CustomInstance(const std::string &instance_name, const ModelConfig &model_config, int gpu_device)

Base constructor for CustomInstance.

Return

Error code indicating success or the type of failure

Parameters

  • name: The name of the custom instance

  • model_config: The model configuration

  • gpu_device: The GPU device ID

int RegisterError(const std::string &error_message)

Register a custom error and error message.

Return

The unique error code registered to this error message

Parameters

  • error_message: A descriptive error message string

Protected Attributes

const std::string instance_name_

The name of this backend instance.

const ModelConfig model_config_

The model configuration.

const int gpu_device_

The GPU device ID to execute on or CUSTOM_NO_GPU_DEVICE if should execute on CPU.