Struct InferenceMemoryFILInterfaceProxy

• Defined in File inference_memory_fil.hpp

Base Type

• public morpheus::InferenceMemoryInterfaceProxy (Struct InferenceMemoryInterfaceProxy)

struct InferenceMemoryFILInterfaceProxy : public morpheus::InferenceMemoryInterfaceProxy

Interface proxy, used to insulate python bindings.

Public Static Functions

static std::shared_ptr<InferenceMemoryFIL> init(TensorIndex count, pybind11::object input__0, pybind11::object seq_ids)

Create and initialize an InferenceMemoryFIL object, and return a shared pointer to the result.

Parameters

• count – : Message count in inference memory object

• input__0 – : Inference input

• seq_ids – : Ids used to index from an inference input to a message. Necessary since there can be more inference inputs than messages (i.e., if some messages get broken into multiple inference requests)

Returns

std::shared_ptr<InferenceMemoryFIL>

static pybind11::object get_input__0(InferenceMemoryFIL &self)

Returns the ‘input__0’ as cupy array.

Parameters

self –

Returns

pybind11::object

static void set_input__0(InferenceMemoryFIL &self, pybind11::object cupy_values)

Sets a tensor named ‘input__0’.

Parameters

• self –

• cupy_values –

static pybind11::object get_seq_ids(InferenceMemoryFIL &self)

Returns the ‘seq_ids’ as a cupy array.

Parameters

self –

Returns

pybind11::object

static void set_seq_ids(InferenceMemoryFIL &self, pybind11::object cupy_values)

Sets a tensor named ‘seq_ids’.

Parameters

• self –

• cupy_values –