Struct InferenceMemoryFILInterfaceProxy#

Inheritance Relationships#

Base Type#

Struct Documentation#

struct InferenceMemoryFILInterfaceProxy : public morpheus::InferenceMemoryInterfaceProxy#

Interface proxy, used to insulate python bindings.

Public Static Functions

static std::shared_ptr<InferenceMemoryFIL> init(
TensorIndex count,
pybind11::object input__0,
pybind11::object seq_ids
)#

Create and initialize an InferenceMemoryFIL object, and return a shared pointer to the result.

Parameters:

  • count – : Message count in inference memory object

  • input__0 – : Inference input

  • seq_ids – : Ids used to index from an inference input to a message. Necessary since there can be more inference inputs than messages (i.e., if some messages get broken into multiple inference requests)

Returns:

std::shared_ptr<InferenceMemoryFIL>

static pybind11::object get_input__0(InferenceMemoryFIL &self)#

Returns the ‘input__0’ as cupy array.

Parameters:

self –

Returns:

pybind11::object

static void set_input__0(
InferenceMemoryFIL &self,
pybind11::object cupy_values
)#

Sets a tensor named ‘input__0’.

Parameters:

  • self –

  • cupy_values –

static pybind11::object get_seq_ids(InferenceMemoryFIL &self)#

Returns the ‘seq_ids’ as a cupy array.

Parameters:

self –

Returns:

pybind11::object

static void set_seq_ids(
InferenceMemoryFIL &self,
pybind11::object cupy_values
)#

Sets a tensor named ‘seq_ids’.

Parameters:

  • self –

  • cupy_values –