Struct InferenceMemoryNLPInterfaceProxy

• Defined in File inference_memory_nlp.hpp

Base Type

• public morpheus::InferenceMemoryInterfaceProxy (Struct InferenceMemoryInterfaceProxy)

struct InferenceMemoryNLPInterfaceProxy : public morpheus::InferenceMemoryInterfaceProxy

Interface proxy, used to insulate python bindings.

Public Static Functions

static std::shared_ptr<InferenceMemoryNLP> init(TensorIndex count, pybind11::object input_ids, pybind11::object input_mask, pybind11::object seq_ids)

Create and initialize an InferenceMemoryNLP object, and return a shared pointer to the result.

Parameters

• count – : Message count in inference memory object

• input_ids – : The token-ids for each string padded with 0s to max_length

• input_mask – : The mask for token-ids result where corresponding positions identify valid token-id values

• seq_ids – : Ids used to index from an inference input to a message. Necessary since there can be more inference inputs than messages (i.e., if some messages get broken into multiple inference requests)

Returns

std::shared_ptr<InferenceMemoryNLP>

static pybind11::object get_input_ids(InferenceMemoryNLP &self)

: Returns token-ids for each string padded with 0s to max_length as python object

Parameters

self –

Throws

pybind11::attribute_error –

Returns

pybind11::object

static void set_input_ids(InferenceMemoryNLP &self, pybind11::object cupy_values)

Set the input ids object.

Parameters

• self –

• cupy_values –

static pybind11::object get_input_mask(InferenceMemoryNLP &self)

Get the input mask object.

Parameters

self –

Throws

pybind11::attribute_error –

Returns

pybind11::object

static void set_input_mask(InferenceMemoryNLP &self, pybind11::object cupy_values)

Set the input mask object.

Parameters

• self –

• cupy_values –

static pybind11::object get_seq_ids(InferenceMemoryNLP &self)

Get the seq ids object.

Parameters

self –

Throws

pybind11::attribute_error –

Returns

pybind11::object

static void set_seq_ids(InferenceMemoryNLP &self, pybind11::object cupy_values)

Set the seq ids object.

Parameters

• self –

• cupy_values –