NVIDIA Morpheus (24.06)
(Latest Version)

Struct InferenceMemoryNLPInterfaceProxy

Base Type

struct InferenceMemoryNLPInterfaceProxy : public morpheus::InferenceMemoryInterfaceProxy

Interface proxy, used to insulate python bindings.

Public Static Functions

static std::shared_ptr<InferenceMemoryNLP> init(TensorIndex count, pybind11::object input_ids, pybind11::object input_mask, pybind11::object seq_ids)

Create and initialize an InferenceMemoryNLP object, and return a shared pointer to the result.

Parameters
  • count – : Message count in inference memory object

  • input_ids – : The token-ids for each string padded with 0s to max_length

  • input_mask – : The mask for token-ids result where corresponding positions identify valid token-id values

  • seq_ids – : Ids used to index from an inference input to a message. Necessary since there can be more inference inputs than messages (i.e., if some messages get broken into multiple inference requests)

Returns

std::shared_ptr<InferenceMemoryNLP>

static pybind11::object get_input_ids(InferenceMemoryNLP &self)

: Returns token-ids for each string padded with 0s to max_length as python object

Parameters

self

Throws

pybind11::attribute_error

Returns

pybind11::object

static void set_input_ids(InferenceMemoryNLP &self, pybind11::object cupy_values)

Set the input ids object.

Parameters
  • self

  • cupy_values

static pybind11::object get_input_mask(InferenceMemoryNLP &self)

Get the input mask object.

Parameters

self

Throws

pybind11::attribute_error

Returns

pybind11::object

static void set_input_mask(InferenceMemoryNLP &self, pybind11::object cupy_values)

Set the input mask object.

Parameters
  • self

  • cupy_values

static pybind11::object get_seq_ids(InferenceMemoryNLP &self)

Get the seq ids object.

Parameters

self

Throws

pybind11::attribute_error

Returns

pybind11::object

static void set_seq_ids(InferenceMemoryNLP &self, pybind11::object cupy_values)

Set the seq ids object.

Parameters
  • self

  • cupy_values

Previous Struct InferenceMemoryInterfaceProxy
Next Struct ITensor
© Copyright 2024, NVIDIA. Last updated on Jul 8, 2024.