NVIDIA Morpheus (24.06)
(Latest Version)

morpheus.messages.memory.inference_memory.InferenceMemoryNLP

class InferenceMemoryNLP(*, count, input_ids, input_mask, seq_ids)[source]

Bases: morpheus.messages.memory.inference_memory.InferenceMemory

This is a container class for data that needs to be submitted to the inference server for NLP category usecases.

Parameters
input_ids

The token-ids for each string padded with 0s to max_length.

input_mask

The mask for token-ids result where corresponding positions identify valid token-id values.

seq_ids

Ids used to index from an inference input to a message. Necessary since there can be more inference inputs than messages (i.e., if some messages get broken into multiple inference requests).

Attributes
input_ids

input_mask

seq_ids

tensor_names

Methods

get_input(name) Get the tensor stored in the container identified by name.
get_tensor(name) Get the Tensor stored in the container identified by name.
get_tensors() Get the tensors contained by this instance.
has_tensor(name) Returns True if a tensor with the requested name exists in the tensors object
set_input(name, tensor) Update the input tensor identified by name.
set_tensor(name, tensor) Update the tensor identified by name.
set_tensors(tensors) Overwrite the tensors stored by this instance.
get_input(name)[source]

Get the tensor stored in the container identified by name. Alias for InferenceMemory.get_tensor.

Parameters
name

Key used to do lookup in inputs dict of the container.

Returns
cupy.ndarray

Inputs corresponding to name.

Raises
KeyError

If input name does not exist in the container.

get_tensor(name)[source]

Get the Tensor stored in the container identified by name.

Parameters
name

Tensor key name.

Returns
cupy.ndarray

Tensor.

Raises
KeyError

If tensor name does not exist in the container.

get_tensors()[source]

Get the tensors contained by this instance. It is important to note that when C++ execution is enabled the returned tensors will be a Python copy of the tensors stored in the C++ object. As such any changes made to the tensors will need to be updated with a call to set_tensors.

Returns
typing.Dict[str, cp.ndarray]

has_tensor(name)[source]

Returns True if a tensor with the requested name exists in the tensors object

Parameters
name

Name to lookup

Returns
bool

True if the tensor was found

set_input(name, tensor)[source]

Update the input tensor identified by name. Alias for InferenceMemory.set_tensor

Parameters
name

Key used to do lookup in inputs dict of the container.

tensor

Tensor as a CuPy array.

set_tensor(name, tensor)[source]

Update the tensor identified by name.

Parameters
name

Tensor key name.

tensor

Tensor as a CuPy array.

Raises
ValueError

If the number of rows in tensor does not match count

set_tensors(tensors)[source]

Overwrite the tensors stored by this instance. If the length of the tensors has changed, then the count property should also be updated.

Parameters
tensors

Collection of tensors uniquely identified by a name.

Previous morpheus.messages.memory.inference_memory.InferenceMemoryFIL
Next morpheus.messages.memory.response_memory
© Copyright 2024, NVIDIA. Last updated on Jul 8, 2024.