morpheus.messages.memory.inference_memory.InferenceMemory

Bases: morpheus.messages.memory.tensor_memory.TensorMemory

This is a base container class for data that will be used for inference stages. This class is designed to hold generic tensor data in cupy arrays.

Attributes

tensor_names

Methods

get_input(name)[source]

Get the tensor stored in the container identified by name. Alias for InferenceMemory.get_tensor.

Parameters

name

Key used to do lookup in inputs dict of the container.

Returns

cupy.ndarray

Inputs corresponding to name.

Raises

KeyError

If input name does not exist in the container.

get_tensor(name)[source]

Get the Tensor stored in the container identified by name.

Parameters

name

Tensor key name.

Returns

cupy.ndarray

Tensor.

Raises

KeyError

If tensor name does not exist in the container.

get_tensors()[source]

Get the tensors contained by this instance. It is important to note that when C++ execution is enabled the returned tensors will be a Python copy of the tensors stored in the C++ object. As such any changes made to the tensors will need to be updated with a call to set_tensors.

Returns

typing.Dict[str, cp.ndarray]

has_tensor(name)[source]

Returns True if a tensor with the requested name exists in the tensors object

Parameters

name

Name to lookup

Returns

bool

True if the tensor was found

set_input(name, tensor)[source]

Update the input tensor identified by name. Alias for InferenceMemory.set_tensor

Parameters

name

Key used to do lookup in inputs dict of the container.

tensor

Tensor as a CuPy array.

set_tensor(name, tensor)[source]

Update the tensor identified by name.

Parameters

name

Tensor key name.

tensor

Tensor as a CuPy array.

Raises

ValueError

If the number of rows in tensor does not match count

set_tensors(tensors)[source]

Overwrite the tensors stored by this instance. If the length of the tensors has changed, then the count property should also be updated.

Parameters

tensors

Collection of tensors uniquely identified by a name.