morpheus.messages.memory.inference_memory.InferenceMemory

class InferenceMemory(**kwargs)

Bases: TensorMemory

This is a base container class for data that will be used for inference stages. This class is designed to hold generic tensor data in either CuPy or NumPy arrays.

Attributes:

tensor_names

Methods

get_input(name)	Get the tensor stored in the container identified by name.
get_tensor(name)	Get the Tensor stored in the container identified by name.
get_tensors()	Get the tensors contained by this instance.
has_tensor(name)	Returns True if a tensor with the requested name exists in the tensors object
set_input(name, tensor)	Update the input tensor identified by name.
set_tensor(name, tensor)	Update the tensor identified by name.
set_tensors(tensors)	Overwrite the tensors stored by this instance.

get_input(name)

Get the tensor stored in the container identified by name. Alias for InferenceMemory.get_tensor.

Parameters:

namestr

Key used to do lookup in inputs dict of the container.

Returns:

NDArrayType

Inputs corresponding to name.

Raises:

KeyError

If input name does not exist in the container.

get_tensor(name)

Get the Tensor stored in the container identified by name.

Parameters:

namestr

Tensor key name.

Returns:

NDArrayType

Tensor.

Raises:

KeyError

If tensor name does not exist in the container.

get_tensors()

Get the tensors contained by this instance. It is important to note that when C++ execution is enabled the returned tensors will be a Python copy of the tensors stored in the C++ object. As such any changes made to the tensors will need to be updated with a call to set_tensors.

Returns:

TensorMapType

has_tensor(name)

Returns True if a tensor with the requested name exists in the tensors object

Parameters:

namestr

Name to lookup

Returns:

bool

True if the tensor was found

set_input(name, tensor)

Update the input tensor identified by name. Alias for InferenceMemory.set_tensor

Parameters:

namestr

Key used to do lookup in inputs dict of the container.

tensorNDArrayType

Tensor as either CuPy or NumPy array.

set_tensor(name, tensor)

Update the tensor identified by name.

Parameters:

namestr

Tensor key name.

tensorNDArrayType

Tensor as either a CuPy or NumPy array.

Raises:

ValueError

If the number of rows in tensor does not match count

set_tensors(tensors)

Overwrite the tensors stored by this instance. If the length of the tensors has changed, then the count property should also be updated.

Parameters:

tensorsTensorMapType

Collection of tensors uniquely identified by a name.