morpheus.messages.memory.inference_memory.InferenceMemory

class InferenceMemory(*, count=None, tensors=None)[source]

Bases: morpheus.messages.memory.tensor_memory.TensorMemory

This is a base container class for data that will be used for inference stages. This class is designed to hold generic tensor data in cupy arrays.

Attributes:
tensor_names

Methods

get_input(name)

Get the tensor stored in the container identified by name.

get_tensor(name)

Get the Tensor stored in the container identified by name.

get_tensors()

Get the tensors contained by this instance.

has_tensor(name)

Returns True if a tensor with the requested name exists in the tensors object

set_input(name, tensor)

Update the input tensor identified by name.

set_tensor(name, tensor)

Update the tensor identified by name.

set_tensors(tensors)

Overwrite the tensors stored by this instance.

get_input(name)[source]

Get the tensor stored in the container identified by name. Alias for InferenceMemory.get_tensor.

Parameters:
namestr

Key used to do lookup in inputs dict of the container.

Returns:
cupy.ndarray

Inputs corresponding to name.

Raises:
KeyError

If input name does not exist in the container.

get_tensor(name)[source]

Get the Tensor stored in the container identified by name.

Parameters:
namestr

Tensor key name.

Returns:
cupy.ndarray

Tensor.

Raises:
KeyError

If tensor name does not exist in the container.

get_tensors()[source]

Get the tensors contained by this instance. It is important to note that when C++ execution is enabled the returned tensors will be a Python copy of the tensors stored in the C++ object. As such any changes made to the tensors will need to be updated with a call to set_tensors.

Returns:
typing.Dict[str, cp.ndarray]

has_tensor(name)[source]

Returns True if a tensor with the requested name exists in the tensors object

Parameters:
namestr

Name to lookup

Returns:
bool

True if the tensor was found

set_input(name, tensor)[source]

Update the input tensor identified by name. Alias for InferenceMemory.set_tensor

Parameters:
namestr

Key used to do lookup in inputs dict of the container.

tensorcupy.ndarray

Tensor as a CuPy array.

set_tensor(name, tensor)[source]

Update the tensor identified by name.

Parameters:
namestr

Tensor key name.

tensorcupy.ndarray

Tensor as a CuPy array.

Raises:
ValueError

If the number of rows in tensor does not match count

set_tensors(tensors)[source]

Overwrite the tensors stored by this instance. If the length of the tensors has changed, then the count property should also be updated.

Parameters:
tensorstyping.Dict[str, cupy.ndarray]

Collection of tensors uniquely identified by a name.

© Copyright 2023, NVIDIA. Last updated on Aug 23, 2023.