morpheus.messages - NVIDIA Docs

class ControlMessage

Methods

`add_task`(self, task_type, task)
`config`(args, *kwargs)	Overloaded function.
`copy`(self)
`filter_timestamp`(self, regex_filter)	Retrieve timestamps matching a regex filter within a given group.
`get_metadata`(self[, key, default_value])
`get_tasks`(self)
`get_timestamp`(self, key[, fail_if_nonexist])	Retrieve the timestamp for a given group and key.
`has_metadata`(self, key)
`has_task`(self, task_type)
`list_metadata`(self)
`payload`(args, *kwargs)	Overloaded function.
`remove_task`(self, task_type)
`set_metadata`(self, key, value)
`set_timestamp`(self, key, timestamp)	Set a timestamp for a given key and group.
`task_type`(args, *kwargs)	Overloaded function.
`tensors`(args, *kwargs)	Overloaded function.

add_task(self: morpheus._lib.messages.ControlMessage, task_type: str, task: dict) → None

config(*args, **kwargs)

copy(self: morpheus._lib.messages.ControlMessage) → morpheus._lib.messages.ControlMessage

filter_timestamp(self: morpheus._lib.messages.ControlMessage, regex_filter: str) → dict

get_metadata(self: morpheus._lib.messages.ControlMessage, key: object = None, default_value: object = None) → object

get_tasks(self: morpheus._lib.messages.ControlMessage) → dict

get_timestamp(self: morpheus._lib.messages.ControlMessage, key: str, fail_if_nonexist: bool = False) → object

has_metadata(self: morpheus._lib.messages.ControlMessage, key: str) → bool

has_task(self: morpheus._lib.messages.ControlMessage, task_type: str) → bool

list_metadata(self: morpheus._lib.messages.ControlMessage) → list

payload(*args, **kwargs)

remove_task(self: morpheus._lib.messages.ControlMessage, task_type: str) → dict

set_metadata(self: morpheus._lib.messages.ControlMessage, key: str, value: object) → None

set_timestamp(self: morpheus._lib.messages.ControlMessage, key: str, timestamp: object) → None

task_type(*args, **kwargs)

tensors(*args, **kwargs)

class DataLoaderRegistry

Methods

`contains`(name)
`list`()
`register_loader`(name, loader[, throw_if_exists])
`unregister_loader`(name[, throw_if_not_exists])

static contains(name: str) → bool

static list() → List[str]

static register_loader(name: str, loader: Callable[[morpheus._lib.messages.ControlMessage, dict], morpheus._lib.messages.ControlMessage], throw_if_exists: bool = True) → None

static unregister_loader(name: str, throw_if_not_exists: bool = True) → None

class InferenceMemory(*, count=None, tensors=None)[source]

This is a base container class for data that will be used for inference stages. This class is designed to hold generic tensor data in cupy arrays.

Attributes

tensor_names

Methods

`get_input`(name)	Get the tensor stored in the container identified by `name`.
`get_tensor`(name)	Get the Tensor stored in the container identified by `name`.
`get_tensors`()	Get the tensors contained by this instance.
`has_tensor`(name)	Returns True if a tensor with the requested name exists in the tensors object
`set_input`(name, tensor)	Update the input tensor identified by `name`.
`set_tensor`(name, tensor)	Update the tensor identified by `name`.
`set_tensors`(tensors)	Overwrite the tensors stored by this instance.

count: int

get_input(name)[source]

Get the tensor stored in the container identified by name. Alias for InferenceMemory.get_tensor.

Parameters

name

Returns

cupy.ndarray

Raises

KeyError

set_input(name, tensor)[source]

Update the input tensor identified by name. Alias for InferenceMemory.set_tensor

Parameters

name
tensor

tensors: Dict[str, cupy.ndarray]

class InferenceMemoryAE(*, count, inputs, seq_ids)[source]

This is a container class for data that needs to be submitted to the inference server for auto encoder usecases.

Parameters

inputs
seq_ids

Attributes

input
seq_ids
tensor_names

Methods

`get_input`(name)	Get the tensor stored in the container identified by `name`.
`get_tensor`(name)	Get the Tensor stored in the container identified by `name`.
`get_tensors`()	Get the tensors contained by this instance.
`has_tensor`(name)	Returns True if a tensor with the requested name exists in the tensors object
`set_input`(name, tensor)	Update the input tensor identified by `name`.
`set_tensor`(name, tensor)	Update the tensor identified by `name`.
`set_tensors`(tensors)	Overwrite the tensors stored by this instance.

input: dataclasses.InitVar[cupy.ndarray] = None

seq_ids: dataclasses.InitVar[cupy.ndarray] = None

class InferenceMemoryFIL(*, count, input__0, seq_ids)[source]

This is a container class for data that needs to be submitted to the inference server for FIL category usecases.

Parameters

input__0
seq_ids

Attributes

input__0
seq_ids
tensor_names

Methods

`get_input`(name)	Get the tensor stored in the container identified by `name`.
`get_tensor`(name)	Get the Tensor stored in the container identified by `name`.
`get_tensors`()	Get the tensors contained by this instance.
`has_tensor`(name)	Returns True if a tensor with the requested name exists in the tensors object
`set_input`(name, tensor)	Update the input tensor identified by `name`.
`set_tensor`(name, tensor)	Update the tensor identified by `name`.
`set_tensors`(tensors)	Overwrite the tensors stored by this instance.

input__0: dataclasses.InitVar[cupy.ndarray] = None

seq_ids: dataclasses.InitVar[cupy.ndarray] = None

class InferenceMemoryNLP(*, count, input_ids, input_mask, seq_ids)[source]

This is a container class for data that needs to be submitted to the inference server for NLP category usecases.

Parameters

input_ids
input_mask
seq_ids

Attributes

input_ids
input_mask
seq_ids
tensor_names

Methods

`get_input`(name)	Get the tensor stored in the container identified by `name`.
`get_tensor`(name)	Get the Tensor stored in the container identified by `name`.
`get_tensors`()	Get the tensors contained by this instance.
`has_tensor`(name)	Returns True if a tensor with the requested name exists in the tensors object
`set_input`(name, tensor)	Update the input tensor identified by `name`.
`set_tensor`(name, tensor)	Update the tensor identified by `name`.
`set_tensors`(tensors)	Overwrite the tensors stored by this instance.

input_ids: dataclasses.InitVar[cupy.ndarray] = None

input_mask: dataclasses.InitVar[cupy.ndarray] = None

seq_ids: dataclasses.InitVar[cupy.ndarray] = None

class MessageBase[source]

Base class for all messages. Returns a C++ implementation if CppConfig.get_should_use_cpp() is True and the class has an associated C++ implementation (cpp_class), returns the Python implementation for all others.

class MessageMeta(df)[source]

This is a container class to hold batch deserialized messages metadata.

Parameters

df

Attributes

count
df

Methods

`ensure_sliceable_index`()	Replaces the index in the underlying dataframe if the existing one is not unique and monotonic.
`get_meta_range`(mess_offset, message_count[, ...])	Return column values from `morpheus.pipeline.messages.MessageMeta.df` from the specified start offset until the message count.
`has_sliceable_index`()	Returns True if the underlying DataFrame's index is unique and monotonic.

copy_dataframe
get_column_names
mutable_dataframe

copy_dataframe()[source]

property count: int

Returns the number of messages in the batch.

Returns

int

property df: Union[pandas.DataFrame, cudf.DataFrame]

ensure_sliceable_index()[source]

Replaces the index in the underlying dataframe if the existing one is not unique and monotonic. The old index will be preserved in a column named _index_{old_index.name}. If has_sliceable_index() == true, this is a no-op.

Returns

str

get_column_names()[source]

get_meta_range(mess_offset, message_count, columns=None)[source]

Return column values from morpheus.pipeline.messages.MessageMeta.df from the specified start offset until the message count.

Parameters

mess_offset
message_count
columns

Returns

Series or Dataframe

has_sliceable_index()[source]

Returns True if the underlying DataFrame’s index is unique and monotonic. Sliceable indices have better performance since a range of rows can be specified by a start and stop index instead of requiring boolean masks.

Returns

bool

mutable_dataframe()[source]

class MultiAEMessage(*, meta, mess_offset=0, mess_count=- 1, model, train_scores_mean=0.0, train_scores_std=1.0)[source]

Subclass of MultiMessage specific to the AutoEncoder pipeline, which contains the model.

Attributes

id
id_col
timestamp

Methods

`copy_meta_ranges`(ranges[, mask])	Perform a copy of the underlying dataframe for the given `ranges` of rows.
`copy_ranges`(ranges)	Perform a copy of the current message instance for the given `ranges` of rows.
`from_message`(message, *[, meta, ...])	Creates a new instance of a derived class from `MultiMessage` using an existing message as the template.
`get_meta`([columns])	Return column values from `morpheus.pipeline.messages.MessageMeta.df`.
`get_meta_column_names`()	Return column names available in the underlying DataFrame.
`get_meta_list`([col_name])	Return a column values from morpheus.messages.MessageMeta.df as a list.
`get_slice`(start, stop)	Returns sliced batches based on offsets supplied.
`set_meta`(columns, value)	Set column values to `morpheus.pipelines.messages.MessageMeta.df`.

model: morpheus.models.dfencoder.autoencoder.AutoEncoder

train_scores_mean: float

train_scores_std: float

class MultiInferenceFILMessage(*, meta, mess_offset=0, mess_count=- 1, memory=None, offset=0, count=- 1)[source]

A stronger typed version of MultiInferenceMessage that is used for FIL workloads. Helps ensure the proper inputs are set and eases debugging.

Attributes

id
id_col
input__0
inputs
seq_ids
tensors
timestamp

Methods

`copy_meta_ranges`(ranges[, mask])	Perform a copy of the underlying dataframe for the given `ranges` of rows.
`copy_ranges`(ranges)	Perform a copy of the current message, dataframe and tensors for the given `ranges` of rows.
`copy_tensor_ranges`(ranges[, mask])	Perform a copy of the underlying tensor tensors for the given `ranges` of rows.
`from_message`(message, *[, meta, ...])	Creates a new instance of a derived class from `MultiMessage` using an existing message as the template.
`get_id_tensor`()	Get the tensor that holds message ID information.
`get_input`(name)	Get input stored in the InferenceMemory container.
`get_meta`([columns])	Return column values from `morpheus.pipeline.messages.MessageMeta.df`.
`get_meta_column_names`()	Return column names available in the underlying DataFrame.
`get_meta_list`([col_name])	Return a column values from morpheus.messages.MessageMeta.df as a list.
`get_slice`(start, stop)	Perform a slice of the current message from `start`:`stop` (excluding `stop`)
`get_tensor`(name)	Get tensor stored in the TensorMemory container.
`set_meta`(columns, value)	Set column values to `morpheus.pipelines.messages.MessageMeta.df`.

property input__0

Input to FIL model inference.

Returns

cupy.ndarray

required_tensors: ClassVar[List[str]] = ['input__0', 'seq_ids']

property seq_ids

Returns sequence ids, which are used to keep track of messages in a multi-threaded environment.

Returns

cupy.ndarray

class MultiInferenceMessage(*, meta, mess_offset=0, mess_count=- 1, memory=None, offset=0, count=- 1)[source]

This is a container class that holds the InferenceMemory container and the metadata of the data contained within it. Builds on top of the MultiTensorMessage class to add additional data for inferencing.

This class requires two separate memory blocks for a batch. One for the message metadata (i.e., start time, IP address, etc.) and another for the raw inference inputs (i.e., input_ids, seq_ids). Since there can be more inference input requests than messages (This happens when some messages get broken into multiple inference requests) this class stores two different offset and count values. mess_offset and mess_count refer to the offset and count in the message metadata batch and offset and count index into the inference batch data.

Attributes

id
id_col
inputs
tensors
timestamp

Methods

`copy_meta_ranges`(ranges[, mask])	Perform a copy of the underlying dataframe for the given `ranges` of rows.
`copy_ranges`(ranges)	Perform a copy of the current message, dataframe and tensors for the given `ranges` of rows.
`copy_tensor_ranges`(ranges[, mask])	Perform a copy of the underlying tensor tensors for the given `ranges` of rows.
`from_message`(message, *[, meta, ...])	Creates a new instance of a derived class from `MultiMessage` using an existing message as the template.
`get_id_tensor`()	Get the tensor that holds message ID information.
`get_input`(name)	Get input stored in the InferenceMemory container.
`get_meta`([columns])	Return column values from `morpheus.pipeline.messages.MessageMeta.df`.
`get_meta_column_names`()	Return column names available in the underlying DataFrame.
`get_meta_list`([col_name])	Return a column values from morpheus.messages.MessageMeta.df as a list.
`get_slice`(start, stop)	Perform a slice of the current message from `start`:`stop` (excluding `stop`)
`get_tensor`(name)	Get tensor stored in the TensorMemory container.
`set_meta`(columns, value)	Set column values to `morpheus.pipelines.messages.MessageMeta.df`.

count: int

get_input(name)[source]

Get input stored in the InferenceMemory container. Alias for MultiInferenceMessage.get_tensor.

Parameters

name

Returns

cupy.ndarray

Raises

KeyError

property inputs

Get inputs stored in the InferenceMemory container. Alias for MultiInferenceMessage.tensors.

Returns

cupy.ndarray

memory: morpheus.messages.memory.tensor_memory.TensorMemory

offset: int

class MultiInferenceNLPMessage(*, meta, mess_offset=0, mess_count=- 1, memory=None, offset=0, count=- 1)[source]

A stronger typed version of MultiInferenceMessage that is used for NLP workloads. Helps ensure the proper inputs are set and eases debugging.

Attributes

id
id_col
input_ids
input_mask
inputs
seq_ids
tensors
timestamp

Methods

`copy_meta_ranges`(ranges[, mask])	Perform a copy of the underlying dataframe for the given `ranges` of rows.
`copy_ranges`(ranges)	Perform a copy of the current message, dataframe and tensors for the given `ranges` of rows.
`copy_tensor_ranges`(ranges[, mask])	Perform a copy of the underlying tensor tensors for the given `ranges` of rows.
`from_message`(message, *[, meta, ...])	Creates a new instance of a derived class from `MultiMessage` using an existing message as the template.
`get_id_tensor`()	Get the tensor that holds message ID information.
`get_input`(name)	Get input stored in the InferenceMemory container.
`get_meta`([columns])	Return column values from `morpheus.pipeline.messages.MessageMeta.df`.
`get_meta_column_names`()	Return column names available in the underlying DataFrame.
`get_meta_list`([col_name])	Return a column values from morpheus.messages.MessageMeta.df as a list.
`get_slice`(start, stop)	Perform a slice of the current message from `start`:`stop` (excluding `stop`)
`get_tensor`(name)	Get tensor stored in the TensorMemory container.
`set_meta`(columns, value)	Set column values to `morpheus.pipelines.messages.MessageMeta.df`.

property input_ids

Returns token-ids for each string padded with 0s to max_length.

Returns

cupy.ndarray

property input_mask

Returns mask for token-ids result where corresponding positions identify valid token-id values.

Returns

cupy.ndarray

required_tensors: ClassVar[List[str]] = ['input_ids', 'input_mask', 'seq_ids']

property seq_ids

Returns sequence ids, which are used to keep track of which inference requests belong to each message.

Returns

cupy.ndarray

class MultiMessage(*, meta, mess_offset=0, mess_count=- 1)[source]

This class holds data for multiple messages at a time. To avoid copying data for slicing operations, it holds a reference to a batched metadata object and stores the offset and count into that batch.

Parameters

meta : MessageMeta
mess_offset
mess_count

Attributes

id
id_col
timestamp

Methods

`copy_meta_ranges`(ranges[, mask])	Perform a copy of the underlying dataframe for the given `ranges` of rows.
`copy_ranges`(ranges)	Perform a copy of the current message instance for the given `ranges` of rows.
`from_message`(message, *[, meta, ...])	Creates a new instance of a derived class from `MultiMessage` using an existing message as the template.
`get_meta`()	Return column values from `morpheus.pipeline.messages.MessageMeta.df`.
`get_meta_column_names`()	Return column names available in the underlying DataFrame.
`get_meta_list`([col_name])	Return a column values from morpheus.messages.MessageMeta.df as a list.
`get_slice`(start, stop)	Returns sliced batches based on offsets supplied.
`set_meta`(columns, value)	Set column values to `morpheus.pipelines.messages.MessageMeta.df`.

copy_meta_ranges(ranges, mask=None)[source]

Perform a copy of the underlying dataframe for the given ranges of rows.

Parameters

ranges
mask

Returns

Dataframe

copy_ranges(ranges)[source]

Perform a copy of the current message instance for the given ranges of rows.

Parameters

ranges

Returns

MultiMessage

classmethod from_message(message, *, meta=None, mess_offset=- 1, mess_count=- 1, **kwargs)[source]

Creates a new instance of a derived class from MultiMessage using an existing message as the template. This is very useful when a new message needs to be created with a single change to an existing MessageMeta.

When creating the new message, all required arguments for the class specified by cls will be pulled from message unless otherwise specified in the kwargs. Special handling is performed depending on whether or not a new meta object is supplied. If one is supplied, the offset and count defaults will be 0 and meta.count respectively. Otherwise offset and count will be pulled from the input message.

Parameters

cls
message
meta
mess_offset
mess_count
**kwargs : dict

Returns

Self

Raises

ValueError
AttributeError

get_meta() → cudf.DataFrame[source]

get_meta(columns: str) → cudf.Series

get_meta(columns: List[str]) → cudf.DataFrame

Return column values from morpheus.pipeline.messages.MessageMeta.df.

Parameters

columns

Returns

Series or Dataframe

get_meta_column_names()[source]

Return column names available in the underlying DataFrame.

Returns

list[str]

get_meta_list(col_name=None)[source]

Return a column values from morpheus.messages.MessageMeta.df as a list.

Parameters

col_name

Returns

List[str]

get_slice(start, stop)[source]

Returns sliced batches based on offsets supplied. Automatically calculates the correct mess_offset and mess_count.

Parameters

start
stop

Returns

MultiInferenceMessage

property id: List[int]

Returns ID column values from morpheus.pipeline.messages.MessageMeta.df as list.

Returns

List[int]

property id_col

Returns ID column values from morpheus.pipeline.messages.MessageMeta.df.

Returns

pandas.Series

mess_count: int

mess_offset: int

meta: morpheus.messages.message_meta.MessageMeta

set_meta(columns, value)[source]

Set column values to morpheus.pipelines.messages.MessageMeta.df.

Parameters

columns
value

property timestamp: List[int]

Returns timestamp column values from morpheus.messages.MessageMeta.df as list.

Returns

List[int]

class MultiResponseAEMessage(*, meta, mess_offset=0, mess_count=- 1, memory=None, offset=0, count=- 1, id_tensor_name='seq_ids', probs_tensor_name='probs', user_id=None)[source]

A stronger typed version of MultiResponseProbsMessage that is used for inference workloads that return a probability array. Helps ensure the proper outputs are set and eases debugging.

Attributes

id
id_col
outputs
tensors
timestamp
user_id

Methods

`copy_meta_ranges`(ranges[, mask])	Perform a copy of the underlying dataframe for the given `ranges` of rows.
`copy_ranges`(ranges)	Perform a copy of the current message, dataframe and tensors for the given `ranges` of rows.
`copy_tensor_ranges`(ranges[, mask])	Perform a copy of the underlying tensor tensors for the given `ranges` of rows.
`from_message`(message, *[, meta, ...])	Creates a new instance of a derived class from `MultiMessage` using an existing message as the template.
`get_id_tensor`()	Get the tensor that holds message ID information.
`get_meta`([columns])	Return column values from `morpheus.pipeline.messages.MessageMeta.df`.
`get_meta_column_names`()	Return column names available in the underlying DataFrame.
`get_meta_list`([col_name])	Return a column values from morpheus.messages.MessageMeta.df as a list.
`get_output`(name)	Get output stored in the TensorMemory container.
`get_probs_tensor`()	Get the tensor that holds output probabilities.
`get_slice`(start, stop)	Perform a slice of the current message from `start`:`stop` (excluding `stop`)
`get_tensor`(name)	Get tensor stored in the TensorMemory container.
`set_meta`(columns, value)	Set column values to `morpheus.pipelines.messages.MessageMeta.df`.

user_id: str = None

class MultiResponseMessage(*, meta, mess_offset=0, mess_count=- 1, memory=None, offset=0, count=- 1, id_tensor_name='seq_ids', probs_tensor_name='probs')[source]

This class contains several inference responses as well as the cooresponding message metadata.

Attributes

id
id_col
outputs
tensors
timestamp

Methods

`copy_meta_ranges`(ranges[, mask])	Perform a copy of the underlying dataframe for the given `ranges` of rows.
`copy_ranges`(ranges)	Perform a copy of the current message, dataframe and tensors for the given `ranges` of rows.
`copy_tensor_ranges`(ranges[, mask])	Perform a copy of the underlying tensor tensors for the given `ranges` of rows.
`from_message`(message, *[, meta, ...])	Creates a new instance of a derived class from `MultiMessage` using an existing message as the template.
`get_id_tensor`()	Get the tensor that holds message ID information.
`get_meta`([columns])	Return column values from `morpheus.pipeline.messages.MessageMeta.df`.
`get_meta_column_names`()	Return column names available in the underlying DataFrame.
`get_meta_list`([col_name])	Return a column values from morpheus.messages.MessageMeta.df as a list.
`get_output`(name)	Get output stored in the TensorMemory container.
`get_probs_tensor`()	Get the tensor that holds output probabilities.
`get_slice`(start, stop)	Perform a slice of the current message from `start`:`stop` (excluding `stop`)
`get_tensor`(name)	Get tensor stored in the TensorMemory container.
`set_meta`(columns, value)	Set column values to `morpheus.pipelines.messages.MessageMeta.df`.

get_output(name)[source]

Get output stored in the TensorMemory container. Alias for MultiResponseMessage.get_tensor.

Parameters

name

Returns

cupy.ndarray

get_probs_tensor()[source]

Get the tensor that holds output probabilities. Equivalent to get_tensor(probs_tensor_name)

Returns

cupy.ndarray

Raises

KeyError

property outputs

Get outputs stored in the TensorMemory container. Alias for MultiResponseMessage.tensors.

Returns

cupy.ndarray

probs_tensor_name: ClassVar[str] = 'probs'

class MultiResponseProbsMessage(*args, **kwargs)[source]

A stronger typed version of MultiResponseMessage that is used for inference workloads that return a probability array. Helps ensure the proper outputs are set and eases debugging.

Attributes

id
id_col
outputs
probs
tensors
timestamp

Methods

`copy_meta_ranges`(ranges[, mask])	Perform a copy of the underlying dataframe for the given `ranges` of rows.
`copy_ranges`(ranges)	Perform a copy of the current message, dataframe and tensors for the given `ranges` of rows.
`copy_tensor_ranges`(ranges[, mask])	Perform a copy of the underlying tensor tensors for the given `ranges` of rows.
`from_message`(message, *[, meta, ...])	Creates a new instance of a derived class from `MultiMessage` using an existing message as the template.
`get_id_tensor`()	Get the tensor that holds message ID information.
`get_meta`([columns])	Return column values from `morpheus.pipeline.messages.MessageMeta.df`.
`get_meta_column_names`()	Return column names available in the underlying DataFrame.
`get_meta_list`([col_name])	Return a column values from morpheus.messages.MessageMeta.df as a list.
`get_output`(name)	Get output stored in the TensorMemory container.
`get_probs_tensor`()	Get the tensor that holds output probabilities.
`get_slice`(start, stop)	Perform a slice of the current message from `start`:`stop` (excluding `stop`)
`get_tensor`(name)	Get tensor stored in the TensorMemory container.
`set_meta`(columns, value)	Set column values to `morpheus.pipelines.messages.MessageMeta.df`.

property probs

Probabilities of prediction.

Returns

cupy.ndarray

required_tensors: ClassVar[List[str]] = ['probs']

class MultiTensorMessage(*, meta, mess_offset=0, mess_count=- 1, memory, offset=0, count=- 1, id_tensor_name='seq_ids')[source]

This class contains several inference responses as well as the corresponding message metadata.

Parameters

memory : TensorMemory
offset
count

Attributes

id
id_col
tensors
timestamp

Methods

`copy_meta_ranges`(ranges[, mask])	Perform a copy of the underlying dataframe for the given `ranges` of rows.
`copy_ranges`(ranges)	Perform a copy of the current message, dataframe and tensors for the given `ranges` of rows.
`copy_tensor_ranges`(ranges[, mask])	Perform a copy of the underlying tensor tensors for the given `ranges` of rows.
`from_message`(message, *[, meta, ...])	Creates a new instance of a derived class from `MultiMessage` using an existing message as the template.
`get_id_tensor`()	Get the tensor that holds message ID information.
`get_meta`([columns])	Return column values from `morpheus.pipeline.messages.MessageMeta.df`.
`get_meta_column_names`()	Return column names available in the underlying DataFrame.
`get_meta_list`([col_name])	Return a column values from morpheus.messages.MessageMeta.df as a list.
`get_slice`(start, stop)	Perform a slice of the current message from `start`:`stop` (excluding `stop`)
`get_tensor`(name)	Get tensor stored in the TensorMemory container.
`set_meta`(columns, value)	Set column values to `morpheus.pipelines.messages.MessageMeta.df`.

copy_ranges(ranges)[source]

Perform a copy of the current message, dataframe and tensors for the given ranges of rows.

Parameters

ranges
——-
`MultiTensorMessage`

copy_tensor_ranges(ranges, mask=None)[source]

Perform a copy of the underlying tensor tensors for the given ranges of rows.

Parameters

ranges
mask

Returns

typing.Dict[str, cupy.ndarray]

count: int

classmethod from_message(message, *, meta=None, mess_offset=- 1, mess_count=- 1, memory=None, offset=- 1, count=- 1, **kwargs)[source]

Creates a new instance of a derived class from MultiMessage using an existing message as the template. This is very useful when a new message needs to be created with a single change to an existing MessageMeta.

When creating the new message, all required arguments for the class specified by cls will be pulled from message unless otherwise specified in the kwargs. Special handling is performed depending on whether or not a new meta object is supplied. If one is supplied, the offset and count defaults will be 0 and meta.count respectively. Otherwise offset and count will be pulled from the input message.

Parameters

cls
message
meta
mess_offset
mess_count
memory
offset
count
**kwargs : dict

Returns

Self

Raises

ValueError

get_id_tensor()[source]

Get the tensor that holds message ID information. Equivalent to get_tensor(id_tensor_name)

Returns

cupy.ndarray

Raises

KeyError

get_slice(start, stop)[source]

Perform a slice of the current message from start:stop (excluding stop)

For example to slice from rows 1-3 use m.get_slice(1, 4). The returned MultiTensorMessage will contain references to the same underlying Dataframe and tensor tensors, and this calling this method is reletively low cost compared to MultiTensorMessage.copy_ranges

Parameters

start
stop
——-
`MultiTensorMessage`

get_tensor(name)[source]

Get tensor stored in the TensorMemory container.

Parameters

name

Returns

cupy.ndarray

id_tensor_name: ClassVar[str] = 'seq_ids'

memory: morpheus.messages.memory.tensor_memory.TensorMemory

offset: int

required_tensors: ClassVar[List[str]] = []

property tensors

Get tensors stored in the TensorMemory container sliced according to offset and count.

Returns

cupy.ndarray

class ResponseMemory(*args, **kwargs)[source]

Output memory block holding the results of inference.

Attributes

tensor_names

Methods

`get_output`(name)	Get the Tensor stored in the container identified by `name`.
`get_tensor`(name)	Get the Tensor stored in the container identified by `name`.
`get_tensors`()	Get the tensors contained by this instance.
`has_tensor`(name)	Returns True if a tensor with the requested name exists in the tensors object
`set_output`(name, tensor)	Update the output tensor identified by `name`.
`set_tensor`(name, tensor)	Update the tensor identified by `name`.
`set_tensors`(tensors)	Overwrite the tensors stored by this instance.

count: int

get_output(name)[source]

Get the Tensor stored in the container identified by name. Alias for ResponseMemory.get_tensor.

Parameters

name

Returns

cupy.ndarray

Raises

KeyError

set_output(name, tensor)[source]

Update the output tensor identified by name. Alias for ResponseMemory.set_tensor

Parameters

name
tensor

Raises

ValueError

tensors: Dict[str, cupy.ndarray]

class ResponseMemoryAE(*args, **kwargs)[source]

Subclass of ResponseMemory specific to the AutoEncoder pipeline.

Parameters

probs
user_id
explain_df

Attributes

explain_df
probs
tensor_names

Methods

`get_output`(name)	Get the Tensor stored in the container identified by `name`.
`get_tensor`(name)	Get the Tensor stored in the container identified by `name`.
`get_tensors`()	Get the tensors contained by this instance.
`has_tensor`(name)	Returns True if a tensor with the requested name exists in the tensors object
`set_output`(name, tensor)	Update the output tensor identified by `name`.
`set_tensor`(name, tensor)	Update the tensor identified by `name`.
`set_tensors`(tensors)	Overwrite the tensors stored by this instance.

explain_df = None

probs: dataclasses.InitVar[cupy.ndarray] = None

user_id = ''

class ResponseMemoryProbs(*args, **kwargs)[source]

Subclass of ResponseMemory containng an output tensor named ‘probs’.

Parameters

probs

Attributes

probs
tensor_names

Methods

`get_output`(name)	Get the Tensor stored in the container identified by `name`.
`get_tensor`(name)	Get the Tensor stored in the container identified by `name`.
`get_tensors`()	Get the tensors contained by this instance.
`has_tensor`(name)	Returns True if a tensor with the requested name exists in the tensors object
`set_output`(name, tensor)	Update the output tensor identified by `name`.
`set_tensor`(name, tensor)	Update the tensor identified by `name`.
`set_tensors`(tensors)	Overwrite the tensors stored by this instance.

probs: dataclasses.InitVar[cupy.ndarray] = None

class TensorMemory(*, count=None, tensors=None)[source]

This is a base container class for data that will be used for inference stages. This class is designed to hold generic tensor data in cupy arrays.

Parameters

count
tensors

Attributes

tensor_names

Methods

`get_tensor`(name)	Get the Tensor stored in the container identified by `name`.
`get_tensors`()	Get the tensors contained by this instance.
`has_tensor`(name)	Returns True if a tensor with the requested name exists in the tensors object
`set_tensor`(name, tensor)	Update the tensor identified by `name`.
`set_tensors`(tensors)	Overwrite the tensors stored by this instance.

count: int

get_tensor(name)[source]

Get the Tensor stored in the container identified by name.

Parameters

name

Returns

cupy.ndarray

Raises

KeyError

get_tensors()[source]

Get the tensors contained by this instance. It is important to note that when C++ execution is enabled the returned tensors will be a Python copy of the tensors stored in the C++ object. As such any changes made to the tensors will need to be updated with a call to set_tensors.

Returns

typing.Dict[str, cp.ndarray]

has_tensor(name)[source]

Returns True if a tensor with the requested name exists in the tensors object

Parameters

name

Returns

bool

set_tensor(name, tensor)[source]

Update the tensor identified by name.

Parameters

name
tensor

Raises

ValueError

set_tensors(tensors)[source]

Overwrite the tensors stored by this instance. If the length of the tensors has changed, then the count property should also be updated.

Parameters

tensors

property tensor_names: List[str]

tensors: Dict[str, cupy.ndarray]

class UserMessageMeta(df, user_id)[source]

This class extends MessageMeta to also hold userid corresponding to batched metadata.

Parameters

df
user_id

Attributes

count
df

Methods

`ensure_sliceable_index`()	Replaces the index in the underlying dataframe if the existing one is not unique and monotonic.
`get_meta_range`(mess_offset, message_count[, ...])	Return column values from `morpheus.pipeline.messages.MessageMeta.df` from the specified start offset until the message count.
`has_sliceable_index`()	Returns True if the underlying DataFrame's index is unique and monotonic.

copy_dataframe
get_column_names
mutable_dataframe

user_id: str

`morpheus.messages.data_class_prop`
`morpheus.messages.memory`	Memory classes
`morpheus.messages.message_base`
`morpheus.messages.message_meta`
`morpheus.messages.multi_ae_message`
`morpheus.messages.multi_inference_ae_message`
`morpheus.messages.multi_inference_message`
`morpheus.messages.multi_message`
`morpheus.messages.multi_response_message`
`morpheus.messages.multi_tensor_message`