morpheus.messages

Message classes, which contain data being transfered between pipeline stages

class InferenceMemory(*args, **kwargs)[source]

This is a base container class for data that will be used for inference stages. This class is designed to hold generic tensor data in cupy arrays.

class InferenceMemoryAE(*args, **kwargs)[source]

This is a container class for data that needs to be submitted to the inference server for auto encoder usecases.

Parameters
inputcupy.ndarray

Inference input.

seq_idscupy.ndarray

Ids used to index from an inference input to a message. Necessary since there can be more inference inputs than messages (i.e., if some messages get broken into multiple inference requests).

Attributes
input
seq_ids
class InferenceMemoryFIL(*args, **kwargs)[source]

This is a container class for data that needs to be submitted to the inference server for FIL category usecases.

Parameters
input__0cupy.ndarray

Inference input.

seq_idscupy.ndarray

Ids used to index from an inference input to a message. Necessary since there can be more inference inputs than messages (i.e., if some messages get broken into multiple inference requests).

Attributes
input__0
seq_ids
class InferenceMemoryNLP(*args, **kwargs)[source]

This is a container class for data that needs to be submitted to the inference server for NLP category usecases.

Parameters
input_idscupy.ndarray

The token-ids for each string padded with 0s to max_length.

input_maskcupy.ndarray

The mask for token-ids result where corresponding positions identify valid token-id values.

seq_idscupy.ndarray

Ids used to index from an inference input to a message. Necessary since there can be more inference inputs than messages (i.e., if some messages get broken into multiple inference requests).

Attributes
input_ids
input_mask
seq_ids
class MessageBase(*args, **kwargs)[source]

Base class for all messages. Returns a C++ implementation if CppConfig.get_should_use_cpp() is True and the class has an associated C++ implementation (cpp_class), returns the Python implementation for all others.

class MessageMeta(*args, **kwargs)[source]

This is a container class to hold batch deserialized messages metadata.

Parameters
dfpandas.DataFrame

Input rows in dataframe.

Attributes
count

Returns the number of messages in the batch.

property count: int

Returns the number of messages in the batch.

Returns
int

number of messages in the MessageMeta.df.

class MultiAEMessage(*args, **kwargs)[source]

Subclass of MultiMessage specific to the AutoEncoder pipeline, which contains the model.

Attributes
id

Returns ID column values from morpheus.pipeline.messages.MessageMeta.df as list.

id_col

Returns ID column values from morpheus.pipeline.messages.MessageMeta.df.

timestamp

Returns timestamp column values from morpheus.messages.MessageMeta.df as list.

Methods

copy_meta_ranges(ranges[, mask])

Perform a copy of the underlying dataframe for the given ranges of rows.

copy_ranges(ranges[, num_selected_rows])

Perform a copy of the current message instance for the given ranges of rows.

get_meta([columns])

Return column values from morpheus.pipeline.messages.MessageMeta.df.

get_meta_list([col_name])

Return a column values from morpheus.messages.MessageMeta.df as a list.

get_slice(start, stop)

Returns sliced batches based on offsets supplied.

set_meta(columns, value)

Set column values to morpheus.pipelines.messages.MessageMeta.df.

copy_ranges(ranges, num_selected_rows=None)[source]

Perform a copy of the current message instance for the given ranges of rows.

Parameters
rangestyping.List[typing.Tuple[int, int]]

Rows to include in the copy in the form of [(`start_row, stop_row),…]` The final output is exclusive of the stop_row, i.e. [start_row, stop_row). For example to copy rows 1-2 & 5-7 ranges=[(1, 3), (5, 8)]

num_selected_rowstyping.Union[None, int]

Optional specify the number of rows selected by ranges, otherwise this is computed by the result.

Returns
MultiAEMessage
get_slice(start, stop)[source]

Returns sliced batches based on offsets supplied. Automatically calculates the correct mess_offset and mess_count.

Parameters
startint

Start offset address.

stopint

Stop offset address.

Returns
morpheus.pipeline.preprocess.autoencoder.MultiAEMessage

A new MultiAEMessage with sliced offset and count.

class MultiInferenceFILMessage(*args, **kwargs)[source]

A stronger typed version of MultiInferenceMessage that is used for FIL workloads. Helps ensure the proper inputs are set and eases debugging.

Attributes
id

Returns ID column values from morpheus.pipeline.messages.MessageMeta.df as list.

id_col

Returns ID column values from morpheus.pipeline.messages.MessageMeta.df.

input__0

Input to FIL model inference.

inputs

Get inputs stored in the TensorMemory container.

seq_ids

Returns sequence ids, which are used to keep track of messages in a multi-threaded environment.

timestamp

Returns timestamp column values from morpheus.messages.MessageMeta.df as list.

Methods

copy_meta_ranges(ranges[, mask])

Perform a copy of the underlying dataframe for the given ranges of rows.

copy_ranges(ranges[, num_selected_rows])

Perform a copy of the current message instance for the given ranges of rows.

get_input(name)

Get input stored in the TensorMemory container.

get_meta([columns])

Return column values from morpheus.pipeline.messages.MessageMeta.df.

get_meta_list([col_name])

Return a column values from morpheus.messages.MessageMeta.df as a list.

get_slice(start, stop)

Returns sliced batches based on offsets supplied.

set_meta(columns, value)

Set column values to morpheus.pipelines.messages.MessageMeta.df.

property input__0

Input to FIL model inference.

Returns
cupy.ndarray

Input data.

property seq_ids

Returns sequence ids, which are used to keep track of messages in a multi-threaded environment.

Returns
cupy.ndarray

Sequence ids.

class MultiInferenceMessage(*args, **kwargs)[source]

This is a container class that holds the TensorMemory container and the metadata of the data contained within it. Builds on top of the MultiMessage class to add additional data for inferencing.

This class requires two separate memory blocks for a batch. One for the message metadata (i.e., start time, IP address, etc.) and another for the raw inference inputs (i.e., input_ids, seq_ids). Since there can be more inference input requests than messages (This happens when some messages get broken into multiple inference requests) this class stores two different offset and count values. mess_offset and mess_count refer to the offset and count in the message metadata batch and offset and count index into the inference batch data.

Parameters
memory<bsp-code-inline code="TensorMemory">TensorMemory</bsp-code-inline>

Inference memory.

offsetint

Message offset in inference memory instance.

countint

Message count in inference memory instance.

Attributes
id

Returns ID column values from morpheus.pipeline.messages.MessageMeta.df as list.

id_col

Returns ID column values from morpheus.pipeline.messages.MessageMeta.df.

inputs

Get inputs stored in the TensorMemory container.

timestamp

Returns timestamp column values from morpheus.messages.MessageMeta.df as list.

Methods

copy_meta_ranges(ranges[, mask])

Perform a copy of the underlying dataframe for the given ranges of rows.

copy_ranges(ranges[, num_selected_rows])

Perform a copy of the current message instance for the given ranges of rows.

get_input(name)

Get input stored in the TensorMemory container.

get_meta([columns])

Return column values from morpheus.pipeline.messages.MessageMeta.df.

get_meta_list([col_name])

Return a column values from morpheus.messages.MessageMeta.df as a list.

get_slice(start, stop)

Returns sliced batches based on offsets supplied.

set_meta(columns, value)

Set column values to morpheus.pipelines.messages.MessageMeta.df.

get_input(name)[source]

Get input stored in the TensorMemory container.

Parameters
namestr

Input key name.

Returns
cupy.ndarray

Inference input.

get_slice(start, stop)[source]

Returns sliced batches based on offsets supplied. Automatically calculates the correct mess_offset and mess_count.

Parameters
startint

Start offset address.

stopint

Stop offset address.

Returns
MultiInferenceMessage

A new MultiInferenceMessage with sliced offset and count.

property inputs

Get inputs stored in the TensorMemory container.

Returns
cupy.ndarray

Inference inputs.

class MultiInferenceNLPMessage(*args, **kwargs)[source]

A stronger typed version of MultiInferenceMessage that is used for NLP workloads. Helps ensure the proper inputs are set and eases debugging.

Attributes
id

Returns ID column values from morpheus.pipeline.messages.MessageMeta.df as list.

id_col

Returns ID column values from morpheus.pipeline.messages.MessageMeta.df.

input_ids

Returns token-ids for each string padded with 0s to max_length.

input_mask

Returns mask for token-ids result where corresponding positions identify valid token-id values.

inputs

Get inputs stored in the TensorMemory container.

seq_ids

Returns sequence ids, which are used to keep track of which inference requests belong to each message.

timestamp

Returns timestamp column values from morpheus.messages.MessageMeta.df as list.

Methods

copy_meta_ranges(ranges[, mask])

Perform a copy of the underlying dataframe for the given ranges of rows.

copy_ranges(ranges[, num_selected_rows])

Perform a copy of the current message instance for the given ranges of rows.

get_input(name)

Get input stored in the TensorMemory container.

get_meta([columns])

Return column values from morpheus.pipeline.messages.MessageMeta.df.

get_meta_list([col_name])

Return a column values from morpheus.messages.MessageMeta.df as a list.

get_slice(start, stop)

Returns sliced batches based on offsets supplied.

set_meta(columns, value)

Set column values to morpheus.pipelines.messages.MessageMeta.df.

property input_ids

Returns token-ids for each string padded with 0s to max_length.

Returns
cupy.ndarray

The token-ids for each string padded with 0s to max_length.

property input_mask

Returns mask for token-ids result where corresponding positions identify valid token-id values.

Returns
cupy.ndarray

The mask for token-ids result where corresponding positions identify valid token-id values.

property seq_ids

Returns sequence ids, which are used to keep track of which inference requests belong to each message.

Returns
cupy.ndarray

Ids used to index from an inference input to a message. Necessary since there can be more inference inputs than messages (i.e., if some messages get broken into multiple inference requests).

class MultiMessage(*args, **kwargs)[source]

This class holds data for multiple messages at a time. To avoid copying data for slicing operations, it holds a reference to a batched metadata object and stores the offset and count into that batch.

Parameters
meta<bsp-code-inline code="<a href="#morpheus.messages.MessageMeta">MessageMeta</a>"><a href="#morpheus.messages.MessageMeta">MessageMeta</a></bsp-code-inline>

Deserialized messages metadata for large batch.

mess_offsetint

Offset into the metadata batch.

mess_countint

Messages count.

Attributes
id

Returns ID column values from morpheus.pipeline.messages.MessageMeta.df as list.

id_col

Returns ID column values from morpheus.pipeline.messages.MessageMeta.df.

timestamp

Returns timestamp column values from morpheus.messages.MessageMeta.df as list.

Methods

copy_meta_ranges(ranges[, mask])

Perform a copy of the underlying dataframe for the given ranges of rows.

copy_ranges(ranges[, num_selected_rows])

Perform a copy of the current message instance for the given ranges of rows.

get_meta([columns])

Return column values from morpheus.pipeline.messages.MessageMeta.df.

get_meta_list([col_name])

Return a column values from morpheus.messages.MessageMeta.df as a list.

get_slice(start, stop)

Returns sliced batches based on offsets supplied.

set_meta(columns, value)

Set column values to morpheus.pipelines.messages.MessageMeta.df.

copy_meta_ranges(ranges, mask=None)[source]

Perform a copy of the underlying dataframe for the given ranges of rows.

Parameters
rangestyping.List[typing.Tuple[int, int]]

Rows to include in the copy in the form of [(`start_row, stop_row),…]` The stop_row isn’t included. For example to copy rows 1-2 & 5-7 ranges=[(1, 3), (5, 8)]

masktyping.Union[None, cupy.ndarray, numpy.ndarray]

Optionally specify rows as a cupy array (when using cudf Dataframes) or a numpy array (when using pandas Dataframes) of booleans. When not-None ranges will be ignored. This is useful as an optimization as this avoids needing to generate the mask on it’s own.

Returns
Dataframe
copy_ranges(ranges, num_selected_rows=None)[source]

Perform a copy of the current message instance for the given ranges of rows.

Parameters
rangestyping.List[typing.Tuple[int, int]]

Rows to include in the copy in the form of [(`start_row, stop_row),…]` The stop_row isn’t included. For example to copy rows 1-2 & 5-7 ranges=[(1, 3), (5, 8)]

num_selected_rowstyping.Union[None, int]

Optional specify the number of rows selected by ranges, otherwise this is computed by the result.

Returns
MultiMessage
get_meta(columns=None)[source]

Return column values from morpheus.pipeline.messages.MessageMeta.df.

Parameters
columnstyping.Union[None, str, typing.List[str]]

Input column names. Returns all columns if None is specified. When a string is passed, a Series is returned. Otherwise a Dataframe is returned.

Returns
Series or Dataframe

Column values from the dataframe.

get_meta_list(col_name=None)[source]

Return a column values from morpheus.messages.MessageMeta.df as a list.

Parameters
col_namestr

Column name in the dataframe.

Returns
List[str]

Column values from the dataframe.

get_slice(start, stop)[source]

Returns sliced batches based on offsets supplied. Automatically calculates the correct mess_offset and mess_count.

Parameters
startint

Start offset address.

stopint

Stop offset address.

Returns
MultiInferenceMessage

A new MultiInferenceMessage with sliced offset and count.

property id: List[int]

Returns ID column values from morpheus.pipeline.messages.MessageMeta.df as list.

Returns
List[int]

ID column values from the dataframe as list.

property id_col

Returns ID column values from morpheus.pipeline.messages.MessageMeta.df.

Returns
pandas.Series

ID column values from the dataframe.

set_meta(columns, value)[source]

Set column values to morpheus.pipelines.messages.MessageMeta.df.

Parameters
columnstyping.Union[None, str, typing.List[str]]

Input column names. Sets the value for the corresponding column names. If None is specified, all columns will be used. If the column does not exist, a new one will be created.

valueAny

Value to apply to the specified columns. If a single value is passed, it will be broadcast to all rows. If a Series or Dataframe is passed, rows will be matched by index.

property timestamp: List[int]

Returns timestamp column values from morpheus.messages.MessageMeta.df as list.

Returns
List[int]

Timestamp column values from the dataframe as list.

class MultiResponseAEMessage(*args, **kwargs)[source]

A stronger typed version of MultiResponseProbsMessage that is used for inference workloads that return a probability array. Helps ensure the proper outputs are set and eases debugging.

Attributes
id

Returns ID column values from morpheus.pipeline.messages.MessageMeta.df as list.

id_col

Returns ID column values from morpheus.pipeline.messages.MessageMeta.df.

outputs

Get outputs stored in the TensorMemory container.

probs

Probabilities of prediction.

timestamp

Returns timestamp column values from morpheus.messages.MessageMeta.df as list.

user_id

Methods

copy_meta_ranges(ranges[, mask])

Perform a copy of the underlying dataframe for the given ranges of rows.

copy_output_ranges(ranges[, mask])

Perform a copy of the underlying output tensors for the given ranges of rows.

copy_ranges(ranges)

Perform a copy of the current message, dataframe and tensors for the given ranges of rows.

get_meta([columns])

Return column values from morpheus.pipeline.messages.MessageMeta.df.

get_meta_list([col_name])

Return a column values from morpheus.messages.MessageMeta.df as a list.

get_output(name)

Get output stored in the TensorMemory container.

get_slice(start, stop)

Perform a slice of the current message from start:stop (excluding stop)

set_meta(columns, value)

Set column values to morpheus.pipelines.messages.MessageMeta.df.

copy_ranges(ranges)[source]

Perform a copy of the current message, dataframe and tensors for the given ranges of rows.

Parameters
rangestyping.List[typing.Tuple[int, int]]

Rows to include in the copy in the form of [(`start_row, stop_row),…]` The stop_row isn’t included. For example to copy rows 1-2 & 5-7 ranges=[(1, 3), (5, 8)]

——-
`MultiResponseAEMessage`
get_slice(start, stop)[source]

Perform a slice of the current message from start:stop (excluding stop)

For example to slice from rows 1-3 use m.get_slice(1, 4). The returned MultiResponseMessage will contain references to the same underlying Dataframe and output tensors, and this calling this method is reletively low cost compared to MultiResponseAEMessage.copy_ranges

Parameters
startint

Starting row of the slice

stopint

Stop of the slice

——-
`MultiResponseAEMessage`
class MultiResponseMessage(*args, **kwargs)[source]

This class contains several inference responses as well as the cooresponding message metadata.

Parameters
memory<bsp-code-inline code="TensorMemory">TensorMemory</bsp-code-inline>

This is a response container instance for triton inference requests.

offsetint

Offset of each response message into the TensorMemory block.

countint

Inference results size of all responses.

Attributes
id

Returns ID column values from morpheus.pipeline.messages.MessageMeta.df as list.

id_col

Returns ID column values from morpheus.pipeline.messages.MessageMeta.df.

outputs

Get outputs stored in the TensorMemory container.

timestamp

Returns timestamp column values from morpheus.messages.MessageMeta.df as list.

Methods

copy_meta_ranges(ranges[, mask])

Perform a copy of the underlying dataframe for the given ranges of rows.

copy_output_ranges(ranges[, mask])

Perform a copy of the underlying output tensors for the given ranges of rows.

copy_ranges(ranges)

Perform a copy of the current message, dataframe and tensors for the given ranges of rows.

get_meta([columns])

Return column values from morpheus.pipeline.messages.MessageMeta.df.

get_meta_list([col_name])

Return a column values from morpheus.messages.MessageMeta.df as a list.

get_output(name)

Get output stored in the TensorMemory container.

get_slice(start, stop)

Perform a slice of the current message from start:stop (excluding stop)

set_meta(columns, value)

Set column values to morpheus.pipelines.messages.MessageMeta.df.

copy_output_ranges(ranges, mask=None)[source]

Perform a copy of the underlying output tensors for the given ranges of rows.

Parameters
rangestyping.List[typing.Tuple[int, int]]

Rows to include in the copy in the form of [(`start_row, stop_row),…]` The stop_row isn’t included. For example to copy rows 1-2 & 5-7 ranges=[(1, 3), (5, 8)]

masktyping.Union[None, cupy.ndarray, numpy.ndarray]

Optionally specify rows as a cupy array (when using cudf Dataframes) or a numpy array (when using pandas Dataframes) of booleans. When not-None ranges will be ignored. This is useful as an optimization as this avoids needing to generate the mask on it’s own.

Returns
typing.Dict[str, cupy.ndarray]
copy_ranges(ranges)[source]

Perform a copy of the current message, dataframe and tensors for the given ranges of rows.

Parameters
rangestyping.List[typing.Tuple[int, int]]

Rows to include in the copy in the form of [(`start_row, stop_row),…]` The stop_row isn’t included. For example to copy rows 1-2 & 5-7 ranges=[(1, 3), (5, 8)]

——-
`MultiResponseMessage`
get_output(name)[source]

Get output stored in the TensorMemory container.

Parameters
namestr

Output key name.

Returns
cupy.ndarray

Inference output.

get_slice(start, stop)[source]

Perform a slice of the current message from start:stop (excluding stop)

For example to slice from rows 1-3 use m.get_slice(1, 4). The returned MultiResponseMessage will contain references to the same underlying Dataframe and output tensors, and this calling this method is reletively low cost compared to MultiResponseMessage.copy_ranges

Parameters
startint

Starting row of the slice

stopint

Stop of the slice

——-
`MultiResponseMessage`
property outputs

Get outputs stored in the TensorMemory container.

Returns
cupy.ndarray

Inference outputs.

class MultiResponseProbsMessage(*args, **kwargs)[source]

A stronger typed version of MultiResponseMessage that is used for inference workloads that return a probability array. Helps ensure the proper outputs are set and eases debugging.

Attributes
id

Returns ID column values from morpheus.pipeline.messages.MessageMeta.df as list.

id_col

Returns ID column values from morpheus.pipeline.messages.MessageMeta.df.

outputs

Get outputs stored in the TensorMemory container.

probs

Probabilities of prediction.

timestamp

Returns timestamp column values from morpheus.messages.MessageMeta.df as list.

Methods

copy_meta_ranges(ranges[, mask])

Perform a copy of the underlying dataframe for the given ranges of rows.

copy_output_ranges(ranges[, mask])

Perform a copy of the underlying output tensors for the given ranges of rows.

copy_ranges(ranges)

Perform a copy of the current message, dataframe and tensors for the given ranges of rows.

get_meta([columns])

Return column values from morpheus.pipeline.messages.MessageMeta.df.

get_meta_list([col_name])

Return a column values from morpheus.messages.MessageMeta.df as a list.

get_output(name)

Get output stored in the TensorMemory container.

get_slice(start, stop)

Perform a slice of the current message from start:stop (excluding stop)

set_meta(columns, value)

Set column values to morpheus.pipelines.messages.MessageMeta.df.

property probs

Probabilities of prediction.

Returns
cupy.ndarray

probabilities

class ResponseMemory(*args, **kwargs)[source]

Output memory block holding the results of inference.

Methods

get_output(name)

Return the output tensor specified by name.

get_output(name)[source]

Return the output tensor specified by name.

Parameters
namestr

Name of output tensor.

Returns
cupy.ndarray

Tensor corresponding to name.

class ResponseMemoryAE(*args, **kwargs)[source]

Subclass of ResponseMemory specific to the AutoEncoder pipeline.

Parameters
probscupy.ndarray

Probabilities tensor

user_idstr

User id the inference was performed against.

explain_dfpd.Dataframe

Explainability Dataframe, for each feature a column will exist with a name in the form of: {feature}_z_loss containing the loss z-score along with max_abs_z and mean_abs_z columns

Attributes
explain_df
probs

Methods

get_output(name)

Return the output tensor specified by name.

class ResponseMemoryProbs(*args, **kwargs)[source]

Subclass of ResponseMemory containng an output tensor named ‘probs’.

Parameters
probscupy.ndarray

Probabilities tensor

Attributes
probs

Methods

get_output(name)

Return the output tensor specified by name.

class UserMessageMeta(*args, **kwargs)[source]

This class extends MessageMeta to also hold userid corresponding to batched metadata.

Parameters
dfpandas.DataFrame

Input rows in dataframe.

user_idstr

User id.

Attributes
count

Returns the number of messages in the batch.

Modules

morpheus.messages.data_class_prop

morpheus.messages.message_base

morpheus.messages.message_meta

morpheus.messages.multi_ae_message

morpheus.messages.multi_inference_ae_message

morpheus.messages.multi_inference_message

morpheus.messages.multi_message

morpheus.messages.multi_response_message

morpheus.messages.tensor_memory

© Copyright 2023, NVIDIA. Last updated on Feb 3, 2023.