morpheus.messages.multi_inference_message.MultiInferenceMessage

NVIDIA Docs Hub NVIDIA Morpheus NVIDIA Morpheus morpheus.messages.multi_inference_message.MultiInferenceMessage

class MultiInferenceMessage(*args, **kwargs)[source]

Bases: morpheus.messages.multi_message.MultiMessage

This is a container class that holds the TensorMemory container and the metadata of the data contained within it. Builds on top of the MultiMessage class to add additional data for inferencing.

This class requires two separate memory blocks for a batch. One for the message metadata (i.e., start time, IP address, etc.) and another for the raw inference inputs (i.e., input_ids, seq_ids). Since there can be more inference input requests than messages (This happens when some messages get broken into multiple inference requests) this class stores two different offset and count values. mess_offset and mess_count refer to the offset and count in the message metadata batch and offset and count index into the inference batch data.

Parameters

memory<bsp-code-inline code="TensorMemory">TensorMemory</bsp-code-inline>: Inference memory.
offsetint: Message offset in inference memory instance.
countint: Message count in inference memory instance.

Attributes

id: Returns ID column values from morpheus.pipeline.messages.MessageMeta.df as list.
id_col: Returns ID column values from morpheus.pipeline.messages.MessageMeta.df.
inputs: Get inputs stored in the TensorMemory container.
timestamp: Returns timestamp column values from morpheus.messages.MessageMeta.df as list.

Methods

`copy_meta_ranges`(ranges[, mask])	Perform a copy of the underlying dataframe for the given `ranges` of rows.
`copy_ranges`(ranges[, num_selected_rows])	Perform a copy of the current message instance for the given `ranges` of rows.
`get_input`(name)	Get input stored in the TensorMemory container.
`get_meta`([columns])	Return column values from `morpheus.pipeline.messages.MessageMeta.df`.
`get_meta_list`([col_name])	Return a column values from morpheus.messages.MessageMeta.df as a list.
`get_slice`(start, stop)	Returns sliced batches based on offsets supplied.
`set_meta`(columns, value)	Set column values to `morpheus.pipelines.messages.MessageMeta.df`.

copy_meta_ranges(ranges, mask=None)[source]

Perform a copy of the underlying dataframe for the given ranges of rows.

Parameters

rangestyping.List[typing.Tuple[int, int]]: Rows to include in the copy in the form of [(`start_row, stop_row),…]` The stop_row isn’t included. For example to copy rows 1-2 & 5-7 ranges=[(1, 3), (5, 8)]
masktyping.Union[None, cupy.ndarray, numpy.ndarray]: Optionally specify rows as a cupy array (when using cudf Dataframes) or a numpy array (when using pandas Dataframes) of booleans. When not-None ranges will be ignored. This is useful as an optimization as this avoids needing to generate the mask on it’s own.

Returns

Dataframe

copy_ranges(ranges, num_selected_rows=None)[source]

Perform a copy of the current message instance for the given ranges of rows.

Parameters

rangestyping.List[typing.Tuple[int, int]]: Rows to include in the copy in the form of [(`start_row, stop_row),…]` The stop_row isn’t included. For example to copy rows 1-2 & 5-7 ranges=[(1, 3), (5, 8)]
num_selected_rowstyping.Union[None, int]: Optional specify the number of rows selected by ranges, otherwise this is computed by the result.

Returns

MultiMessage

get_input(name)[source]

Get input stored in the TensorMemory container.

Parameters

namestr: Input key name.

Returns

cupy.ndarray: Inference input.

get_meta(columns=None)[source]

Return column values from morpheus.pipeline.messages.MessageMeta.df.

Parameters

columnstyping.Union[None, str, typing.List[str]]: Input column names. Returns all columns if None is specified. When a string is passed, a Series is returned. Otherwise a Dataframe is returned.

Returns

Series or Dataframe: Column values from the dataframe.

get_meta_list(col_name=None)[source]

Return a column values from morpheus.messages.MessageMeta.df as a list.

Parameters

col_namestr: Column name in the dataframe.

Returns

List[str]: Column values from the dataframe.

get_slice(start, stop)[source]

Returns sliced batches based on offsets supplied. Automatically calculates the correct mess_offset and mess_count.

Parameters

startint: Start offset address.
stopint: Stop offset address.

Returns

MultiInferenceMessage: A new MultiInferenceMessage with sliced offset and count.

property id: List[int]

Returns ID column values from morpheus.pipeline.messages.MessageMeta.df as list.

Returns

List[int]: ID column values from the dataframe as list.

property id_col

Returns ID column values from morpheus.pipeline.messages.MessageMeta.df.

Returns

pandas.Series: ID column values from the dataframe.

property inputs

Get inputs stored in the TensorMemory container.

Returns

cupy.ndarray: Inference inputs.

set_meta(columns, value)[source]

Set column values to morpheus.pipelines.messages.MessageMeta.df.

Parameters

columnstyping.Union[None, str, typing.List[str]]: Input column names. Sets the value for the corresponding column names. If None is specified, all columns will be used. If the column does not exist, a new one will be created.
valueAny: Value to apply to the specified columns. If a single value is passed, it will be broadcast to all rows. If a Series or Dataframe is passed, rows will be matched by index.

property timestamp: List[int]

Returns timestamp column values from morpheus.messages.MessageMeta.df as list.

Returns

List[int]: Timestamp column values from the dataframe as list.