- class MultiInferenceMessage(*args, **kwargs)[source]
Bases:
morpheus.messages.multi_message.MultiMessage
This is a container class that holds the TensorMemory container and the metadata of the data contained within it. Builds on top of the
MultiMessage
class to add additional data for inferencing.This class requires two separate memory blocks for a batch. One for the message metadata (i.e., start time, IP address, etc.) and another for the raw inference inputs (i.e., input_ids, seq_ids). Since there can be more inference input requests than messages (This happens when some messages get broken into multiple inference requests) this class stores two different offset and count values.
mess_offset
andmess_count
refer to the offset and count in the message metadata batch andoffset
andcount
index into the inference batch data.- Parameters
- memory<bsp-code-inline code="TensorMemory">TensorMemory</bsp-code-inline>
Inference memory.
- offsetint
Message offset in inference memory instance.
- countint
Message count in inference memory instance.
- Attributes
id
Returns ID column values from
morpheus.pipeline.messages.MessageMeta.df
as list.id_col
Returns ID column values from
morpheus.pipeline.messages.MessageMeta.df
.inputs
Get inputs stored in the TensorMemory container.
timestamp
Returns timestamp column values from morpheus.messages.MessageMeta.df as list.
Methods
copy_meta_ranges
(ranges[, mask])Perform a copy of the underlying dataframe for the given
ranges
of rows.copy_ranges
(ranges[, num_selected_rows])Perform a copy of the current message instance for the given
ranges
of rows.get_input
(name)Get input stored in the TensorMemory container.
get_meta
([columns])Return column values from
morpheus.pipeline.messages.MessageMeta.df
.get_meta_list
([col_name])Return a column values from morpheus.messages.MessageMeta.df as a list.
get_slice
(start, stop)Returns sliced batches based on offsets supplied.
set_meta
(columns, value)Set column values to
morpheus.pipelines.messages.MessageMeta.df
.- copy_meta_ranges(ranges, mask=None)[source]
Perform a copy of the underlying dataframe for the given
ranges
of rows.- Parameters
- rangestyping.List[typing.Tuple[int, int]]
Rows to include in the copy in the form of
[(`start_row
,stop_row
),…]` Thestop_row
isn’t included. For example to copy rows 1-2 & 5-7ranges=[(1, 3), (5, 8)]
- masktyping.Union[None, cupy.ndarray, numpy.ndarray]
Optionally specify rows as a cupy array (when using cudf Dataframes) or a numpy array (when using pandas Dataframes) of booleans. When not-None
ranges
will be ignored. This is useful as an optimization as this avoids needing to generate the mask on it’s own.
- Returns
Dataframe
- copy_ranges(ranges, num_selected_rows=None)[source]
Perform a copy of the current message instance for the given
ranges
of rows.- Parameters
- rangestyping.List[typing.Tuple[int, int]]
Rows to include in the copy in the form of
[(`start_row
,stop_row
),…]` Thestop_row
isn’t included. For example to copy rows 1-2 & 5-7ranges=[(1, 3), (5, 8)]
- num_selected_rowstyping.Union[None, int]
Optional specify the number of rows selected by
ranges
, otherwise this is computed by the result.
- Returns
MultiMessage
- get_input(name)[source]
Get input stored in the TensorMemory container.
- Parameters
- namestr
Input key name.
- Returns
- cupy.ndarray
Inference input.
- get_meta(columns=None)[source]
Return column values from
morpheus.pipeline.messages.MessageMeta.df
.- Parameters
- columnstyping.Union[None, str, typing.List[str]]
Input column names. Returns all columns if
None
is specified. When a string is passed, aSeries
is returned. Otherwise aDataframe
is returned.
- Returns
- Series or Dataframe
Column values from the dataframe.
- get_meta_list(col_name=None)[source]
Return a column values from morpheus.messages.MessageMeta.df as a list.
- Parameters
- col_namestr
Column name in the dataframe.
- Returns
- List[str]
Column values from the dataframe.
- get_slice(start, stop)[source]
Returns sliced batches based on offsets supplied. Automatically calculates the correct
mess_offset
andmess_count
.- Parameters
- startint
Start offset address.
- stopint
Stop offset address.
- Returns
MultiInferenceMessage
A new
MultiInferenceMessage
with sliced offset and count.
- property id: List[int]
Returns ID column values from
morpheus.pipeline.messages.MessageMeta.df
as list.- Returns
- List[int]
ID column values from the dataframe as list.
- property id_col
Returns ID column values from
morpheus.pipeline.messages.MessageMeta.df
.- Returns
- pandas.Series
ID column values from the dataframe.
- property inputs
Get inputs stored in the TensorMemory container.
- Returns
- cupy.ndarray
Inference inputs.
- set_meta(columns, value)[source]
Set column values to
morpheus.pipelines.messages.MessageMeta.df
.- Parameters
- columnstyping.Union[None, str, typing.List[str]]
Input column names. Sets the value for the corresponding column names. If
None
is specified, all columns will be used. If the column does not exist, a new one will be created.- valueAny
Value to apply to the specified columns. If a single value is passed, it will be broadcast to all rows. If a
Series
orDataframe
is passed, rows will be matched by index.
- property timestamp: List[int]
Returns timestamp column values from morpheus.messages.MessageMeta.df as list.
- Returns
- List[int]
Timestamp column values from the dataframe as list.