Message classes, which contain data being transfered between pipeline stages
- class ControlMessage
Methods
add_task
(self, task_type, task)config
(*args, **kwargs)Overloaded function.
copy
(self)get_metadata
(self, key)has_metadata
(self, key)has_task
(self, task_type)payload
(*args, **kwargs)Overloaded function.
remove_task
(self, task_type)set_metadata
(self, key, value)task_type
(*args, **kwargs)Overloaded function.
- add_task(self: morpheus._lib.messages.ControlMessage, task_type: str, task: dict) → None
- config(*args, **kwargs)
config(self: morpheus._lib.messages.ControlMessage) -> dict
config(self: morpheus._lib.messages.ControlMessage, config: dict) -> None
Overloaded function.
- get_metadata(self: morpheus._lib.messages.ControlMessage, key: str) → object
- has_metadata(self: morpheus._lib.messages.ControlMessage, key: str) → bool
- has_task(self: morpheus._lib.messages.ControlMessage, task_type: str) → bool
- payload(*args, **kwargs)
payload(self: morpheus._lib.messages.ControlMessage) -> morpheus._lib.messages.MessageMeta
payload(self: morpheus._lib.messages.ControlMessage, arg0: morpheus._lib.messages.MessageMeta) -> None
Overloaded function.
- remove_task(self: morpheus._lib.messages.ControlMessage, task_type: str) → dict
- set_metadata(self: morpheus._lib.messages.ControlMessage, key: str, value: object) → None
- task_type(*args, **kwargs)
task_type(self: morpheus._lib.messages.ControlMessage) -> morpheus._lib.messages.ControlMessageType
task_type(self: morpheus._lib.messages.ControlMessage, task_type: morpheus._lib.messages.ControlMessageType) -> None
Overloaded function.
- class DataLoaderRegistry
Methods
contains
(name)list
()register_loader
(name, loader[, throw_if_exists])unregister_loader
(name[, throw_if_not_exists])- static list() → List[str]
- static register_loader(name: str, loader: Callable[[morpheus._lib.messages.ControlMessage, dict], morpheus._lib.messages.ControlMessage], throw_if_exists: bool = True) → None
- class InferenceMemory(*, count=None, tensors=None)[source]
This is a base container class for data that will be used for inference stages. This class is designed to hold generic tensor data in cupy arrays.
- Attributes
- tensor_names
Methods
get_input
(name)Get the tensor stored in the container identified by
name
.get_tensor
(name)Get the Tensor stored in the container identified by
name
.get_tensors
()Get the tensors contained by this instance.
has_tensor
(name)Returns True if a tensor with the requested name exists in the tensors object
set_input
(name, tensor)Update the input tensor identified by
name
.set_tensor
(name, tensor)Update the tensor identified by
name
.set_tensors
(tensors)Overwrite the tensors stored by this instance.
- count: int
- get_input(name)[source]
Get the tensor stored in the container identified by
name
. Alias forInferenceMemory.get_tensor
.- Parameters
- namestr
Key used to do lookup in inputs dict of the container.
- Returns
- cupy.ndarray
Inputs corresponding to name.
- Raises
- KeyError
If input name does not exist in the container.
- set_input(name, tensor)[source]
Update the input tensor identified by
name
. Alias forInferenceMemory.set_tensor
- Parameters
- namestr
- tensorcupy.ndarray
Key used to do lookup in inputs dict of the container.
Tensor as a CuPy array.
- tensors: Dict[str, cupy._core.core.ndarray]
- class InferenceMemoryAE(*, count, input, seq_ids)[source]
This is a container class for data that needs to be submitted to the inference server for auto encoder usecases.
- Parameters
- inputcupy.ndarray
- seq_idscupy.ndarray
Inference input.
Ids used to index from an inference input to a message. Necessary since there can be more inference inputs than messages (i.e., if some messages get broken into multiple inference requests).
- Attributes
- input
- seq_ids
- tensor_names
Methods
get_input
(name)Get the tensor stored in the container identified by
name
.get_tensor
(name)Get the Tensor stored in the container identified by
name
.get_tensors
()Get the tensors contained by this instance.
has_tensor
(name)Returns True if a tensor with the requested name exists in the tensors object
set_input
(name, tensor)Update the input tensor identified by
name
.set_tensor
(name, tensor)Update the tensor identified by
name
.set_tensors
(tensors)Overwrite the tensors stored by this instance.
- input: dataclasses.InitVar[ndarray] = None
- seq_ids: dataclasses.InitVar[ndarray] = None
- class InferenceMemoryFIL(*, count, input__0, seq_ids)[source]
This is a container class for data that needs to be submitted to the inference server for FIL category usecases.
- Parameters
- input__0cupy.ndarray
- seq_idscupy.ndarray
Inference input.
Ids used to index from an inference input to a message. Necessary since there can be more inference inputs than messages (i.e., if some messages get broken into multiple inference requests).
- Attributes
- input__0
- seq_ids
- tensor_names
Methods
get_input
(name)Get the tensor stored in the container identified by
name
.get_tensor
(name)Get the Tensor stored in the container identified by
name
.get_tensors
()Get the tensors contained by this instance.
has_tensor
(name)Returns True if a tensor with the requested name exists in the tensors object
set_input
(name, tensor)Update the input tensor identified by
name
.set_tensor
(name, tensor)Update the tensor identified by
name
.set_tensors
(tensors)Overwrite the tensors stored by this instance.
- input__0: dataclasses.InitVar[ndarray] = None
- seq_ids: dataclasses.InitVar[ndarray] = None
- class InferenceMemoryNLP(*, count, input_ids, input_mask, seq_ids)[source]
This is a container class for data that needs to be submitted to the inference server for NLP category usecases.
- Parameters
- input_idscupy.ndarray
- input_maskcupy.ndarray
- seq_idscupy.ndarray
The token-ids for each string padded with 0s to max_length.
The mask for token-ids result where corresponding positions identify valid token-id values.
Ids used to index from an inference input to a message. Necessary since there can be more inference inputs than messages (i.e., if some messages get broken into multiple inference requests).
- Attributes
- input_ids
- input_mask
- seq_ids
- tensor_names
Methods
get_input
(name)Get the tensor stored in the container identified by
name
.get_tensor
(name)Get the Tensor stored in the container identified by
name
.get_tensors
()Get the tensors contained by this instance.
has_tensor
(name)Returns True if a tensor with the requested name exists in the tensors object
set_input
(name, tensor)Update the input tensor identified by
name
.set_tensor
(name, tensor)Update the tensor identified by
name
.set_tensors
(tensors)Overwrite the tensors stored by this instance.
- input_ids: dataclasses.InitVar[ndarray] = None
- input_mask: dataclasses.InitVar[ndarray] = None
- seq_ids: dataclasses.InitVar[ndarray] = None
- class MessageBase[source]
Base class for all messages. Returns a C++ implementation if CppConfig.get_should_use_cpp()
is True
and the
class has an associated C++ implementation (cpp_class
), returns the Python implementation for all others.
- class MessageMeta(df)[source]
This is a container class to hold batch deserialized messages metadata.
- Parameters
- dfpandas.DataFrame
Input rows in dataframe.
- Attributes
count
- df
Returns the number of messages in the batch.
Methods
Replaces the index in the underlying dataframe if the existing one is not unique and monotonic.
get_meta_range
(mess_offset, message_count[, ...])Return column values from
morpheus.pipeline.messages.MessageMeta.df
from the specified start offset until the message count.Returns True if the underlying DataFrame's index is unique and monotonic.
copy_dataframe
mutable_dataframe
- copy_dataframe()[source]
- property count: int
Returns the number of messages in the batch.
- Returns
- int
number of messages in the MessageMeta.df.
- property df: pandas.core.frame.DataFrame
- ensure_sliceable_index()[source]
Replaces the index in the underlying dataframe if the existing one is not unique and monotonic. The old index will be preserved in a column named
_index_{old_index.name}
. Ifhas_sliceable_index() == true
, this is a no-op.- Returns
- str
The name of the column with the old index or
None
if no changes were made
- get_meta_range(mess_offset, message_count, columns=None)[source]
Return column values from
morpheus.pipeline.messages.MessageMeta.df
from the specified start offset until the message count.- Parameters
- mess_offsetint
- mess_countint
- columnstyping.Union[None, str, typing.List[str]]
Offset into the metadata batch.
Messages count.
Input column names. Returns all columns if
None
is specified. When a string is passed, aSeries
is returned. Otherwise aDataframe
is returned.- Returns
- Series or Dataframe
Column values from the dataframe.
- has_sliceable_index()[source]
Returns True if the underlying DataFrame’s index is unique and monotonic. Sliceable indices have better performance since a range of rows can be specified by a start and stop index instead of requiring boolean masks.
- Returns
- bool
- mutable_dataframe()[source]
- class MultiAEMessage(*, meta, mess_offset=0, mess_count=- 1, model, train_scores_mean=0.0, train_scores_std=1.0)[source]
Subclass of
MultiMessage
specific to the AutoEncoder pipeline, which contains the model.- Attributes
id
id_col
timestamp
Returns ID column values from
morpheus.pipeline.messages.MessageMeta.df
as list.Returns ID column values from
morpheus.pipeline.messages.MessageMeta.df
.Returns timestamp column values from morpheus.messages.MessageMeta.df as list.
Methods
copy_meta_ranges
(ranges[, mask])Perform a copy of the underlying dataframe for the given
ranges
of rows.copy_ranges
(ranges)Perform a copy of the current message instance for the given
ranges
of rows.from_message
(message, *[, meta, ...])Creates a new instance of a derived class from
MultiMessage
using an existing message as the template.get_meta
([columns])Return column values from
morpheus.pipeline.messages.MessageMeta.df
.get_meta_list
([col_name])Return a column values from morpheus.messages.MessageMeta.df as a list.
get_slice
(start, stop)Returns sliced batches based on offsets supplied.
set_meta
(columns, value)Set column values to
morpheus.pipelines.messages.MessageMeta.df
.- train_scores_mean: float
- train_scores_std: float
- class MultiInferenceFILMessage(*, meta, mess_offset=0, mess_count=- 1, memory=None, offset=0, count=- 1)[source]
A stronger typed version of
MultiInferenceMessage
that is used for FIL workloads. Helps ensure the proper inputs are set and eases debugging.- Attributes
id
id_col
input__0
inputs
seq_ids
tensors
timestamp
Returns ID column values from
morpheus.pipeline.messages.MessageMeta.df
as list.Returns ID column values from
morpheus.pipeline.messages.MessageMeta.df
.Input to FIL model inference.
Get inputs stored in the InferenceMemory container.
Returns sequence ids, which are used to keep track of messages in a multi-threaded environment.
Get tensors stored in the TensorMemory container sliced according to
offset
andcount
.Returns timestamp column values from morpheus.messages.MessageMeta.df as list.
Methods
copy_meta_ranges
(ranges[, mask])Perform a copy of the underlying dataframe for the given
ranges
of rows.copy_ranges
(ranges)Perform a copy of the current message, dataframe and tensors for the given
ranges
of rows.copy_tensor_ranges
(ranges[, mask])Perform a copy of the underlying tensor tensors for the given
ranges
of rows.from_message
(message, *[, meta, ...])Creates a new instance of a derived class from
MultiMessage
using an existing message as the template.get_id_tensor
()Get the tensor that holds message ID information.
get_input
(name)Get input stored in the InferenceMemory container.
get_meta
([columns])Return column values from
morpheus.pipeline.messages.MessageMeta.df
.get_meta_list
([col_name])Return a column values from morpheus.messages.MessageMeta.df as a list.
get_slice
(start, stop)Perform a slice of the current message from
start
:stop
(excludingstop
)get_tensor
(name)Get tensor stored in the TensorMemory container.
set_meta
(columns, value)Set column values to
morpheus.pipelines.messages.MessageMeta.df
.- property input__0
Input to FIL model inference.
- Returns
- cupy.ndarray
Input data.
- required_tensors: ClassVar[List[str]] = ['input__0', 'seq_ids']
The tensor names that are required for instantiation
- property seq_ids
Returns sequence ids, which are used to keep track of messages in a multi-threaded environment.
- Returns
- cupy.ndarray
Sequence ids.
- class MultiInferenceMessage(*, meta, mess_offset=0, mess_count=- 1, memory=None, offset=0, count=- 1)[source]
This is a container class that holds the InferenceMemory container and the metadata of the data contained within it. Builds on top of the
MultiTensorMessage
class to add additional data for inferencing.This class requires two separate memory blocks for a batch. One for the message metadata (i.e., start time, IP address, etc.) and another for the raw inference inputs (i.e., input_ids, seq_ids). Since there can be more inference input requests than messages (This happens when some messages get broken into multiple inference requests) this class stores two different offset and count values.
mess_offset
andmess_count
refer to the offset and count in the message metadata batch andoffset
andcount
index into the inference batch data.- Attributes
id
id_col
inputs
tensors
timestamp
Returns ID column values from
morpheus.pipeline.messages.MessageMeta.df
as list.Returns ID column values from
morpheus.pipeline.messages.MessageMeta.df
.Get inputs stored in the InferenceMemory container.
Get tensors stored in the TensorMemory container sliced according to
offset
andcount
.Returns timestamp column values from morpheus.messages.MessageMeta.df as list.
Methods
copy_meta_ranges
(ranges[, mask])Perform a copy of the underlying dataframe for the given
ranges
of rows.copy_ranges
(ranges)Perform a copy of the current message, dataframe and tensors for the given
ranges
of rows.copy_tensor_ranges
(ranges[, mask])Perform a copy of the underlying tensor tensors for the given
ranges
of rows.from_message
(message, *[, meta, ...])Creates a new instance of a derived class from
MultiMessage
using an existing message as the template.get_id_tensor
()Get the tensor that holds message ID information.
get_input
(name)Get input stored in the InferenceMemory container.
get_meta
([columns])Return column values from
morpheus.pipeline.messages.MessageMeta.df
.get_meta_list
([col_name])Return a column values from morpheus.messages.MessageMeta.df as a list.
get_slice
(start, stop)Perform a slice of the current message from
start
:stop
(excludingstop
)get_tensor
(name)Get tensor stored in the TensorMemory container.
set_meta
(columns, value)Set column values to
morpheus.pipelines.messages.MessageMeta.df
.- count: int
- get_input(name)[source]
Get input stored in the InferenceMemory container. Alias for
MultiInferenceMessage.get_tensor
.- Parameters
- namestr
Input key name.
- Returns
- cupy.ndarray
Inference input.
- Raises
- KeyError
When no matching input tensor exists.
- property inputs
Get inputs stored in the InferenceMemory container. Alias for
MultiInferenceMessage.tensors
.- Returns
- cupy.ndarray
Inference inputs.
- offset: int
- class MultiInferenceNLPMessage(*, meta, mess_offset=0, mess_count=- 1, memory=None, offset=0, count=- 1)[source]
A stronger typed version of
MultiInferenceMessage
that is used for NLP workloads. Helps ensure the proper inputs are set and eases debugging.- Attributes
id
id_col
input_ids
input_mask
inputs
seq_ids
tensors
timestamp
Returns ID column values from
morpheus.pipeline.messages.MessageMeta.df
as list.Returns ID column values from
morpheus.pipeline.messages.MessageMeta.df
.Returns token-ids for each string padded with 0s to max_length.
Returns mask for token-ids result where corresponding positions identify valid token-id values.
Get inputs stored in the InferenceMemory container.
Returns sequence ids, which are used to keep track of which inference requests belong to each message.
Get tensors stored in the TensorMemory container sliced according to
offset
andcount
.Returns timestamp column values from morpheus.messages.MessageMeta.df as list.
Methods
copy_meta_ranges
(ranges[, mask])Perform a copy of the underlying dataframe for the given
ranges
of rows.copy_ranges
(ranges)Perform a copy of the current message, dataframe and tensors for the given
ranges
of rows.copy_tensor_ranges
(ranges[, mask])Perform a copy of the underlying tensor tensors for the given
ranges
of rows.from_message
(message, *[, meta, ...])Creates a new instance of a derived class from
MultiMessage
using an existing message as the template.get_id_tensor
()Get the tensor that holds message ID information.
get_input
(name)Get input stored in the InferenceMemory container.
get_meta
([columns])Return column values from
morpheus.pipeline.messages.MessageMeta.df
.get_meta_list
([col_name])Return a column values from morpheus.messages.MessageMeta.df as a list.
get_slice
(start, stop)Perform a slice of the current message from
start
:stop
(excludingstop
)get_tensor
(name)Get tensor stored in the TensorMemory container.
set_meta
(columns, value)Set column values to
morpheus.pipelines.messages.MessageMeta.df
.- property input_ids
Returns token-ids for each string padded with 0s to max_length.
- Returns
- cupy.ndarray
The token-ids for each string padded with 0s to max_length.
- property input_mask
Returns mask for token-ids result where corresponding positions identify valid token-id values.
- Returns
- cupy.ndarray
The mask for token-ids result where corresponding positions identify valid token-id values.
- required_tensors: ClassVar[List[str]] = ['input_ids', 'input_mask', 'seq_ids']
The tensor names that are required for instantiation
- property seq_ids
Returns sequence ids, which are used to keep track of which inference requests belong to each message.
- Returns
- cupy.ndarray
Ids used to index from an inference input to a message. Necessary since there can be more inference inputs than messages (i.e., if some messages get broken into multiple inference requests).
- class MultiMessage(*, meta, mess_offset=0, mess_count=- 1)[source]
This class holds data for multiple messages at a time. To avoid copying data for slicing operations, it holds a reference to a batched metadata object and stores the offset and count into that batch.
- Parameters
- meta
MessageMeta
- mess_offsetint
- mess_countint
Deserialized messages metadata for large batch.
Offset into the metadata batch.
Messages count.
- meta
- Attributes
Methods
copy_meta_ranges
(ranges[, mask])Perform a copy of the underlying dataframe for the given
ranges
of rows.copy_ranges
(ranges)Perform a copy of the current message instance for the given
ranges
of rows.from_message
(message, *[, meta, ...])Creates a new instance of a derived class from
MultiMessage
using an existing message as the template.get_meta
()Return column values from
morpheus.pipeline.messages.MessageMeta.df
.get_meta_list
([col_name])Return a column values from morpheus.messages.MessageMeta.df as a list.
get_slice
(start, stop)Returns sliced batches based on offsets supplied.
set_meta
(columns, value)Set column values to
morpheus.pipelines.messages.MessageMeta.df
.- copy_meta_ranges(ranges, mask=None)[source]
Perform a copy of the underlying dataframe for the given
ranges
of rows.- Parameters
- rangestyping.List[typing.Tuple[int, int]]
- masktyping.Union[None, cupy.ndarray, numpy.ndarray]
Rows to include in the copy in the form of
[(`start_row
,stop_row
),…]` Thestop_row
isn’t included. For example to copy rows 1-2 & 5-7ranges=[(1, 3), (5, 8)]
Optionally specify rows as a cupy array (when using cudf Dataframes) or a numpy array (when using pandas Dataframes) of booleans. When not-None
ranges
will be ignored. This is useful as an optimization as this avoids needing to generate the mask on it’s own.- Returns
Dataframe
- copy_ranges(ranges)[source]
Perform a copy of the current message instance for the given
ranges
of rows.- Parameters
- rangestyping.List[typing.Tuple[int, int]]
Rows to include in the copy in the form of
[(`start_row
,stop_row
),…]` Thestop_row
isn’t included. For example to copy rows 1-2 & 5-7ranges=[(1, 3), (5, 8)]
- Returns
- classmethod from_message(message, *, meta=None, mess_offset=- 1, mess_count=- 1, **kwargs)[source]
Creates a new instance of a derived class from
MultiMessage
using an existing message as the template. This is very useful when a new message needs to be created with a single change to an existingMessageMeta
.When creating the new message, all required arguments for the class specified by
cls
will be pulled frommessage
unless otherwise specified in theargs
orkwargs
. Special handling is performed depending on whether or not a newmeta
object is supplied. If one is supplied, the offset and count defaults will be 0 andmeta.count
respectively. Otherwise offset and count will be pulled from the inputmessage
.- Parameters
- clstyping.Type[Self]
- messageMultiMessage
- metaMessageMeta, optional
- mess_offsetint, optional
- mess_countint, optional
The class to create
An existing message to use as a template. Can be a base or derived from
cls
as long as all arguments can be pulled frommessage
or proveded inkwargs
A new
MessageMeta
to use, by default NoneA new
mess_offset
to use, by default -1A new
mess_count
to use, by default -1- Returns
- Self
A new instance of type
cls
- Raises
- ValueError
- AttributeError
If the incoming
message
is NoneIf some required arguments were not supplied by
kwargs
and could not be pulled frommessage
- get_meta() → cudf.core.dataframe.DataFrame[source]
- get_meta(columns: str) → cudf.core.series.Series
- get_meta(columns: List[str]) → cudf.core.dataframe.DataFrame
Return column values from
morpheus.pipeline.messages.MessageMeta.df
.- Parameters
- columnstyping.Union[None, str, typing.List[str]]
Input column names. Returns all columns if
None
is specified. When a string is passed, aSeries
is returned. Otherwise aDataframe
is returned.- Returns
- Series or Dataframe
Column values from the dataframe.
- get_meta_list(col_name=None)[source]
Return a column values from morpheus.messages.MessageMeta.df as a list.
- Parameters
- col_namestr
Column name in the dataframe.
- Returns
- List[str]
Column values from the dataframe.
- get_slice(start, stop)[source]
Returns sliced batches based on offsets supplied. Automatically calculates the correct
mess_offset
andmess_count
.- Parameters
- startint
- stopint
Start offset address.
Stop offset address.
- Returns
MultiInferenceMessage
A new
MultiInferenceMessage
with sliced offset and count.
- property id: List[int]
Returns ID column values from
morpheus.pipeline.messages.MessageMeta.df
as list.- Returns
- List[int]
ID column values from the dataframe as list.
- property id_col
Returns ID column values from
morpheus.pipeline.messages.MessageMeta.df
.- Returns
- pandas.Series
ID column values from the dataframe.
- mess_count: int
- mess_offset: int
- set_meta(columns, value)[source]
Set column values to
morpheus.pipelines.messages.MessageMeta.df
.- Parameters
- columnstyping.Union[None, str, typing.List[str]]
- valueAny
Input column names. Sets the value for the corresponding column names. If
None
is specified, all columns will be used. If the column does not exist, a new one will be created.Value to apply to the specified columns. If a single value is passed, it will be broadcast to all rows. If a
Series
orDataframe
is passed, rows will be matched by index.
- property timestamp: List[int]
Returns timestamp column values from morpheus.messages.MessageMeta.df as list.
- Returns
- List[int]
Timestamp column values from the dataframe as list.
- class MultiResponseAEMessage(*, meta, mess_offset=0, mess_count=- 1, memory=None, offset=0, count=- 1, id_tensor_name='seq_ids', probs_tensor_name='probs', user_id=None)[source]
A stronger typed version of
MultiResponseProbsMessage
that is used for inference workloads that return a probability array. Helps ensure the proper outputs are set and eases debugging.- Attributes
id
id_col
outputs
tensors
timestamp
- user_id
Returns ID column values from
morpheus.pipeline.messages.MessageMeta.df
as list.Returns ID column values from
morpheus.pipeline.messages.MessageMeta.df
.Get outputs stored in the TensorMemory container.
Get tensors stored in the TensorMemory container sliced according to
offset
andcount
.Returns timestamp column values from morpheus.messages.MessageMeta.df as list.
Methods
copy_meta_ranges
(ranges[, mask])Perform a copy of the underlying dataframe for the given
ranges
of rows.copy_ranges
(ranges)Perform a copy of the current message, dataframe and tensors for the given
ranges
of rows.copy_tensor_ranges
(ranges[, mask])Perform a copy of the underlying tensor tensors for the given
ranges
of rows.from_message
(message, *[, meta, ...])Creates a new instance of a derived class from
MultiMessage
using an existing message as the template.get_id_tensor
()Get the tensor that holds message ID information.
get_meta
([columns])Return column values from
morpheus.pipeline.messages.MessageMeta.df
.get_meta_list
([col_name])Return a column values from morpheus.messages.MessageMeta.df as a list.
get_output
(name)Get output stored in the TensorMemory container.
get_probs_tensor
()Get the tensor that holds output probabilities.
get_slice
(start, stop)Perform a slice of the current message from
start
:stop
(excludingstop
)get_tensor
(name)Get tensor stored in the TensorMemory container.
set_meta
(columns, value)Set column values to
morpheus.pipelines.messages.MessageMeta.df
.- user_id: str = None
- class MultiResponseMessage(*, meta, mess_offset=0, mess_count=- 1, memory=None, offset=0, count=- 1, id_tensor_name='seq_ids', probs_tensor_name='probs')[source]
This class contains several inference responses as well as the cooresponding message metadata.
- Attributes
id
id_col
outputs
tensors
timestamp
Returns ID column values from
morpheus.pipeline.messages.MessageMeta.df
as list.Returns ID column values from
morpheus.pipeline.messages.MessageMeta.df
.Get outputs stored in the TensorMemory container.
Get tensors stored in the TensorMemory container sliced according to
offset
andcount
.Returns timestamp column values from morpheus.messages.MessageMeta.df as list.
Methods
copy_meta_ranges
(ranges[, mask])Perform a copy of the underlying dataframe for the given
ranges
of rows.copy_ranges
(ranges)Perform a copy of the current message, dataframe and tensors for the given
ranges
of rows.copy_tensor_ranges
(ranges[, mask])Perform a copy of the underlying tensor tensors for the given
ranges
of rows.from_message
(message, *[, meta, ...])Creates a new instance of a derived class from
MultiMessage
using an existing message as the template.get_id_tensor
()Get the tensor that holds message ID information.
get_meta
([columns])Return column values from
morpheus.pipeline.messages.MessageMeta.df
.get_meta_list
([col_name])Return a column values from morpheus.messages.MessageMeta.df as a list.
get_output
(name)Get output stored in the TensorMemory container.
Get the tensor that holds output probabilities.
get_slice
(start, stop)Perform a slice of the current message from
start
:stop
(excludingstop
)get_tensor
(name)Get tensor stored in the TensorMemory container.
set_meta
(columns, value)Set column values to
morpheus.pipelines.messages.MessageMeta.df
.- get_output(name)[source]
Get output stored in the TensorMemory container. Alias for
MultiResponseMessage.get_tensor
.- Parameters
- namestr
Output key name.
- Returns
- cupy.ndarray
Inference output.
- get_probs_tensor()[source]
Get the tensor that holds output probabilities. Equivalent to
get_tensor(probs_tensor_name)
- Returns
- cupy.ndarray
The probabilities tensor
- Raises
- KeyError
If
self.probs_tensor_name
is not found in the tensors
- property outputs
Get outputs stored in the TensorMemory container. Alias for
MultiResponseMessage.tensors
.- Returns
- cupy.ndarray
Inference outputs.
- probs_tensor_name: ClassVar[str] = 'probs'
Name of the tensor that holds output probabilities
- class MultiResponseProbsMessage(*args, **kwargs)[source]
A stronger typed version of
MultiResponseMessage
that is used for inference workloads that return a probability array. Helps ensure the proper outputs are set and eases debugging.- Attributes
id
id_col
outputs
probs
tensors
timestamp
Returns ID column values from
morpheus.pipeline.messages.MessageMeta.df
as list.Returns ID column values from
morpheus.pipeline.messages.MessageMeta.df
.Get outputs stored in the TensorMemory container.
Probabilities of prediction.
Get tensors stored in the TensorMemory container sliced according to
offset
andcount
.Returns timestamp column values from morpheus.messages.MessageMeta.df as list.
Methods
copy_meta_ranges
(ranges[, mask])Perform a copy of the underlying dataframe for the given
ranges
of rows.copy_ranges
(ranges)Perform a copy of the current message, dataframe and tensors for the given
ranges
of rows.copy_tensor_ranges
(ranges[, mask])Perform a copy of the underlying tensor tensors for the given
ranges
of rows.from_message
(message, *[, meta, ...])Creates a new instance of a derived class from
MultiMessage
using an existing message as the template.get_id_tensor
()Get the tensor that holds message ID information.
get_meta
([columns])Return column values from
morpheus.pipeline.messages.MessageMeta.df
.get_meta_list
([col_name])Return a column values from morpheus.messages.MessageMeta.df as a list.
get_output
(name)Get output stored in the TensorMemory container.
get_probs_tensor
()Get the tensor that holds output probabilities.
get_slice
(start, stop)Perform a slice of the current message from
start
:stop
(excludingstop
)get_tensor
(name)Get tensor stored in the TensorMemory container.
set_meta
(columns, value)Set column values to
morpheus.pipelines.messages.MessageMeta.df
.- property probs
Probabilities of prediction.
- Returns
- cupy.ndarray
probabilities
- required_tensors: ClassVar[List[str]] = ['probs']
The tensor names that are required for instantiation
- class MultiTensorMessage(*, meta, mess_offset=0, mess_count=- 1, memory, offset=0, count=- 1, id_tensor_name='seq_ids')[source]
This class contains several inference responses as well as the cooresponding message metadata.
- Parameters
- memory
TensorMemory
- offsetint
- countint
Container holding generic tensor data in cupy arrays
Offset of each message into the
TensorMemory
block.Number of rows in the
TensorMemory
block.- memory
- Attributes
id
id_col
tensors
timestamp
Returns ID column values from
morpheus.pipeline.messages.MessageMeta.df
as list.Returns ID column values from
morpheus.pipeline.messages.MessageMeta.df
.Get tensors stored in the TensorMemory container sliced according to
offset
andcount
.Returns timestamp column values from morpheus.messages.MessageMeta.df as list.
Methods
copy_meta_ranges
(ranges[, mask])Perform a copy of the underlying dataframe for the given
ranges
of rows.copy_ranges
(ranges)Perform a copy of the current message, dataframe and tensors for the given
ranges
of rows.copy_tensor_ranges
(ranges[, mask])Perform a copy of the underlying tensor tensors for the given
ranges
of rows.from_message
(message, *[, meta, ...])Creates a new instance of a derived class from
MultiMessage
using an existing message as the template.Get the tensor that holds message ID information.
get_meta
([columns])Return column values from
morpheus.pipeline.messages.MessageMeta.df
.get_meta_list
([col_name])Return a column values from morpheus.messages.MessageMeta.df as a list.
get_slice
(start, stop)Perform a slice of the current message from
start
:stop
(excludingstop
)get_tensor
(name)Get tensor stored in the TensorMemory container.
set_meta
(columns, value)Set column values to
morpheus.pipelines.messages.MessageMeta.df
.- copy_ranges(ranges)[source]
Perform a copy of the current message, dataframe and tensors for the given
ranges
of rows.- Parameters
- rangestyping.List[typing.Tuple[int, int]]
- ——-
- `MultiTensorMessage`
Rows to include in the copy in the form of
[(`start_row
,stop_row
),…]` Thestop_row
isn’t included. For example to copy rows 1-2 & 5-7ranges=[(1, 3), (5, 8)]
- copy_tensor_ranges(ranges, mask=None)[source]
Perform a copy of the underlying tensor tensors for the given
ranges
of rows.- Parameters
- rangestyping.List[typing.Tuple[int, int]]
- masktyping.Union[None, cupy.ndarray, numpy.ndarray]
Rows to include in the copy in the form of
[(`start_row
,stop_row
),…]` Thestop_row
isn’t included. For example to copy rows 1-2 & 5-7ranges=[(1, 3), (5, 8)]
Optionally specify rows as a cupy array (when using cudf Dataframes) or a numpy array (when using pandas Dataframes) of booleans. When not-None
ranges
will be ignored. This is useful as an optimization as this avoids needing to generate the mask on it’s own.- Returns
- typing.Dict[str, cupy.ndarray]
- count: int
- classmethod from_message(message, *, meta=None, mess_offset=- 1, mess_count=- 1, memory=None, offset=- 1, count=- 1, **kwargs)[source]
Creates a new instance of a derived class from
MultiMessage
using an existing message as the template. This is very useful when a new message needs to be created with a single change to an existingMessageMeta
.When creating the new message, all required arguments for the class specified by
cls
will be pulled frommessage
unless otherwise specified in theargs
orkwargs
. Special handling is performed depending on whether or not a newmeta
object is supplied. If one is supplied, the offset and count defaults will be 0 andmeta.count
respectively. Otherwise offset and count will be pulled from the inputmessage
.- Parameters
- clstyping.Type[Self]
- messageMultiMessage
- metaMessageMeta, optional
- mess_offsetint, optional
- mess_countint, optional
- memoryTensorMemory, optional
- offsetint, optional
- countint, optional
The class to create
An existing message to use as a template. Can be a base or derived from
cls
as long as all arguments can be pulled frommessage
or proveded inkwargs
A new
MessageMeta
to use, by default NoneA new
mess_offset
to use, by default -1A new
mess_count
to use, by default -1A new
TensorMemory
to use. If supplied,offset
andcount
default to0
andmemory.count
respectively. By default NoneA new
offset
to use, by default -1A new
count
to use, by default -1- Returns
- Self
A new instance of type
cls
- Raises
- ValueError
If the incoming
message
is None
- get_id_tensor()[source]
Get the tensor that holds message ID information. Equivalent to
get_tensor(id_tensor_name)
- Returns
- cupy.ndarray
Array containing the ID information
- Raises
- KeyError
If
self.id_tensor_name
is not found in the tensors
- get_slice(start, stop)[source]
Perform a slice of the current message from
start
:stop
(excludingstop
)For example to slice from rows 1-3 use
m.get_slice(1, 4)
. The returnedMultiTensorMessage
will contain references to the same underlying Dataframe and tensor tensors, and this calling this method is reletively low cost compared toMultiTensorMessage.copy_ranges
- Parameters
- startint
- stopint
- ——-
- `MultiTensorMessage`
Starting row of the slice
Stop of the slice
- get_tensor(name)[source]
Get tensor stored in the TensorMemory container.
- Parameters
- namestr
tensor key name.
- Returns
- cupy.ndarray
Inference tensor.
- id_tensor_name: ClassVar[str] = 'seq_ids'
Name of the tensor that correlates tensor rows to message IDs
- offset: int
- required_tensors: ClassVar[List[str]] = []
The tensor names that are required for instantiation
- class ResponseMemory(*args, **kwargs)[source]
Output memory block holding the results of inference.
- Attributes
- tensor_names
Methods
get_output
(name)Get the Tensor stored in the container identified by
name
.get_tensor
(name)Get the Tensor stored in the container identified by
name
.get_tensors
()Get the tensors contained by this instance.
has_tensor
(name)Returns True if a tensor with the requested name exists in the tensors object
set_output
(name, tensor)Update the output tensor identified by
name
.set_tensor
(name, tensor)Update the tensor identified by
name
.set_tensors
(tensors)Overwrite the tensors stored by this instance.
- count: int
- get_output(name)[source]
Get the Tensor stored in the container identified by
name
. Alias forResponseMemory.get_tensor
.- Parameters
- namestr
Key used to do lookup in tensors dict of message container.
- Returns
- cupy.ndarray
Tensors corresponding to name.
- Raises
- KeyError
If output name does not exist in message container.
- set_output(name, tensor)[source]
Update the output tensor identified by
name
. Alias forResponseMemory.set_tensor
- Parameters
- namestr
- tensorcupy.ndarray
Key used to do lookup in tensors dict of the container.
Tensor as a CuPy array.
- Raises
- ValueError
If the number of rows in
tensor
does not matchcount
- tensors: Dict[str, cupy._core.core.ndarray]
- class ResponseMemoryAE(*args, **kwargs)[source]
Subclass of
ResponseMemory
specific to the AutoEncoder pipeline.- Parameters
- probscupy.ndarray
- user_idstr
- explain_dfpd.Dataframe
Probabilities tensor
User id the inference was performed against.
Explainability Dataframe, for each feature a column will exist with a name in the form of:
{feature}_z_loss
containing the loss z-score along withmax_abs_z
andmean_abs_z
columns- Attributes
- explain_df
- probs
- tensor_names
Methods
get_output
(name)Get the Tensor stored in the container identified by
name
.get_tensor
(name)Get the Tensor stored in the container identified by
name
.get_tensors
()Get the tensors contained by this instance.
has_tensor
(name)Returns True if a tensor with the requested name exists in the tensors object
set_output
(name, tensor)Update the output tensor identified by
name
.set_tensor
(name, tensor)Update the tensor identified by
name
.set_tensors
(tensors)Overwrite the tensors stored by this instance.
- explain_df = None
- probs: dataclasses.InitVar[ndarray] = None
- user_id = ''
- class ResponseMemoryProbs(*args, **kwargs)[source]
Subclass of
ResponseMemory
containng an output tensor named ‘probs’.- Parameters
- probscupy.ndarray
Probabilities tensor
- Attributes
- probs
- tensor_names
Methods
get_output
(name)Get the Tensor stored in the container identified by
name
.get_tensor
(name)Get the Tensor stored in the container identified by
name
.get_tensors
()Get the tensors contained by this instance.
has_tensor
(name)Returns True if a tensor with the requested name exists in the tensors object
set_output
(name, tensor)Update the output tensor identified by
name
.set_tensor
(name, tensor)Update the tensor identified by
name
.set_tensors
(tensors)Overwrite the tensors stored by this instance.
- probs: dataclasses.InitVar[ndarray] = None
- class TensorMemory(*, count=None, tensors=None)[source]
This is a base container class for data that will be used for inference stages. This class is designed to hold generic tensor data in cupy arrays.
- Parameters
- countint
- tensorstyping.Dict[str, cupy.ndarray]
Length of each tensor contained in
tensors
.Collection of tensors uniquely identified by a name.
- Attributes
- tensor_names
Methods
get_tensor
(name)Get the Tensor stored in the container identified by
name
.Get the tensors contained by this instance.
has_tensor
(name)Returns True if a tensor with the requested name exists in the tensors object
set_tensor
(name, tensor)Update the tensor identified by
name
.set_tensors
(tensors)Overwrite the tensors stored by this instance.
- count: int
- get_tensor(name)[source]
Get the Tensor stored in the container identified by
name
.- Parameters
- namestr
Tensor key name.
- Returns
- cupy.ndarray
Tensor.
- Raises
- KeyError
If tensor name does not exist in the container.
- get_tensors()[source]
Get the tensors contained by this instance. It is important to note that when C++ execution is enabled the returned tensors will be a Python copy of the tensors stored in the C++ object. As such any changes made to the tensors will need to be updated with a call to
set_tensors
.- Returns
- typing.Dict[str, cp.ndarray]
- has_tensor(name)[source]
Returns True if a tensor with the requested name exists in the tensors object
- Parameters
- namestr
Name to lookup
- Returns
- bool
True if the tensor was found
- set_tensor(name, tensor)[source]
Update the tensor identified by
name
.- Parameters
- namestr
- tensorcupy.ndarray
Tensor key name.
Tensor as a CuPy array.
- Raises
- ValueError
If the number of rows in
tensor
does not matchcount
- set_tensors(tensors)[source]
Overwrite the tensors stored by this instance. If the length of the tensors has changed, then the
count
property should also be updated.- Parameters
- tensorstyping.Dict[str, cupy.ndarray]
Collection of tensors uniquely identified by a name.
- property tensor_names: List[str]
- tensors: Dict[str, cupy._core.core.ndarray]
- class UserMessageMeta(df, user_id)[source]
This class extends MessageMeta to also hold userid corresponding to batched metadata.
- Parameters
- dfpandas.DataFrame
- user_idstr
Input rows in dataframe.
User id.
- Attributes
count
- df
Returns the number of messages in the batch.
Methods
ensure_sliceable_index
()Replaces the index in the underlying dataframe if the existing one is not unique and monotonic.
get_meta_range
(mess_offset, message_count[, ...])Return column values from
morpheus.pipeline.messages.MessageMeta.df
from the specified start offset until the message count.has_sliceable_index
()Returns True if the underlying DataFrame's index is unique and monotonic.
copy_dataframe
mutable_dataframe
- user_id: str
Modules
Memory classes |
|