Class MultiResponseProbsMessage

• Defined in File multi_response_probs.hpp

Base Type

• public morpheus::DerivedMultiMessage< MultiResponseProbsMessage, MultiResponseMessage > (Template Class DerivedMultiMessage)

class MultiResponseProbsMessage : public morpheus::DerivedMultiMessage<MultiResponseProbsMessage, MultiResponseMessage>

A stronger typed version of MultiResponseMessage that is used for inference workloads that return a probability array. Helps ensure the proper outputs are set and eases debugging

Public Functions

MultiResponseProbsMessage(const MultiResponseProbsMessage &other) = default

Default copy constructor.

MultiResponseProbsMessage(std::shared_ptr<MessageMeta> meta, TensorIndex mess_offset = 0, TensorIndex mess_count = -1, std::shared_ptr<TensorMemory> memory = nullptr, TensorIndex offset = 0, TensorIndex count = -1, std::string id_tensor_name = "seq_ids", std::string probs_tensor_name = "probs")

Construct a new Multi Response Probs Message object

Parameters

• meta – Holds a data table, in practice a cudf DataFrame, with the ability to return both Python and C++ representations of the table

• mess_offset – Offset into the metadata batch

• mess_count – Messages count

• memory – Holds the inference response probabilites as a tensor

• offset – Message offset in inference memory instance

• count – Message count in inference memory instance

• id_tensor_name – Name of the tensor that correlates tensor rows to message IDs

• probs_tensor_name – Name of the tensor that holds output probabilities

const TensorObject get_probs() const

Returns the probs (probabilities) output tensor.

Returns

const TensorObject

void set_probs(const TensorObject &probs)

Update the probs output tensor. Will halt on a fatal error if the probs output tensor does not exist.

Parameters

probs –