Class MultiResponseMessage

• Defined in File multi_response.hpp

Base Type

• public morpheus::DerivedMultiMessage< MultiResponseMessage, MultiTensorMessage > (Template Class DerivedMultiMessage)

class MultiResponseMessage : public morpheus::DerivedMultiMessage<MultiResponseMessage, MultiTensorMessage>

This class is used to get or set the inference output from message containers derived from ResponseMemory.

Public Functions

MultiResponseMessage(const MultiResponseMessage &other) = default

Default copy constructor.

MultiResponseMessage(std::shared_ptr<MessageMeta> meta, TensorIndex mess_offset = 0, TensorIndex mess_count = -1, std::shared_ptr<TensorMemory> memory = nullptr, TensorIndex offset = 0, TensorIndex count = -1, std::string id_tensor_name = "seq_ids", std::string probs_tensor_name = "probs")

Construct a new Multi Response Message object.

Parameters

• meta – Holds a data table, in practice a cudf DataFrame, with the ability to return both Python and C++ representations of the table

• mess_offset – Offset into the metadata batch

• mess_count – Messages count

• memory – Shared pointer of a tensor memory

• offset – Message offset in inference memory instance

• count – Message count in inference memory instance

• id_tensor_name – Name of the tensor that correlates tensor rows to message IDs

• probs_tensor_name – Name of the tensor that holds output probabilities

const TensorObject get_output(const std::string &name) const

Returns the output tensor with the given name.

Parameters

name –

Throws

std::runtime_error – If no tensor matching name exists

Returns

const TensorObject

TensorObject get_output(const std::string &name)

Returns the output tensor with the given name.

Parameters

name –

Throws

std::runtime_error – If no tensor matching name exists

Returns

TensorObject

void set_output(const std::string &name, const TensorObject &value)

Update the value of a given output tensor. The tensor must already exist, otherwise this will halt on a fatal error.

Parameters

• name –

• value –

TensorObject get_probs_tensor() const

Get the tensor that holds output probabilities. Equivalent to get_tensor(probs_tensor_name)

Returns

const TensorObject

Public Members

std::string probs_tensor_name