Class MultiTensorMessage

• Defined in File multi_tensor.hpp

Base Type

• public morpheus::DerivedMultiMessage< MultiTensorMessage, MultiMessage > (Template Class DerivedMultiMessage)

class MultiTensorMessage : public morpheus::DerivedMultiMessage<MultiTensorMessage, MultiMessage>

Base class for MultiInferenceMessage & MultiResponseMessage Contains a pointer to an instance of TensorMemory along with an offset & count to those tensors

mess_offset & mess_count refer to the range of records in meta. offset & count refer to the range of records in TensorMemory

While TensorMemory can contain multiple tensors, it is a requirement that they are all of the same length and that element N in each tensor refers to the same record

Public Functions

MultiTensorMessage(const MultiTensorMessage &other) = default

Default copy constructor.

MultiTensorMessage(std::shared_ptr<MessageMeta> meta, TensorIndex mess_offset = 0, TensorIndex mess_count = -1, std::shared_ptr<TensorMemory> memory = nullptr, TensorIndex offset = 0, TensorIndex count = -1, std::string id_tensor_name = "seq_ids")

Construct a new Multi Tensor Message object.

Parameters

• meta – Holds a data table, in practice a cudf DataFrame, with the ability to return both Python and C++ representations of the table

• mess_offset – Offset into the metadata batch

• mess_count – Messages count

• memory – Shared pointer of a tensor memory

• offset – Message offset in tensor memory instance

• count – Message count in tensor memory instance

• id_tensor_name – Name of the tensor that correlates tensor rows to message IDs

const TensorObject get_tensor(const std::string &name) const

Returns a tensor with the given name.

Parameters

name –

Throws

std::runtime_error – If no tensor matching name exists

Returns

const TensorObject

TensorObject get_tensor(const std::string &name)

Returns a tensor with the given name.

Parameters

name –

Throws

std::runtime_error – If no tensor matching name exists

Returns

TensorObject

void set_tensor(const std::string &name, const TensorObject &value)

Update the value of a given tensor. The tensor must already exist, otherwise a runtime_error is thrown. error.

Parameters

• name –

• value –

Throws

std::runtime_error – If no tensor matching name exists

TensorObject get_id_tensor() const

Get the tensor that holds message ID information. Equivalent to get_tensor(id_tensor_name)

Returns

const TensorObject

Public Members

std::shared_ptr<morpheus::TensorMemory> memory

TensorIndex offset = {0}

TensorIndex count = {0}

std::string id_tensor_name

Protected Functions

virtual void get_slice_impl(std::shared_ptr<MultiMessage> new_message, TensorIndex start, TensorIndex stop) const override

Applies a slice of the attribures contained in new_message. Subclasses need only be concerned with their own attributes, and can safely avoid overriding this method if they don’t add any new attributes to their base.

Parameters

• new_message –

• start –

• stop –

virtual void copy_ranges_impl(std::shared_ptr<MultiMessage> new_message, const std::vector<RangeType> &ranges, TensorIndex num_selected_rows) const override

Similar to get_slice_impl, performs a copy of all attributes in new_message according to the rows specified by ranges. Subclasses need only be concerned with copying their own attributes, and can safely avoid overriding this method if they don’t add any new attributes to their base.

Parameters

• new_message –

• ranges –

• num_selected_rows –

std::shared_ptr<morpheus::TensorMemory> copy_input_ranges(const std::vector<RangeType> &ranges, TensorIndex num_selected_rows) const

TensorObject get_tensor_impl(const std::string &name) const