Class MultiResponseProbsMessage

Base Type

class MultiResponseProbsMessage : public morpheus::DerivedMultiMessage<MultiResponseProbsMessage, MultiResponseMessage>

A stronger typed version of MultiResponseMessage that is used for inference workloads that return a probability array. Helps ensure the proper outputs are set and eases debugging

Public Functions

MultiResponseProbsMessage(const MultiResponseProbsMessage &other) = default

Default copy constructor.

MultiResponseProbsMessage(std::shared_ptr<morpheus::MessageMeta> meta, size_t mess_offset, size_t mess_count, std::shared_ptr<morpheus::ResponseMemoryProbs> memory, size_t offset, size_t count)

Construct a new Multi Response Probs Message object

  • meta – Holds a data table, in practice a cudf DataFrame, with the ability to return both Python and C++ representations of the table

  • mess_offset – Offset into the metadata batch

  • mess_count – Messages count

  • memory – Holds the inference response probabilites as a tensor

  • offset – Message offset in inference memory instance

  • count – Message count in inference memory instance

const TensorObject get_probs() const

Returns the probs (probabilities) output tensor.


const TensorObject

void set_probs(const TensorObject &probs)

Update the probs output tensor. Will halt on a fatal error if the probs output tensor does not exist.



© Copyright 2023, NVIDIA. Last updated on Feb 3, 2023.