Class InferenceMemoryFIL#

Inheritance Relationships#

Base Type#

Class Documentation#

class InferenceMemoryFIL : public morpheus::InferenceMemory#

This is a container class for data that needs to be submitted to the inference server for FIL category usecases.

Public Functions

InferenceMemoryFIL(
TensorIndex count,
TensorObject &&input__0,
TensorObject &&seq_ids
)#

Construct a new Inference Memory FIL object.

Parameters:

  • count – : Message count in inference memory object

  • input__0 – : Inference input

  • seq_ids – : Ids used to index from an inference input to a message. Necessary since there can be more inference inputs than messages (i.e., if some messages get broken into multiple inference requests)

const TensorObject &get_input__0() const#

Returns the ‘input__0’ tensor, throws a std::runtime_error if it does not exist.

Throws:

std::runtime_error –

Returns:

const TensorObject&

const TensorObject &get_seq_ids() const#

Returns the ‘seq_ids’ tensor, throws a std::runtime_error if it does not exist.

Throws:

std::runtime_error –

Returns:

const TensorObject&

void set_input__0(TensorObject &&input__0)#

Sets a tensor named ‘input__0’.

Parameters:

input__0 –

Throws:

std::length_error – If the number of rows in input__0 does not match count.

void set_seq_ids(TensorObject &&seq_ids)#

Sets a tensor named ‘seq_ids’.

Parameters:

seq_ids –

Throws:

std::length_error – If the number of rows in seq_ids does not match count.