Defined in File inference_memory_fil.hpp
Base Type
public morpheus::InferenceMemory(Class InferenceMemory)
-
class InferenceMemoryFIL : public morpheus::InferenceMemory
This is a container class for data that needs to be submitted to the inference server for FIL category usecases.
Public Functions
-
InferenceMemoryFIL(size_t count, TensorObject input__0, TensorObject seq_ids)
Construct a new Inference Memory FIL object.
- Parameters
count – : Message count in inference memory object
input__0 – : Inference input
seq_ids – : Ids used to index from an inference input to a message. Necessary since there can be more inference inputs than messages (i.e., if some messages get broken into multiple inference requests)
-
const TensorObject &get_input__0() const
Returns the ‘input__0’ tensor, throws a
std::runtime_errorif it does not exist.- Throws
std::runtime_error –
- Returns
const TensorObject&
-
const TensorObject &get_seq_ids() const
Returns the ‘seq_ids’ tensor, throws a
std::runtime_errorif it does not exist.- Throws
std::runtime_error –
- Returns
const TensorObject&
-
void set_input__0(TensorObject input_ids)
Sets a tensor named ‘input__0’.
- Parameters
input_ids –
- Throws
std::runtime_error –
std::runtime_error –
-
void set_seq_ids(TensorObject seq_ids)
Sets a tensor named ‘seq_ids’.
- Parameters
seq_ids –
- Throws
std::runtime_error –
-
InferenceMemoryFIL(size_t count, TensorObject input__0, TensorObject seq_ids)