Class InferContext::Options

Nested Relationships

This class is a nested type of Class InferContext.

Class Documentation

class Options

Run options to be applied to all subsequent Run() invocations.

Public Functions

virtual ~Options() = 0
virtual bool Flag(InferRequestHeader::Flag flag) const = 0

Get the value of a request flag being used for all subsequent inferences.

Cannot be used with FLAG_NONE.

Return

The true/false value currently set for the flag. If ‘flag’ is FLAG_NONE then return false.

Parameters

  • flag: The flag to get the value for.

virtual void SetFlag(InferRequestHeader::Flag flag, bool value) = 0

Set a request flag to be used for all subsequent inferences.

Parameters

  • flag: The flag to set. Cannot be used with FLAG_NONE.

  • value: The true/false value to set for the flag. If ‘flag’ is FLAG_NONE then do nothing.

virtual uint32_t Flags() const = 0

Get the value of all request flags being used for all subsequent inferences.

Return

The bitwise-or of flag values as a single uint32_t value.

virtual void SetFlags(uint32_t flags) = 0

Set all request flags to be used for all subsequent inferences.

Parameters

  • flags: The bitwise-or of flag values to set.

virtual size_t BatchSize() const = 0

Return

The batch size to use for all subsequent inferences.

virtual void SetBatchSize(size_t batch_size) = 0

Set the batch size to use for all subsequent inferences.

Parameters

  • batch_size: The batch size.

virtual CorrelationID CorrelationId() const = 0

Return

The correlation id for the inferences.

virtual void SetCorrelationId(CorrelationID correlation_id) = 0

Set the correlation id to use for subsequent inferences.

Parameters

  • correlation_id: The value of correlation id to be set. If non-zero this correlation ID overrides the context’s correlation ID for all subsequent inference requests, else the context retains its current correlation ID.

virtual Error AddRawResult(const std::shared_ptr<InferContext::Output> &output) = 0

Add ‘output’ to the list of requested RAW results.

Run() will return the output’s full tensor as a result.

Return

Error object indicating success or failure.

Parameters

  • output: The output.

virtual Error AddClassResult(const std::shared_ptr<InferContext::Output> &output, uint64_t k) = 0

Add ‘output’ to the list of requested CLASS results.

Run() will return the highest ‘k’ values of ‘output’ as a result.

Return

Error object indicating success or failure.

Parameters

  • output: The output.

  • k: Set how many class results to return for the output.

virtual Error AddSharedMemoryResult(const std::shared_ptr<InferContext::Output> &output, const std::string &name, size_t offset, size_t byte_size) = 0

Indicate that the result values for this output should be placed in a shared memory region instead of being returned in the inference response.

The shared memory region must not be modified or destroyed written the output completely). For batched outputs, all tensor values are copied into a contiguous space in a single shared memory region.

Return

Error object indicating success or failure.

Parameters

  • output: The output.

  • name: The user-given name for the registered shared memory region where the tensor values for this output should be stored.

  • offset: The offset into the shared memory region upto the start of the output tensor values.

  • byte_size: The size, in bytes of the output tensor data. Must match the size expected by the output.

Public Static Functions

static Error Create(std::unique_ptr<Options> *options)

Create a new Options object with default values.

Return

Error object indicating success or failure.