Class InferInput

Class Documentation

class InferInput

An interface for InferInput object to describe the model input for inference.

Public Functions

const std::string &Name() const

Gets name of the associated input tensor.

Return

The name of the tensor.

const std::string &Datatype() const

Gets datatype of the associated input tensor.

Return

The datatype of the tensor.

const std::vector<int64_t> &Shape() const

Gets the shape of the input tensor.

Return

The shape of the tensor.

Error SetShape(const std::vector<int64_t> &dims)

Set the shape of input associated with this object.

Return

Error object indicating success or failure of the request.

Parameters

  • dims: the vector of dims representing the new shape of input.

Error Reset()

Prepare this input to receive new tensor values.

Forget any existing values that were set by previous calls to SetSharedMemory() or AppendRaw().

Return

Error object indicating success or failure.

Error AppendRaw(const std::vector<uint8_t> &input)

Append tensor values for this input from a byte vector.

The vector is not copied and so it must not be modified or destroyed until this input is no longer needed (that is until the Infer() call(s) that use the input have completed). Multiple calls can be made to this API to keep adding tensor data for this input. The data will be delivered in the order it was added.

Return

Error object indicating success or failure.

Parameters

  • input: The vector holding tensor values.

Error AppendRaw(const uint8_t *input, size_t input_byte_size)

Append tensor values for this input from a byte array.

The array is not copied and so it must not be modified or destroyed until this input is no longer needed (that is until the Infer() call(s) that use the input have completed). Multiple calls can be made to this API to keep adding tensor data for this input. The data will be delivered in the order it was added.

Return

Error object indicating success or failure.

Parameters

  • input: The pointer to the array holding the tensor value.

  • input_byte_size: The size of the array in bytes.

Error SetSharedMemory(const std::string &name, size_t byte_size, size_t offset = 0)

Set tensor values for this input by reference into a shared memory region.

The values are not copied and so the shared memory region and its contents must not be modified or destroyed until this input is no longer needed (that is until the Infer() call(s) that use the input have completed. This function must be called a single time for an input that is using shared memory. The entire tensor data required by this input must be contiguous in a single shared memory region.

Return

Error object indicating success or failure

Parameters

  • name: The user-given name for the registered shared memory region where the tensor values for this input is stored.

  • byte_size: The size, in bytes of the input tensor data. Must match the size expected for the input shape.

  • offset: The offset into the shared memory region upto the start of the input tensor values. The default value is 0.

bool IsSharedMemory() const

Return

true if this input is being provided in shared memory.

Error SharedMemoryInfo(std::string *name, size_t *byte_size, size_t *offset) const

Get information about the shared memory being used for this input.

Return

Error object indicating success or failure.

Parameters

  • name: Returns the name of the shared memory region.

  • byte_size: Returns the size, in bytes, of the shared memory region.

  • offset: Returns the offset within the shared memory region.

Error AppendFromString(const std::vector<std::string> &input)

Append tensor values for this input from a vector or strings.

This method can only be used for tensors with BYTES data-type. The strings are assigned in row-major order to the elements of the tensor. The strings are copied and so the ‘input’ does not need to be preserved as with AppendRaw(). Multiple calls can be made to this API to keep adding tensor data for this input. The data will be delivered in the order it was added.

Return

Error object indicating success or failure.

Parameters

  • input: The vector holding tensor string values.

Error ByteSize(size_t *byte_size) const

Gets the size of data added into this input in bytes.

Return

Error object indicating success or failure.

Parameters

  • byte_size: The size of data added in bytes.

Public Static Functions

static Error Create(InferInput **infer_input, const std::string &name, const std::vector<int64_t> &dims, const std::string &datatype)

Create a InferInput instance that describes a model input.

Return

Error object indicating success or failure.

Parameters

  • infer_input: Returns a new InferInput object.

  • name: The name of input whose data will be described by this object.

  • dims: The shape of the input.

  • datatype: The datatype of the input.