Class InferContext::Input¶
Defined in File request.h
Nested Relationships¶
This class is a nested type of Class InferContext.
Class Documentation¶
-
class
Input An input to the model.
Public Functions
-
virtual
~Input() = 0
-
virtual const std::string &
Name() const = 0 - Return
The name of the input.
-
virtual int64_t
ByteSize() const = 0 - Return
The size in bytes of this input. This is the size for one instance of the input, not the entire size of a batched input. When the byte-size is not known, for example for non-fixed-sized types like TYPE_STRING or for inputs with variable-size dimensions, this will return -1.
-
virtual size_t
TotalByteSize() const = 0 - Return
The size in bytes of entire batch of this input. For fixed-sized types this is just ByteSize() * batch-size, but for non-fixed-sized types like TYPE_STRING it is the only way to get the entire input size.
-
virtual DataType
DType() const = 0 - Return
The data-type of the input.
-
virtual bool
IsShapeTensor() const = 0 - Return
Whether the input is a shape tensor
-
virtual ModelInput::Format
Format() const = 0 - Return
The format of the input.
-
virtual const DimsList &
Dims() const = 0 - Return
The dimensions/shape of the input specified in the model configuration. Variable-size dimensions are reported as -1.
-
virtual Error
Reset() = 0 Prepare this input to receive new tensor values.
Forget any existing values that were set by previous calls to SetSharedMemory() or SetRaw().
- Return
Error object indicating success or failure.
-
virtual const std::vector<int64_t> &
Shape() const = 0 Get the shape for this input that was most recently set by SetShape.
- Return
The shape, or empty vector if SetShape has not been called.
-
virtual Error
SetShape(const std::vector<int64_t> &dims) = 0 Set the shape for this input.
The shape must be set for inputs that have variable-size dimensions and is optional for other inputs. The shape must be set before calling SetRaw or SetFromString.
- Return
Error object indicating success or failure.
- Parameters
dims: The dimensions of the shape.
-
virtual Error
SetRaw(const uint8_t *input, size_t input_byte_size) = 0 Set tensor values for this input from a byte array.
The array is not copied and so it must not be modified or destroyed until this input is no longer needed (that is until the Run() call(s) that use the input have completed). For batched inputs this function must be called batch-size times to provide all tensor values for a batch of this input. However, in case of shape tensor, this function must be called only once per batch.
- Return
Error object indicating success or failure.
- Parameters
input: The pointer to the array holding the tensor value.input_byte_size: The size of the array in bytes, must match the size expected by the input.
-
virtual Error
SetRaw(const std::vector<uint8_t> &input) = 0 Set tensor values for this input from a byte vector.
The vector is not copied and so it must not be modified or destroyed until this input is no longer needed (that is until the Run() call(s) that use the input have completed). For batched inputs this function must be called batch-size times to provide all tensor values for a batch of this input.
- Return
Error object indicating success or failure.
- Parameters
input: The vector holding tensor values.
-
virtual Error
SetFromString(const std::vector<std::string> &input) = 0 Set tensor values for this input from a vector or strings.
This method can only be used for tensors with STRING data-type. The strings are assigned in row-major order to the elements of the tensor. The strings are copied and so the ‘input’ does not need to be preserved as with SetRaw(). For batched inputs this function must be called batch-size times to provide all tensor values for a batch of this input.
- Return
Error object indicating success or failure.
- Parameters
input: The vector holding tensor string values.
Set tensor values for this input by reference into a shared memory region.
The values are not copied and so the shared memory region and its contents must not be modified or destroyed until this input is no longer needed (that is until the Run() call(s) that use the input have completed. This function must be called a single time for an input that is using shared memory. For batched inputs, the tensor values for the entire batch must be contiguous in a single shared memory region. Note: The options must be set using SetRunOptions before calling the SetSharedMemory function since the batch size is needed for validation.
- Return
Error object indicating success or failure.
- Parameters
name: The user-given name for the registered shared memory region where the tensor values for this input is stored.offset: The offset into the shared memory region upto the start of the input tensor values.byte_size: The size, in bytes of the input tensor data. Must match the size expected for the input batch.
-
virtual