A simple record summarizing various properties of a network IO Tensor. More...

#include <NvInferSafeRuntime.h>

Public Attributes
AsciiChar const *	tensorName {nullptr}
	Name of the IO Tensor. More...

PhysicalDims	shape {-1, {}}

PhysicalDims	stride {-1, {}}
	Stride vector for each element of the IO Tensor. More...

DataType	dataType {DataType::kFLOAT}

uint64_t	bytesPerComponent {0U}
	The size of the tensor data type in bytes (4 for float and int32, 2 for half, 1 for int8) More...

uint64_t	componentsPerVector {1U}
	The vector length (in scalars) for a vectorized tensor, 1 if the tensor is not vectorized. More...

int64_t	vectorizedDim {-1}
	The dimension index along which the tensor is vectorized, -1 if the tensor is not vectorized. More...

uint64_t	sizeInBytes {0U}
	Total size in bytes for the IO Tensor. More...

TensorIOMode	ioMode {TensorIOMode::kNONE}
	Enum to denote whether the Tensor is for input or output. More...

MemoryPlacement	memPlacement {MemoryPlacement::kNONE}
	Enum to denote whether the Tensor memory is allocated on the GPU, CPU, or CPU_PINNED. More...

PhysicalDims	strideOrder {-1, {}}
	The order in which the dimensions are laid out in memory. More...

Dims	userShape {-1, {}}

Detailed Description

A simple record summarizing various properties of a network IO Tensor.

All properties needed to query, allocate, read, and write IO Tensors are recorded here. The user can determine the physical tensor shape and layout used by the compiler, which may be different from the original user-specified shape due to padding or tiling for vectorized layouts. For example, a user-specified shape [1,10,3,4] with a TensorFormat specification of NHWC8 would lead to tiling by 8 elements on the C dimension with the following descriptor fields:

User-specified properties (NHWC8): userShape : [1,10,3,4] (in NCHW order) componentsPerVector : 8 vectorizedDim : 1 (C)

Compiler-determined layout: shape : [1,2,8,3,4] (C dimension is split into two dims) stride : [192,8,1,64,16] strideOrder : [4,1,0,3,2] (Tiled C dimensions are changing fastest)

Member Data Documentation

◆ bytesPerComponent

uint64_t nvinfer2::safe::TensorDescriptor::bytesPerComponent {0U}

The size of the tensor data type in bytes (4 for float and int32, 2 for half, 1 for int8)

◆ componentsPerVector

uint64_t nvinfer2::safe::TensorDescriptor::componentsPerVector {1U}

The vector length (in scalars) for a vectorized tensor, 1 if the tensor is not vectorized.

◆ dataType

DataType nvinfer2::safe::TensorDescriptor::dataType {DataType::kFLOAT}

DataType enum for the IO Tensor.

Warning: The actual type of the tensor data must correspond to this DataType.

◆ ioMode

TensorIOMode nvinfer2::safe::TensorDescriptor::ioMode {TensorIOMode::kNONE}

Enum to denote whether the Tensor is for input or output.

◆ memPlacement

MemoryPlacement nvinfer2::safe::TensorDescriptor::memPlacement {MemoryPlacement::kNONE}

Enum to denote whether the Tensor memory is allocated on the GPU, CPU, or CPU_PINNED.

◆ shape

PhysicalDims nvinfer2::safe::TensorDescriptor::shape {-1, {}}

Static shape of the IO Tensor.

Note: The static shape will depend on the IO Profile selected.

◆ sizeInBytes

uint64_t nvinfer2::safe::TensorDescriptor::sizeInBytes {0U}

Total size in bytes for the IO Tensor.

◆ stride

PhysicalDims nvinfer2::safe::TensorDescriptor::stride {-1, {}}

Stride vector for each element of the IO Tensor.

◆ strideOrder

PhysicalDims nvinfer2::safe::TensorDescriptor::strideOrder {-1, {}}

The order in which the dimensions are laid out in memory.

◆ tensorName

AsciiChar const* nvinfer2::safe::TensorDescriptor::tensorName {nullptr}

Name of the IO Tensor.

◆ userShape

Dims nvinfer2::safe::TensorDescriptor::userShape {-1, {}}

The original user-specified shape of the tensor, provided as a reference if the tensor is vectorized. When a tensor is vectorized by size T along some dimension with size K, the vectorized dimension is split into two dimensions of sizes ceil(K/T) and T.

◆ vectorizedDim

int64_t nvinfer2::safe::TensorDescriptor::vectorizedDim {-1}

The dimension index along which the tensor is vectorized, -1 if the tensor is not vectorized.

The documentation for this struct was generated from the following file:

NvInferSafeRuntime.h

Public Attributes