TensorRT 8.6.1
nvinfer1::ITensor Class Reference

A tensor in a network definition. More...

#include <NvInfer.h>

Inheritance diagram for nvinfer1::ITensor:
nvinfer1::INoCopy

Public Member Functions

void setName (char const *name) noexcept
 Set the tensor name. More...
 
char const * getName () const noexcept
 Get the tensor name. More...
 
void setDimensions (Dims dimensions) noexcept
 Set the dimensions of a tensor. More...
 
Dims getDimensions () const noexcept
 Get the dimensions of a tensor. More...
 
void setType (DataType type) noexcept
 Set the data type of a tensor. More...
 
DataType getType () const noexcept
 Get the data type of a tensor. More...
 
bool setDynamicRange (float min, float max) noexcept
 Set dynamic range for the tensor. More...
 
bool isNetworkInput () const noexcept
 Whether the tensor is a network input. More...
 
bool isNetworkOutput () const noexcept
 Whether the tensor is a network output. More...
 
void setBroadcastAcrossBatch (bool broadcastAcrossBatch) noexcept
 Set whether to enable broadcast of tensor across the batch. More...
 
bool getBroadcastAcrossBatch () const noexcept
 Check if tensor is broadcast across the batch. More...
 
TensorLocation getLocation () const noexcept
 Get the storage location of a tensor. More...
 
void setLocation (TensorLocation location) noexcept
 Set the storage location of a tensor. More...
 
bool dynamicRangeIsSet () const noexcept
 Query whether dynamic range is set. More...
 
void resetDynamicRange () noexcept
 Undo effect of setDynamicRange. More...
 
float getDynamicRangeMin () const noexcept
 Get minimum of dynamic range. More...
 
float getDynamicRangeMax () const noexcept
 Get maximum of dynamic range. More...
 
void setAllowedFormats (TensorFormats formats) noexcept
 Set allowed formats for this tensor. By default all formats are allowed. Shape tensors (for which isShapeTensor() returns true) may only have row major linear format. More...
 
TensorFormats getAllowedFormats () const noexcept
 Get a bitmask of TensorFormat values that the tensor supports. For a shape tensor, only row major linear format is allowed. More...
 
bool isShapeTensor () const noexcept
 Whether the tensor is a shape tensor. More...
 
bool isExecutionTensor () const noexcept
 Whether the tensor is an execution tensor. More...
 
void setDimensionName (int32_t index, char const *name) noexcept
 Name a dimension of an input tensor. More...
 
char const * getDimensionName (int32_t index) const noexcept
 Get the name of an input dimension. More...
 

Protected Member Functions

virtual ~ITensor () noexcept=default
 
- Protected Member Functions inherited from nvinfer1::INoCopy
 INoCopy ()=default
 
virtual ~INoCopy ()=default
 
 INoCopy (INoCopy const &other)=delete
 
INoCopyoperator= (INoCopy const &other)=delete
 
 INoCopy (INoCopy &&other)=delete
 
INoCopyoperator= (INoCopy &&other)=delete
 

Protected Attributes

apiv::VTensor * mImpl
 

Detailed Description

A tensor in a network definition.

To remove a tensor from a network definition, use INetworkDefinition::removeTensor().

When using the DLA, the cumulative size of all Tensors that are not marked as Network Input or Output tensors, must be less than 1GB in size to fit into a single subgraph. If the build option kGPU_FALLBACK is specified, then multiple subgraphs can be created, with each subgraph limited to less than 1GB of internal tensors data.

Warning
The volume of the tensor must be less than 2^31 elements. If the tensor is a shape tensor, its volume must not exceed 64.
Do not inherit from this class, as doing so will break forward-compatibility of the API and ABI.

Constructor & Destructor Documentation

◆ ~ITensor()

virtual nvinfer1::ITensor::~ITensor ( )
protectedvirtualdefaultnoexcept

Member Function Documentation

◆ dynamicRangeIsSet()

bool nvinfer1::ITensor::dynamicRangeIsSet ( ) const
inlinenoexcept

Query whether dynamic range is set.

Returns
True if dynamic range is set, false otherwise.

◆ getAllowedFormats()

TensorFormats nvinfer1::ITensor::getAllowedFormats ( ) const
inlinenoexcept

Get a bitmask of TensorFormat values that the tensor supports. For a shape tensor, only row major linear format is allowed.

Returns
The value specified by setAllowedFormats or all possible formats.
See also
ITensor::setAllowedFormats()

◆ getBroadcastAcrossBatch()

bool nvinfer1::ITensor::getBroadcastAcrossBatch ( ) const
inlinenoexcept

Check if tensor is broadcast across the batch.

When a tensor is broadcast across a batch, it has the same value for every member in the batch. Memory is only allocated once for the single member. If the network is in explicit batch mode, this function returns true if the leading dimension is 1.

Returns
True if tensor is broadcast across the batch, false otherwise.
See also
setBroadcastAcrossBatch()

◆ getDimensionName()

char const * nvinfer1::ITensor::getDimensionName ( int32_t  index) const
inlinenoexcept

Get the name of an input dimension.

Parameters
indexindex of the dimension
Returns
The name of the input dimension, or nullptr if the dimension has no name. The name is a pointer to a null-terminated character sequence.
See also
setDimensionName()

◆ getDimensions()

Dims nvinfer1::ITensor::getDimensions ( ) const
inlinenoexcept

Get the dimensions of a tensor.

Returns
The dimensions of the tensor.
Warning
getDimensions() returns a -1 for dimensions that are derived from a wildcard dimension.
See also
setDimensions()

◆ getDynamicRangeMax()

float nvinfer1::ITensor::getDynamicRangeMax ( ) const
inlinenoexcept

Get maximum of dynamic range.

Returns
Maximum of dynamic range, or quiet NaN if range was not set.

◆ getDynamicRangeMin()

float nvinfer1::ITensor::getDynamicRangeMin ( ) const
inlinenoexcept

Get minimum of dynamic range.

Returns
Minimum of dynamic range, or quiet NaN if range was not set.

◆ getLocation()

TensorLocation nvinfer1::ITensor::getLocation ( ) const
inlinenoexcept

Get the storage location of a tensor.

Returns
The location of tensor data.
See also
setLocation()

◆ getName()

char const * nvinfer1::ITensor::getName ( ) const
inlinenoexcept

Get the tensor name.

Returns
The name as a null-terminated C-style string.
See also
setName()

◆ getType()

DataType nvinfer1::ITensor::getType ( ) const
inlinenoexcept

Get the data type of a tensor.

Returns
The data type of the tensor.
See also
setType()

◆ isExecutionTensor()

bool nvinfer1::ITensor::isExecutionTensor ( ) const
inlinenoexcept

Whether the tensor is an execution tensor.

Tensors are usually execution tensors. The exceptions are tensors used solely for shape calculations or whose contents not needed to compute the outputs.

The result of isExecutionTensor() is reliable only when network construction is complete. For example, if a partially built network has no path from a tensor to a network output, isExecutionTensor() returns false. Completing the path would cause it to become true.

If a tensor is an execution tensor and becomes an engine input or output, then ICudaEngine::isExecutionBinding will be true for that tensor.

A tensor with isShapeTensor() == false and isExecutionTensor() == false can still show up as an input to the engine if its dimensions are required. In that case, only its dimensions need to be set at runtime and a nullptr can be passed instead of a pointer to its contents.

◆ isNetworkInput()

bool nvinfer1::ITensor::isNetworkInput ( ) const
inlinenoexcept

Whether the tensor is a network input.

◆ isNetworkOutput()

bool nvinfer1::ITensor::isNetworkOutput ( ) const
inlinenoexcept

Whether the tensor is a network output.

◆ isShapeTensor()

bool nvinfer1::ITensor::isShapeTensor ( ) const
inlinenoexcept

Whether the tensor is a shape tensor.

A shape tensor is a tensor that is related to shape calculations. It must have type Int32, Bool, or Float, and its shape must be determinable at build time. Furthermore, it must be needed as a shape tensor, either marked as a network shape output via markOutputForShapes(), or as a layer input that is required to be a shape tensor, such as the second input to IShuffleLayer. Some layers are "polymorphic" in this respect. For example, the inputs to IElementWiseLayer must be shape tensors if the output is a shape tensor.

The TensorRT Developer Guide give the formal rules for what tensors are shape tensors.

The result of isShapeTensor() is reliable only when network construction is complete. For example, if a partially built network sums two tensors T1 and T2 to create tensor T3, and none are yet needed as shape tensors, isShapeTensor() returns false for all three tensors. Setting the second input of IShuffleLayer to be T3 would cause all three tensors to be shape tensors, because IShuffleLayer requires that its second optional input be a shape tensor, and IElementWiseLayer is "polymorphic".

If a tensor is a shape tensor and becomes an engine input or output, then ICudaEngine::isShapeBinding will be true for that tensor. Such a shape tensor must have type Int32.

It is possible for a tensor to be both a shape tensor and an execution tensor.

Returns
True if tensor is a shape tensor, false otherwise.
See also
INetworkDefinition::markOutputForShapes(), ICudaEngine::isShapeBinding()

◆ resetDynamicRange()

void nvinfer1::ITensor::resetDynamicRange ( )
inlinenoexcept

Undo effect of setDynamicRange.

◆ setAllowedFormats()

void nvinfer1::ITensor::setAllowedFormats ( TensorFormats  formats)
inlinenoexcept

Set allowed formats for this tensor. By default all formats are allowed. Shape tensors (for which isShapeTensor() returns true) may only have row major linear format.

When running network on DLA and the build option kGPU_FALLBACK is not specified, if DLA format(kCHW4 with Int8, kCHW4 with FP16, kCHW16 with FP16, kCHW32 with Int8) is set, the input format is treated as native DLA format with line stride requirement. Input/output binding with these format should have correct layout during inference.

Parameters
formatsA bitmask of TensorFormat values that are supported for this tensor.
See also
ITensor::getAllowedFormats()
TensorFormats

◆ setBroadcastAcrossBatch()

void nvinfer1::ITensor::setBroadcastAcrossBatch ( bool  broadcastAcrossBatch)
inlinenoexcept

Set whether to enable broadcast of tensor across the batch.

When a tensor is broadcast across a batch, it has the same value for every member in the batch. Memory is only allocated once for the single member.

This method is only valid for network input tensors, since the flags of layer output tensors are inferred based on layer inputs and parameters. If this state is modified for a tensor in the network, the states of all dependent tensors will be recomputed. If the tensor is for an explicit batch network, then this function does nothing.

Warning
The broadcast flag is ignored when using explicit batch network mode.
Parameters
broadcastAcrossBatchWhether to enable broadcast of tensor across the batch.
See also
getBroadcastAcrossBatch()

◆ setDimensionName()

void nvinfer1::ITensor::setDimensionName ( int32_t  index,
char const *  name 
)
inlinenoexcept

Name a dimension of an input tensor.

Associate a runtime dimension of an input tensor with a symbolic name. Dimensions with the same non-empty name must be equal at runtime. Knowing this equality for runtime dimensions may help the TensorRT optimizer. Both runtime and build-time dimensions can be named.

For example, setDimensionName(0, "n") associates the symbolic name "n" with the leading dimension.

This method copies the name string. If the function is called again, with the same index, it will overwrite the previous name. If nullptr is passed as name, it will clear the name of the dimension.

Parameters
indexindex of the dimension
nameof the dimension, as a pointer to a null-terminated character sequence.
Warning
The string name must be null-terminated, and be at most 4096 bytes including the terminator.
See also
getDimensionName()

◆ setDimensions()

void nvinfer1::ITensor::setDimensions ( Dims  dimensions)
inlinenoexcept

Set the dimensions of a tensor.

For a network input, the dimensions are assigned by the application. For a network output, the dimensions are computed based on the layer parameters and the inputs to the layer. If a tensor size or a parameter is modified in the network, the dimensions of all dependent tensors will be recomputed.

This call is only legal for network input tensors, since the dimensions of layer output tensors are inferred based on layer inputs and parameters. The volume must be less than 2^31 elements.

Parameters
dimensionsThe dimensions of the tensor.
See also
getDimensions()

◆ setDynamicRange()

bool nvinfer1::ITensor::setDynamicRange ( float  min,
float  max 
)
inlinenoexcept

Set dynamic range for the tensor.

Currently, only symmetric ranges are supported. Therefore, the larger of the absolute values of the provided bounds is used.

Returns
Whether the dynamic range was set successfully.

Requires that min and max be finite, and min <= max.

◆ setLocation()

void nvinfer1::ITensor::setLocation ( TensorLocation  location)
inlinenoexcept

Set the storage location of a tensor.

Parameters
locationthe location of tensor data

Only network input tensors for storing sequence lengths for RNNv2 are supported. Using host storage for layers that do not support it will generate errors at build time.

See also
getLocation()

◆ setName()

void nvinfer1::ITensor::setName ( char const *  name)
inlinenoexcept

Set the tensor name.

For a network input, the name is assigned by the application. For tensors which are layer outputs, a default name is assigned consisting of the layer name followed by the index of the output in brackets.

This method copies the name string.

Parameters
nameThe name.
Warning
The string name must be null-terminated, and be at most 4096 bytes including the terminator.
See also
getName()

◆ setType()

void nvinfer1::ITensor::setType ( DataType  type)
inlinenoexcept

Set the data type of a tensor.

Parameters
typeThe data type of the tensor.

The type is unchanged if the tensor is not a network input tensor, or marked as an output tensor or shape output tensor.

See also
getType()

Member Data Documentation

◆ mImpl

apiv::VTensor* nvinfer1::ITensor::mImpl
protected

The documentation for this class was generated from the following file: