TensorRT 8.6.0
nvinfer1::INetworkDefinition Class Reference

A network definition for input to the builder. More...

#include <NvInfer.h>

Inheritance diagram for nvinfer1::INetworkDefinition:
nvinfer1::INoCopy

Public Member Functions

virtual ~INetworkDefinition () noexcept=default
 
ITensoraddInput (char const *name, DataType type, Dims dimensions) noexcept
 Add an input tensor to the network. More...
 
void markOutput (ITensor &tensor) noexcept
 Mark a tensor as a network output. More...
 
TRT_DEPRECATED IConvolutionLayeraddConvolution (ITensor &input, int32_t nbOutputMaps, DimsHW kernelSize, Weights kernelWeights, Weights biasWeights) noexcept
 Add a convolution layer to the network. More...
 
TRT_DEPRECATED IFullyConnectedLayeraddFullyConnected (ITensor &input, int32_t nbOutputs, Weights kernelWeights, Weights biasWeights) noexcept
 Add a fully connected layer to the network. More...
 
IActivationLayeraddActivation (ITensor &input, ActivationType type) noexcept
 Add an activation layer to the network. More...
 
TRT_DEPRECATED IPoolingLayeraddPooling (ITensor &input, PoolingType type, DimsHW windowSize) noexcept
 Add a pooling layer to the network. More...
 
ILRNLayeraddLRN (ITensor &input, int32_t window, float alpha, float beta, float k) noexcept
 Add a LRN layer to the network. More...
 
IScaleLayeraddScale (ITensor &input, ScaleMode mode, Weights shift, Weights scale, Weights power) noexcept
 Add a Scale layer to the network. More...
 
ISoftMaxLayeraddSoftMax (ITensor &input) noexcept
 Add a SoftMax layer to the network. More...
 
IConcatenationLayeraddConcatenation (ITensor *const *inputs, int32_t nbInputs) noexcept
 Add a concatenation layer to the network. More...
 
TRT_DEPRECATED IDeconvolutionLayeraddDeconvolution (ITensor &input, int32_t nbOutputMaps, DimsHW kernelSize, Weights kernelWeights, Weights biasWeights) noexcept
 Add a deconvolution layer to the network. More...
 
IElementWiseLayeraddElementWise (ITensor &input1, ITensor &input2, ElementWiseOperation op) noexcept
 Add an elementwise layer to the network. More...
 
IUnaryLayeraddUnary (ITensor &input, UnaryOperation operation) noexcept
 Add a unary layer to the network. More...
 
TRT_DEPRECATED IPaddingLayeraddPadding (ITensor &input, DimsHW prePadding, DimsHW postPadding) noexcept
 Add a padding layer to the network. More...
 
IShuffleLayeraddShuffle (ITensor &input) noexcept
 Add a shuffle layer to the network. More...
 
IOneHotLayeraddOneHot (ITensor &indices, ITensor &values, ITensor &depth, int32_t axis) noexcept
 Add a OneHot layer to the network. More...
 
int32_t getNbLayers () const noexcept
 Get the number of layers in the network. More...
 
ILayergetLayer (int32_t index) const noexcept
 Get the layer specified by the given index. More...
 
int32_t getNbInputs () const noexcept
 Get the number of inputs in the network. More...
 
ITensorgetInput (int32_t index) const noexcept
 Get the input tensor specified by the given index. More...
 
int32_t getNbOutputs () const noexcept
 Get the number of outputs in the network. More...
 
ITensorgetOutput (int32_t index) const noexcept
 Get the output tensor specified by the given index. More...
 
TRT_DEPRECATED void destroy () noexcept
 Destroy this INetworkDefinition object. More...
 
IReduceLayeraddReduce (ITensor &input, ReduceOperation operation, uint32_t reduceAxes, bool keepDimensions) noexcept
 Add a reduce layer to the network. More...
 
ITopKLayeraddTopK (ITensor &input, TopKOperation op, int32_t k, uint32_t reduceAxes) noexcept
 Add a TopK layer to the network. More...
 
IGatherLayeraddGather (ITensor &data, ITensor &indices, int32_t axis) noexcept
 Add gather with mode GatherMode::kDEFAULT and specified axis and nbElementWiseDims=0. More...
 
IGatherLayeraddGatherV2 (ITensor &data, ITensor &indices, GatherMode mode) noexcept
 Add gather with specified mode, axis=0 and nbElementWiseDims=0. More...
 
IRaggedSoftMaxLayeraddRaggedSoftMax (ITensor &input, ITensor &bounds) noexcept
 Add a RaggedSoftMax layer to the network. More...
 
IMatrixMultiplyLayeraddMatrixMultiply (ITensor &input0, MatrixOperation op0, ITensor &input1, MatrixOperation op1) noexcept
 Add a MatrixMultiply layer to the network. More...
 
INonZeroLayeraddNonZero (ITensor &input) noexcept
 Add a nonzero layer to the network. More...
 
IConstantLayeraddConstant (Dims dimensions, Weights weights) noexcept
 Add a constant layer to the network. More...
 
TRT_DEPRECATED IRNNv2LayeraddRNNv2 (ITensor &input, int32_t layerCount, int32_t hiddenSize, int32_t maxSeqLen, RNNOperation op) noexcept
 Add an layerCount deep RNN layer to the network with hiddenSize internal states that can take a batch with fixed or variable sequence lengths. More...
 
IIdentityLayeraddIdentity (ITensor &input) noexcept
 Add an identity layer. More...
 
ICastLayeraddCast (ITensor &input, DataType toType) noexcept
 Add a cast layer. More...
 
void removeTensor (ITensor &tensor) noexcept
 remove a tensor from the network definition. More...
 
void unmarkOutput (ITensor &tensor) noexcept
 unmark a tensor as a network output. More...
 
IPluginV2LayeraddPluginV2 (ITensor *const *inputs, int32_t nbInputs, IPluginV2 &plugin) noexcept
 Add a plugin layer to the network using the IPluginV2 interface. More...
 
ISliceLayeraddSlice (ITensor &input, Dims start, Dims size, Dims stride) noexcept
 Add a slice layer to the network. More...
 
void setName (char const *name) noexcept
 Sets the name of the network. More...
 
char const * getName () const noexcept
 Returns the name associated with the network. More...
 
IShapeLayeraddShape (ITensor &input) noexcept
 Add a shape layer to the network. More...
 
bool hasImplicitBatchDimension () const noexcept
 Query whether the network was created with an implicit batch dimension. More...
 
bool markOutputForShapes (ITensor &tensor) noexcept
 Enable tensor's value to be computed by IExecutionContext::getShapeBinding. More...
 
bool unmarkOutputForShapes (ITensor &tensor) noexcept
 Undo markOutputForShapes. More...
 
IParametricReLULayeraddParametricReLU (ITensor &input, ITensor &slope) noexcept
 Add a parametric ReLU layer to the network. More...
 
IConvolutionLayeraddConvolutionNd (ITensor &input, int32_t nbOutputMaps, Dims kernelSize, Weights kernelWeights, Weights biasWeights) noexcept
 Add a multi-dimension convolution layer to the network. More...
 
IPoolingLayeraddPoolingNd (ITensor &input, PoolingType type, Dims windowSize) noexcept
 Add a multi-dimension pooling layer to the network. More...
 
IDeconvolutionLayeraddDeconvolutionNd (ITensor &input, int32_t nbOutputMaps, Dims kernelSize, Weights kernelWeights, Weights biasWeights) noexcept
 Add a multi-dimension deconvolution layer to the network. More...
 
IScaleLayeraddScaleNd (ITensor &input, ScaleMode mode, Weights shift, Weights scale, Weights power, int32_t channelAxis) noexcept
 Add a multi-dimension scale layer to the network. More...
 
IResizeLayeraddResize (ITensor &input) noexcept
 Add a resize layer to the network. More...
 
TRT_DEPRECATED bool hasExplicitPrecision () const noexcept
 True if network is an explicit precision network. More...
 
ILoopaddLoop () noexcept
 Add a loop to the network. More...
 
ISelectLayeraddSelect (ITensor &condition, ITensor &thenInput, ITensor &elseInput) noexcept
 Add a select layer to the network. More...
 
IAssertionLayeraddAssertion (ITensor &condition, char const *message) noexcept
 Add an assertion layer to the network. More...
 
IFillLayeraddFill (Dims dimensions, FillOperation op) noexcept
 Add a fill layer to the network. More...
 
TRT_DEPRECATED IPaddingLayeraddPaddingNd (ITensor &input, Dims prePadding, Dims postPadding) noexcept
 Add a padding layer to the network. Only 2D padding is currently supported. More...
 
bool setWeightsName (Weights weights, char const *name) noexcept
 Associate a name with all current uses of the given weights. More...
 
void setErrorRecorder (IErrorRecorder *recorder) noexcept
 Set the ErrorRecorder for this interface. More...
 
IErrorRecordergetErrorRecorder () const noexcept
 get the ErrorRecorder assigned to this interface. More...
 
IDequantizeLayeraddDequantize (ITensor &input, ITensor &scale) noexcept
 Add a dequantization layer to the network. More...
 
IScatterLayeraddScatter (ITensor &data, ITensor &indices, ITensor &updates, ScatterMode mode) noexcept
 Add a Scatter layer to the network with specified mode and axis=0. More...
 
IQuantizeLayeraddQuantize (ITensor &input, ITensor &scale) noexcept
 Add a quantization layer to the network. More...
 
IIfConditionaladdIfConditional () noexcept
 Add an If-conditional layer to the network. More...
 
IEinsumLayeraddEinsum (ITensor *const *inputs, int32_t nbInputs, char const *equation) noexcept
 Add an Einsum layer to the network. More...
 
IGridSampleLayeraddGridSample (ITensor &input, ITensor &grid) noexcept
 Add a GridSample layer to the network. More...
 
INMSLayeraddNMS (ITensor &boxes, ITensor &scores, ITensor &maxOutputBoxesPerClass) noexcept
 Add a non-maximum suppression layer to the network. More...
 
IReverseSequenceLayeraddReverseSequence (ITensor &input, ITensor &sequenceLens) noexcept
 Add a ReverseSequence layer to the network. More...
 
INormalizationLayeraddNormalization (ITensor &input, ITensor &scale, ITensor &bias, uint32_t axesMask) noexcept
 Add a normalization layer to the network. More...
 
virtual IBuildergetBuilder () const noexcept
 Return the builder from which this INetworkDefinition was created. More...
 

Protected Attributes

apiv::VNetworkDefinition * mImpl
 

Additional Inherited Members

- Protected Member Functions inherited from nvinfer1::INoCopy
 INoCopy ()=default
 
virtual ~INoCopy ()=default
 
 INoCopy (INoCopy const &other)=delete
 
INoCopyoperator= (INoCopy const &other)=delete
 
 INoCopy (INoCopy &&other)=delete
 
INoCopyoperator= (INoCopy &&other)=delete
 

Detailed Description

A network definition for input to the builder.

A network definition defines the structure of the network, and combined with a IBuilderConfig, is built into an engine using an IBuilder. An INetworkDefinition can either have an implicit batch dimensions, specified at runtime, or all dimensions explicit, full dims mode, in the network definition. The former mode, i.e. the implicit batch size mode, has been deprecated. The function hasImplicitBatchDimension() can be used to query the mode of the network.

A network with implicit batch dimensions returns the dimensions of a layer without the implicit dimension, and instead the batch is specified at execute/enqueue time. If the network has all dimensions specified, then the first dimension follows elementwise broadcast rules: if it is 1 for some inputs and is some value N for all other inputs, then the first dimension of each output is N, and the inputs with 1 for the first dimension are broadcast. Having divergent batch sizes across inputs to a layer is not supported.

Warning
Do not inherit from this class, as doing so will break forward-compatibility of the API and ABI.

Constructor & Destructor Documentation

◆ ~INetworkDefinition()

virtual nvinfer1::INetworkDefinition::~INetworkDefinition ( )
virtualdefaultnoexcept

Member Function Documentation

◆ addActivation()

IActivationLayer * nvinfer1::INetworkDefinition::addActivation ( ITensor input,
ActivationType  type 
)
inlinenoexcept

Add an activation layer to the network.

Parameters
inputThe input tensor to the layer.
typeThe type of activation function to apply.

Note that the setAlpha() and setBeta() methods must be used on the output for activations that require these parameters.

See also
IActivationLayer ActivationType
Warning
Int32 tensors are not valid input tensors.
Returns
The new activation layer, or nullptr if it could not be created.

◆ addAssertion()

IAssertionLayer * nvinfer1::INetworkDefinition::addAssertion ( ITensor condition,
char const *  message 
)
inlinenoexcept

Add an assertion layer to the network.

Parameters
conditionThe input tensor to the layer.
messageA message to print if the assertion fails.
See also
IAssertionLayer
Returns
The new assertion layer, or nullptr if it could not be created.

The input tensor must be a boolean shape tensor.

◆ addCast()

ICastLayer * nvinfer1::INetworkDefinition::addCast ( ITensor input,
DataType  toType 
)
inlinenoexcept

Add a cast layer.

Parameters
inputThe input tensor to the layer.
toTypeThe DataType of the output tensor
See also
ICastLayer
Returns
The new cast layer, or nullptr if it could not be created.

◆ addConcatenation()

IConcatenationLayer * nvinfer1::INetworkDefinition::addConcatenation ( ITensor *const *  inputs,
int32_t  nbInputs 
)
inlinenoexcept

Add a concatenation layer to the network.

Parameters
inputsThe input tensors to the layer.
nbInputsThe number of input tensors.
See also
IConcatenationLayer
Returns
The new concatenation layer, or nullptr if it could not be created.
Warning
All tensors must have the same dimensions except along the concatenation axis.

◆ addConstant()

IConstantLayer * nvinfer1::INetworkDefinition::addConstant ( Dims  dimensions,
Weights  weights 
)
inlinenoexcept

Add a constant layer to the network.

Parameters
dimensionsThe dimensions of the constant.
weightsThe constant value, represented as weights.
See also
IConstantLayer
Returns
The new constant layer, or nullptr if it could not be created.

If weights.type is DataType::kINT32, the output is a tensor of 32-bit indices. Otherwise the output is a tensor of real values and the output type will be follow TensorRT's normal precision rules.

If tensors in the network have an implicit batch dimension, the constant is broadcast over that dimension.

If a wildcard dimension is used, the volume of the runtime dimensions must equal the number of weights specified.

Warning
DataType::kUINT8 not supported.

◆ addConvolution()

TRT_DEPRECATED IConvolutionLayer * nvinfer1::INetworkDefinition::addConvolution ( ITensor input,
int32_t  nbOutputMaps,
DimsHW  kernelSize,
Weights  kernelWeights,
Weights  biasWeights 
)
inlinenoexcept

Add a convolution layer to the network.

Parameters
inputThe input tensor to the convolution.
nbOutputMapsThe number of output feature maps for the convolution.
kernelSizeThe HW-dimensions of the convolution kernel.
kernelWeightsThe kernel weights for the convolution.
biasWeightsThe bias weights for the convolution. Weights{} represents no bias.
See also
IConvolutionLayer
Warning
It is an error to specify a wildcard value for the 'C' dimension of the input tensor.
Int32 tensors are not valid input tensors.
Returns
The new convolution layer, or nullptr if it could not be created.
Deprecated:
Superseded by addConvolutionNd. Deprecated prior to TensorRT 8.0 and will be removed in 9.0

◆ addConvolutionNd()

IConvolutionLayer * nvinfer1::INetworkDefinition::addConvolutionNd ( ITensor input,
int32_t  nbOutputMaps,
Dims  kernelSize,
Weights  kernelWeights,
Weights  biasWeights 
)
inlinenoexcept

Add a multi-dimension convolution layer to the network.

Parameters
inputThe input tensor to the convolution.
nbOutputMapsThe number of output feature maps for the convolution.
kernelSizeThe multi-dimensions of the convolution kernel.
kernelWeightsThe kernel weights for the convolution.
biasWeightsThe bias weights for the convolution. Weights{} represents no bias.
See also
IConvolutionLayer
Warning
It is an error to specify a wildcard value for the 'C' dimension of the input tensor.
Int32 tensors are not valid input tensors.
Only 2D or 3D convolution is supported.
Returns
The new convolution layer, or nullptr if it could not be created.

◆ addDeconvolution()

TRT_DEPRECATED IDeconvolutionLayer * nvinfer1::INetworkDefinition::addDeconvolution ( ITensor input,
int32_t  nbOutputMaps,
DimsHW  kernelSize,
Weights  kernelWeights,
Weights  biasWeights 
)
inlinenoexcept

Add a deconvolution layer to the network.

Parameters
inputThe input tensor to the layer.
nbOutputMapsThe number of output feature maps.
kernelSizeThe HW-dimensions of the deconvolution kernel.
kernelWeightsThe kernel weights for the deconvolution.
biasWeightsThe bias weights for the deconvolution. Weights{} represents no bias.
See also
IDeconvolutionLayer
Warning
It is an error to specify a wildcard value for the 'C' dimension of the input tensor.
Int32 tensors are not valid input tensors.
Returns
The new deconvolution layer, or nullptr if it could not be created.
Deprecated:
Superseded by addDeconvolutionNd. Deprecated prior to TensorRT 8.0 and will be removed in 9.0

◆ addDeconvolutionNd()

IDeconvolutionLayer * nvinfer1::INetworkDefinition::addDeconvolutionNd ( ITensor input,
int32_t  nbOutputMaps,
Dims  kernelSize,
Weights  kernelWeights,
Weights  biasWeights 
)
inlinenoexcept

Add a multi-dimension deconvolution layer to the network.

Parameters
inputThe input tensor to the layer.
nbOutputMapsThe number of output feature maps.
kernelSizeThe multi-dimensions of the deconvolution kernel.
kernelWeightsThe kernel weights for the deconvolution.
biasWeightsThe bias weights for the deconvolution. Weights{} represents no bias.
See also
IDeconvolutionLayer
Warning
It is an error to specify a wildcard value for the 'C' dimension of the input tensor.
Int32 tensors are not valid input tensors.
Only 2D or 3D deconvolution is supported.
Returns
The new deconvolution layer, or nullptr if it could not be created.

◆ addDequantize()

IDequantizeLayer * nvinfer1::INetworkDefinition::addDequantize ( ITensor input,
ITensor scale 
)
inlinenoexcept

Add a dequantization layer to the network.

Parameters
inputThe input tensor to be quantized.
scaleA tensor with the scale value.
See also
IDequantizeLayer

input tensor data type must be DataType::kFLOAT. scale tensor data type must be DataType::kFLOAT. The subgraph which terminates with the scale tensor must be a build-time constant.

Returns
The new quantization layer, or nullptr if it could not be created.

◆ addEinsum()

IEinsumLayer * nvinfer1::INetworkDefinition::addEinsum ( ITensor *const *  inputs,
int32_t  nbInputs,
char const *  equation 
)
inlinenoexcept

Add an Einsum layer to the network.

Parameters
inputsThe input tensors to the layer.
nbInputsThe number of input tensors.
equationThe equation of the layer
See also
IEinsumLayer
Returns
The new Einsum layer, or nullptr if it could not be created.

◆ addElementWise()

IElementWiseLayer * nvinfer1::INetworkDefinition::addElementWise ( ITensor input1,
ITensor input2,
ElementWiseOperation  op 
)
inlinenoexcept

Add an elementwise layer to the network.

Parameters
input1The first input tensor to the layer.
input2The second input tensor to the layer.
opThe binary operation that the layer applies.

The input tensors must have the same rank and compatible type. Two types are compatible if they are the same type or are both in the set {kFLOAT, kHALF}. For each dimension, their lengths must match, or one of them must be one. In the latter case, the tensor is broadcast along that axis.

The output tensor has the same rank as the inputs. For each dimension, its length is the maximum of the lengths of the corresponding input dimension.

The inputs are shape tensors if the output is a shape tensor.

See also
IElementWiseLayer
Returns
The new elementwise layer, or nullptr if it could not be created.

◆ addFill()

IFillLayer * nvinfer1::INetworkDefinition::addFill ( Dims  dimensions,
FillOperation  op 
)
inlinenoexcept

Add a fill layer to the network.

Parameters
dimensionsThe output tensor dimensions.
opThe fill operation that the layer applies.
Warning
For FillOperation::kLINSPACE, dimensions.nbDims must be 1.

This layer is non-deterministic across subsequent calls as the same inputs will produce different output tensors if op is either FillOperation::kRANDOM_UNIFORM or FillOperation::kRANDOM_NORMAL due to random state being shared across calls. The output tensors generated are determinstic when starting from the same initial state.

The network must not have an implicit batch dimension.

See also
IFillLayer
Returns
The new fill layer, or nullptr if it could not be created.

◆ addFullyConnected()

TRT_DEPRECATED IFullyConnectedLayer * nvinfer1::INetworkDefinition::addFullyConnected ( ITensor input,
int32_t  nbOutputs,
Weights  kernelWeights,
Weights  biasWeights 
)
inlinenoexcept

Add a fully connected layer to the network.

Parameters
inputThe input tensor to the layer.
nbOutputsThe number of outputs of the layer.
kernelWeightsThe kernel weights for the fully connected layer.
biasWeightsThe bias weights for the fully connected layer. Weights{} represents no bias.
See also
IFullyConnectedLayer
Warning
It is an error to specify a wildcard value for the 'C' dimension of the input tensor.
Int32 tensors are not valid input tensors.
Returns
The new fully connected layer, or nullptr if it could not be created.
Deprecated:
Deprecated in TensorRT 8.4. Superseded by addMatrixMultiply().

◆ addGather()

IGatherLayer * nvinfer1::INetworkDefinition::addGather ( ITensor data,
ITensor indices,
int32_t  axis 
)
inlinenoexcept

Add gather with mode GatherMode::kDEFAULT and specified axis and nbElementWiseDims=0.

Parameters
dataThe tensor to gather values from.
indicesThe tensor to get indices from to populate the output tensor.
axisThe axis in the data tensor to gather on.
See also
IGatherLayer
Returns
The new gather layer, or nullptr if it could not be created.

◆ addGatherV2()

IGatherLayer * nvinfer1::INetworkDefinition::addGatherV2 ( ITensor data,
ITensor indices,
GatherMode  mode 
)
inlinenoexcept

Add gather with specified mode, axis=0 and nbElementWiseDims=0.

Parameters
dataThe tensor to gather values from.
indicesThe tensor to get indices from to populate the output tensor.
modeThe gather mode.
See also
IGatherLayer
Returns
The new gather layer, or nullptr if it could not be created.

◆ addGridSample()

IGridSampleLayer * nvinfer1::INetworkDefinition::addGridSample ( ITensor input,
ITensor grid 
)
inlinenoexcept

Add a GridSample layer to the network.

Parameters
inputThe input tensor to the layer.
gridThe grid tensor to the layer.
See also
IGridSampleLayer

Creates a GridSample layer with a InterpolationMode::kLINEAR, unaligned corners, and SampleMode::kFILL for 4d-shape input tensors.

Returns
The new GridSample layer, or nullptr if it could not be created.

◆ addIdentity()

IIdentityLayer * nvinfer1::INetworkDefinition::addIdentity ( ITensor input)
inlinenoexcept

Add an identity layer.

Parameters
inputThe input tensor to the layer.
See also
IIdentityLayer
Returns
The new identity layer, or nullptr if it could not be created.

◆ addIfConditional()

IIfConditional * nvinfer1::INetworkDefinition::addIfConditional ( )
inlinenoexcept

Add an If-conditional layer to the network.

An IIfConditional provides a way to conditionally execute parts of the network.

See also
IIfConditional
Returns
The new conditional layer, or nullptr if network has an implicit batch dimension or this version of TensorRT does not support conditional execution.

◆ addInput()

ITensor * nvinfer1::INetworkDefinition::addInput ( char const *  name,
DataType  type,
Dims  dimensions 
)
inlinenoexcept

Add an input tensor to the network.

The name of the input tensor is used to find the index into the buffer array for an engine built from the network. The volume must be less than 2^31 elements.

For networks with an implicit batch dimension, this volume includes the batch dimension with its length set to the maximum batch size. For networks with all explicit dimensions and with wildcard dimensions, the volume is based on the maxima specified by an IOptimizationProfile.Dimensions are normally non-negative integers. The exception is that in networks with all explicit dimensions, -1 can be used as a wildcard for a dimension to be specified at runtime. Input tensors with such a wildcard must have a corresponding entry in the IOptimizationProfiles indicating the permitted extrema, and the input dimensions must be set by IExecutionContext::setBindingDimensions. Different IExecutionContext instances can have different dimensions. Wildcard dimensions are only supported for EngineCapability::kSTANDARD. They are not supported in safety contexts. DLA does not support Wildcard dimensions.

Tensor dimensions are specified independent of format. For example, if a tensor is formatted in "NHWC" or a vectorized format, the dimensions are still specified in the order{N, C, H, W}. For 2D images with a channel dimension, the last three dimensions are always {C,H,W}. For 3D images with a channel dimension, the last four dimensions are always {C,D,H,W}.

Parameters
nameThe name of the tensor.
typeThe type of the data held in the tensor.
dimensionsThe dimensions of the tensor.
Warning
It is an error to specify a wildcard value on a dimension that is determined by trained parameters.
If run on DLA with explicit dimensions, only leading dimension can be a wildcard. And provided profile must have same minimum, optimum, and maximum dimensions.
The string name must be null-terminated, and be at most 4096 bytes including the terminator.
See also
ITensor
Returns
The new tensor or nullptr if there is an error.

◆ addLoop()

ILoop * nvinfer1::INetworkDefinition::addLoop ( )
inlinenoexcept

Add a loop to the network.

An ILoop provides a way to specify a recurrent subgraph.

Returns
Pointer to ILoop that can be used to add loop boundary layers for the loop, or nullptr if network has an implicit batch dimension or this version of TensorRT does not support loops.

The network must not have an implicit batch dimension.

◆ addLRN()

ILRNLayer * nvinfer1::INetworkDefinition::addLRN ( ITensor input,
int32_t  window,
float  alpha,
float  beta,
float  k 
)
inlinenoexcept

Add a LRN layer to the network.

Parameters
inputThe input tensor to the layer.
windowThe size of the window.
alphaThe alpha value for the LRN computation.
betaThe beta value for the LRN computation.
kThe k value for the LRN computation.
See also
ILRNLayer
Warning
Int32 tensors are not valid input tensors.
Returns
The new LRN layer, or nullptr if it could not be created.

◆ addMatrixMultiply()

IMatrixMultiplyLayer * nvinfer1::INetworkDefinition::addMatrixMultiply ( ITensor input0,
MatrixOperation  op0,
ITensor input1,
MatrixOperation  op1 
)
inlinenoexcept

Add a MatrixMultiply layer to the network.

Parameters
input0The first input tensor (commonly A).
op0The operation to apply to input0.
input1The second input tensor (commonly B).
op1The operation to apply to input1.

The inputs are shape tensors if the output is a shape tensor.

See also
IMatrixMultiplyLayer
Warning
Int32 tensors are not valid input tensors.
Returns
The new matrix multiply layer, or nullptr if it could not be created.

◆ addNMS()

INMSLayer * nvinfer1::INetworkDefinition::addNMS ( ITensor boxes,
ITensor scores,
ITensor maxOutputBoxesPerClass 
)
inlinenoexcept

Add a non-maximum suppression layer to the network.

Parameters
boxesThe input boxes tensor to the layer.
scoresThe input scores tensor to the layer.
maxOutputBoxesPerClassThe input maxOutputBoxesPerClass tensor to the layer.
See also
INMSLayer
Returns
The new NMS layer, or nullptr if it could not be created.

◆ addNonZero()

INonZeroLayer * nvinfer1::INetworkDefinition::addNonZero ( ITensor input)
inlinenoexcept

Add a nonzero layer to the network.

Parameters
inputThe input tensor to the layer.
See also
INonZeroLayer
Returns
The new nonzero layer, or nullptr if it could be created.

◆ addNormalization()

INormalizationLayer * nvinfer1::INetworkDefinition::addNormalization ( ITensor input,
ITensor scale,
ITensor bias,
uint32_t  axesMask 
)
inlinenoexcept

Add a normalization layer to the network.

Parameters
inputThe input tensor to the layer.
scaleThe scale tensor used to scale the normalized output.
biasThe bias tensor used to scale the normalized output.
axesMaskThe axes on which to perform mean calculations. The bit in position i of bitmask axesMask corresponds to explicit dimension i of the result. E.g., the least significant bit corresponds to the first explicit dimension and the next to least significant bit corresponds to the second explicit dimension.

The normalization layer works by performing normalization of the tensor input on the specified axesMask. The result is then scaled by multiplying with scale and adding bias.

The shape of scale and bias are expected the be the same, and must have the same rank and be unidirectionally broadcastable to the shape of input.

See also
INormalizationLayer
Returns
The new normalization layer, or nullptr if it could not be created.

◆ addOneHot()

IOneHotLayer * nvinfer1::INetworkDefinition::addOneHot ( ITensor indices,
ITensor values,
ITensor depth,
int32_t  axis 
)
inlinenoexcept

Add a OneHot layer to the network.

Parameters
indices- tensor containing indices where on_value should be set.
values- a 2-element tensor, consisting of [off_value, on_value].
depth- tensor containing the width of the added one-hot dimension.
axis- the axis to add the one-hot encoding to.
See also
IOneHotLayer
Returns
The new OneHot layer, or nullptr if it could not be created.

◆ addPadding()

TRT_DEPRECATED IPaddingLayer * nvinfer1::INetworkDefinition::addPadding ( ITensor input,
DimsHW  prePadding,
DimsHW  postPadding 
)
inlinenoexcept

Add a padding layer to the network.

Parameters
inputThe input tensor to the layer.
prePaddingThe padding to apply to the start of the tensor.
postPaddingThe padding to apply to the end of the tensor.
See also
IPaddingLayer
Returns
The new padding layer, or nullptr if it could not be created.
Deprecated:
Superseded by addPaddingNd. Deprecated prior to TensorRT 8.0 and will be removed in 9.0

◆ addPaddingNd()

TRT_DEPRECATED IPaddingLayer * nvinfer1::INetworkDefinition::addPaddingNd ( ITensor input,
Dims  prePadding,
Dims  postPadding 
)
inlinenoexcept

Add a padding layer to the network. Only 2D padding is currently supported.

Parameters
inputThe input tensor to the layer.
prePaddingThe padding to apply to the start of the tensor.
postPaddingThe padding to apply to the end of the tensor.
See also
IPaddingLayer
Returns
The new padding layer, or nullptr if it could not be created.
Deprecated:
Deprecated in TensorRT 8.0. Superseded by addSlice().

◆ addParametricReLU()

IParametricReLULayer * nvinfer1::INetworkDefinition::addParametricReLU ( ITensor input,
ITensor slope 
)
inlinenoexcept

Add a parametric ReLU layer to the network.

Parameters
inputThe input tensor to the layer.
slopeThe slope tensor to the layer. This tensor should be unidirectionally broadcastable to the input tensor.
See also
IParametricReLULayer
Warning
Int32 tensors are not valid input tensors.
Returns
The new parametric ReLU layer, or nullptr if it could not be created.

◆ addPluginV2()

IPluginV2Layer * nvinfer1::INetworkDefinition::addPluginV2 ( ITensor *const *  inputs,
int32_t  nbInputs,
IPluginV2 plugin 
)
inlinenoexcept

Add a plugin layer to the network using the IPluginV2 interface.

Parameters
inputsThe input tensors to the layer.
nbInputsThe number of input tensors.
pluginThe layer plugin.
See also
IPluginV2Layer
Warning
Dimension wildcard are only supported with IPluginV2DynamicExt or IPluginV2IOExt plugins.
Int32 tensors are not valid input tensors.
Returns
The new plugin layer, or nullptr if it could not be created.

◆ addPooling()

TRT_DEPRECATED IPoolingLayer * nvinfer1::INetworkDefinition::addPooling ( ITensor input,
PoolingType  type,
DimsHW  windowSize 
)
inlinenoexcept

Add a pooling layer to the network.

Parameters
inputThe input tensor to the layer.
typeThe type of pooling to apply.
windowSizeThe size of the pooling window.
See also
IPoolingLayer PoolingType
Warning
Int32 tensors are not valid input tensors.
Returns
The new pooling layer, or nullptr if it could not be created.
Deprecated:
Superseded by addPoolingNd. Deprecated prior to TensorRT 8.0 and will be removed in 9.0

◆ addPoolingNd()

IPoolingLayer * nvinfer1::INetworkDefinition::addPoolingNd ( ITensor input,
PoolingType  type,
Dims  windowSize 
)
inlinenoexcept

Add a multi-dimension pooling layer to the network.

Parameters
inputThe input tensor to the layer.
typeThe type of pooling to apply.
windowSizeThe size of the pooling window.
See also
IPoolingLayer PoolingType
Warning
Int32 tensors are not valid input tensors.
Only 2D or 3D pooling is supported.
Returns
The new pooling layer, or nullptr if it could not be created.

◆ addQuantize()

IQuantizeLayer * nvinfer1::INetworkDefinition::addQuantize ( ITensor input,
ITensor scale 
)
inlinenoexcept

Add a quantization layer to the network.

Parameters
inputThe input tensor to be quantized.
scaleA tensor with the scale value.
See also
IQuantizeLayer

input tensor data type must be DataType::kFLOAT. scale tensor data type must be DataType::kFLOAT. The subgraph which terminates with the scale tensor must be a build-time constant.

Returns
The new quantization layer, or nullptr if it could not be created.

◆ addRaggedSoftMax()

IRaggedSoftMaxLayer * nvinfer1::INetworkDefinition::addRaggedSoftMax ( ITensor input,
ITensor bounds 
)
inlinenoexcept

Add a RaggedSoftMax layer to the network.

Parameters
inputThe ZxS input tensor.
boundsThe Zx1 bounds tensor.
See also
IRaggedSoftMaxLayer
Warning
The bounds tensor cannot have the last dimension be the wildcard character.
Int32 tensors are not valid input tensors.
Returns
The new RaggedSoftMax layer, or nullptr if it could not be created.

◆ addReduce()

IReduceLayer * nvinfer1::INetworkDefinition::addReduce ( ITensor input,
ReduceOperation  operation,
uint32_t  reduceAxes,
bool  keepDimensions 
)
inlinenoexcept

Add a reduce layer to the network.

Parameters
inputThe input tensor to the layer.
operationThe reduction operation to perform.
reduceAxesThe reduction dimensions. The bit in position i of bitmask reduceAxes corresponds to explicit dimension i if result. E.g., the least significant bit corresponds to the first explicit dimension and the next to least significant bit corresponds to the second explicit dimension.
keepDimensionsThe boolean that specifies whether or not to keep the reduced dimensions in the output of the layer.

The reduce layer works by performing an operation specified by operation to reduce the tensor input across the axes specified by reduceAxes.

See also
IReduceLayer
Warning
If output is an Int32 shape tensor, ReduceOperation::kAVG is unsupported.
Returns
The new reduce layer, or nullptr if it could not be created.

◆ addResize()

IResizeLayer * nvinfer1::INetworkDefinition::addResize ( ITensor input)
inlinenoexcept

Add a resize layer to the network.

Parameters
inputThe input tensor to the layer.
See also
IResizeLayer
Warning
Int32 tensors are not valid input tensors.
Returns
The new resize layer, or nullptr if it could not be created.

◆ addReverseSequence()

IReverseSequenceLayer * nvinfer1::INetworkDefinition::addReverseSequence ( ITensor input,
ITensor sequenceLens 
)
inlinenoexcept

Add a ReverseSequence layer to the network.

Parameters
inputThe input tensor to the layer. Must have rank >= 2.
sequenceLens1D tensor specifying lengths of sequences to reverse in a batch. The length of the sequenceLens tensor must be equal to the size of the dimension in input tensor specified by batchAxis.
See also
IReverseSequenceLayer
Returns
The new ReverseSequence layer, or nullptr if it could not be created.

◆ addRNNv2()

TRT_DEPRECATED IRNNv2Layer * nvinfer1::INetworkDefinition::addRNNv2 ( ITensor input,
int32_t  layerCount,
int32_t  hiddenSize,
int32_t  maxSeqLen,
RNNOperation  op 
)
inlinenoexcept

Add an layerCount deep RNN layer to the network with hiddenSize internal states that can take a batch with fixed or variable sequence lengths.

Parameters
inputThe input tensor to the layer (see below).
layerCountThe number of layers in the RNN.
hiddenSizeSize of the internal hidden state for each layer.
maxSeqLenMaximum sequence length for the input.
opThe type of RNN to execute.

By default, the layer is configured with RNNDirection::kUNIDIRECTION and RNNInputMode::kLINEAR. To change these settings, use IRNNv2Layer::setDirection() and IRNNv2Layer::setInputMode().

Weights and biases for the added layer should be set using IRNNv2Layer::setWeightsForGate() and IRNNv2Layer::setBiasForGate() prior to building an engine using this network.

The input tensors must be of the type DataType::kFLOAT or DataType::kHALF. The layout of the weights is row major and must be the same datatype as the input tensor. weights contain 8 matrices and bias contains 8 vectors.

See IRNNv2Layer::setWeightsForGate() and IRNNv2Layer::setBiasForGate() for details on the required input format for weights and bias.

The input ITensor should contain zero or more index dimensions {N1, ..., Np}, followed by two dimensions, defined as follows:

  • S_max is the maximum allowed sequence length (number of RNN iterations)
  • E specifies the embedding length (unless RNNInputMode::kSKIP is set, in which case it should match getHiddenSize()).

By default, all sequences in the input are assumed to be size maxSeqLen. To provide explicit sequence lengths for each input sequence in the batch, use IRNNv2Layer::setSequenceLengths().

The RNN layer outputs up to three tensors.

The first output tensor is the output of the final RNN layer across all timesteps, with dimensions {N1, ..., Np, S_max, H}:

  • N1..Np are the index dimensions specified by the input tensor
  • S_max is the maximum allowed sequence length (number of RNN iterations)
  • H is an output hidden state (equal to getHiddenSize() or 2x getHiddenSize())

The second tensor is the final hidden state of the RNN across all layers, and if the RNN is an LSTM (i.e. getOperation() is RNNOperation::kLSTM), then the third tensor is the final cell state of the RNN across all layers. Both the second and third output tensors have dimensions {N1, ..., Np, L, H}:

  • N1..Np are the index dimensions specified by the input tensor
  • L is the number of layers in the RNN, equal to getLayerCount() if getDirection is RNNDirection::kUNIDIRECTION, and 2x getLayerCount() if getDirection is RNNDirection::kBIDIRECTION. In the bi-directional case, layer l's final forward hidden state is stored in L = 2*l, and final backward hidden state is stored in L= 2*l + 1.
  • H is the hidden state for each layer, equal to getHiddenSize().
See also
IRNNv2Layer
Deprecated:
Deprecated prior to TensorRT 8.0 and will be removed in 9.0. Superseded by INetworkDefinition::addLoop().
Warning
RNN inputs do not support wildcard dimensions or explicit batch size networks.
Int32 tensors are not valid input tensors, only for sequence lengths.
Returns
The new RNN layer, or nullptr if it could not be created.

◆ addScale()

IScaleLayer * nvinfer1::INetworkDefinition::addScale ( ITensor input,
ScaleMode  mode,
Weights  shift,
Weights  scale,
Weights  power 
)
inlinenoexcept

Add a Scale layer to the network.

Parameters
inputThe input tensor to the layer. This tensor is required to have a minimum of 3 dimensions in implicit batch mode and a minimum of 4 dimensions in explicit batch mode.
modeThe scaling mode.
shiftThe shift value.
scaleThe scale value.
powerThe power value.

If the weights are available, then the size of weights are dependent on the ScaleMode. For ScaleMode::kUNIFORM, the number of weights equals 1. For ScaleMode::kCHANNEL, the number of weights equals the channel dimension. For ScaleMode::kELEMENTWISE, the number of weights equals the product of the last three dimensions of the input.

See also
addScaleNd
IScaleLayer
Warning
Int32 tensors are not valid input tensors.
Returns
The new Scale layer, or nullptr if it could not be created.

◆ addScaleNd()

IScaleLayer * nvinfer1::INetworkDefinition::addScaleNd ( ITensor input,
ScaleMode  mode,
Weights  shift,
Weights  scale,
Weights  power,
int32_t  channelAxis 
)
inlinenoexcept

Add a multi-dimension scale layer to the network.

Parameters
inputThe input tensor to the layer.
modeThe scaling mode.
shiftThe shift value.
scaleThe scale value.
powerThe power value.
channelAxisThe channel axis.

If the weights are available, then the size of weights are dependent on the ScaleMode. For ScaleMode::kUNIFORM, the number of weights equals 1. For ScaleMode::kCHANNEL, the number of weights equals the channel dimension. For ScaleMode::kELEMENTWISE, the number of weights equals the product of all input dimensions at channelAxis and beyond.

For example, if the inputs dimensions are [A,B,C,D,E,F], and channelAxis=2: For ScaleMode::kUNIFORM, the number of weights is equal to 1. For ScaleMode::kCHANNEL, the number of weights is C. For ScaleMode::kELEMENTWISE, the number of weights is C*D*E*F.

channelAxis can also be set explicitly using setChannelAxis().

See also
IScaleLayer
setChannelAxis()
Warning
Int32 tensors are not valid input tensors.
Only 2D or 3D scale is supported.
Returns
The new Scale layer, or nullptr if it could not be created.

◆ addScatter()

IScatterLayer * nvinfer1::INetworkDefinition::addScatter ( ITensor data,
ITensor indices,
ITensor updates,
ScatterMode  mode 
)
inlinenoexcept

Add a Scatter layer to the network with specified mode and axis=0.

Parameters
dataThe input tensor to be updated with additional values.
indicesindices of the elements to be updated.
updatesvalues to be used for updates.
modescatter mode.
See also
IScatterLayer

indices tensor data type must be DataType::kINT32. updates tensor data type must be the same as data

Returns
The new Scatter layer, or nullptr if it could not be created.

◆ addSelect()

ISelectLayer * nvinfer1::INetworkDefinition::addSelect ( ITensor condition,
ITensor thenInput,
ITensor elseInput 
)
inlinenoexcept

Add a select layer to the network.

Parameters
conditionThe condition tensor to the layer. Must have type DataType::kBOOL.
thenInputThe "then" input tensor to the layer.
elseInputThe "else" input tensor to the layer.

All three input tensors must have the same rank, and along each axis must have the same length or a length of one. If the length is one, the tensor is broadcast along that axis. The output tensor has the dimensions of the inputs AFTER the broadcast rule is applied. For example, given:

dimensions of condition: [1,1,5,9] dimensions of thenInput: [1,1,5,9] dimensions of elseInput: [1,3,1,9]

the output dimensions are [1,3,5,9], and the output contents are defined by:

 output[0,i,j,k] = condition[0,0,j,k] ? thenInput[0,0,j,k] : elseInput[0,i,0,k]

The output dimensions are not necessarily the max of the input dimensions if any input is an empty tensor. For example, if in the preceding example, 5 is changed to 0:

dimensions of condition: [1,1,0,9] dimensions of thenInput: [1,1,0,9] dimensions of elseInput: [1,3,1,9]

then the output dimensions are [1,3,0,9].

The network must not have an implicit batch dimension.

The inputs are shape tensors if the output is a shape tensor.

See also
ISelectLayer
Returns
The new select layer, or nullptr if it could not be created.

◆ addShape()

IShapeLayer * nvinfer1::INetworkDefinition::addShape ( ITensor input)
inlinenoexcept

Add a shape layer to the network.

Parameters
inputThe input tensor to the layer.
See also
IShapeLayer
Warning
addShape is only supported when hasImplicitBatchDimensions is false.
Returns
The new shape layer, or nullptr if it could not be created.

◆ addShuffle()

IShuffleLayer * nvinfer1::INetworkDefinition::addShuffle ( ITensor input)
inlinenoexcept

Add a shuffle layer to the network.

Parameters
inputThe input tensor to the layer.
See also
IShuffleLayer
Returns
The new shuffle layer, or nullptr if it could not be created.

◆ addSlice()

ISliceLayer * nvinfer1::INetworkDefinition::addSlice ( ITensor input,
Dims  start,
Dims  size,
Dims  stride 
)
inlinenoexcept

Add a slice layer to the network.

Parameters
inputThe input tensor to the layer.
startThe start offset
sizeThe output dimension
strideThe slicing stride

Positive, negative, zero stride values, and combinations of them in different dimensions are allowed.

See also
ISliceLayer
Returns
The new slice layer, or nullptr if it could not be created.

◆ addSoftMax()

ISoftMaxLayer * nvinfer1::INetworkDefinition::addSoftMax ( ITensor input)
inlinenoexcept

Add a SoftMax layer to the network.

See also
ISoftMaxLayer
Warning
Int32 tensors are not valid input tensors.
Returns
The new SoftMax layer, or nullptr if it could not be created.

◆ addTopK()

ITopKLayer * nvinfer1::INetworkDefinition::addTopK ( ITensor input,
TopKOperation  op,
int32_t  k,
uint32_t  reduceAxes 
)
inlinenoexcept

Add a TopK layer to the network.

The TopK layer has two outputs of the same dimensions. The first contains data values, the second contains index positions for the values. Output values are sorted, largest first for operation kMAX and smallest first for operation kMIN.

Currently only values of K up to 3840 are supported.

Parameters
inputThe input tensor to the layer.
opOperation to perform.
kThe number of elements to keep. For dynamic k, use the setInput() method to pass in k as a tensor instead, which will override the static k value passed here in calculations.
reduceAxesThe reduction dimensions. The bit in position i of bitmask reduceAxes corresponds to explicit dimension i of the result. E.g., the least significant bit corresponds to the first explicit dimension and the next to least significant bit corresponds to the second explicit dimension.

Currently reduceAxes must specify exactly one dimension, and it must be one of the last four dimensions.

See also
ITopKLayer
Warning
Int32 tensors are not valid input tensors.
Returns
The new TopK layer, or nullptr if it could not be created.

◆ addUnary()

IUnaryLayer * nvinfer1::INetworkDefinition::addUnary ( ITensor input,
UnaryOperation  operation 
)
inlinenoexcept

Add a unary layer to the network.

Parameters
inputThe input tensor to the layer.
operationThe operation to apply.
See also
IUnaryLayer

Generally the input must have a floating-point type (or kINT8 as a quantized float), except for the following operations:

  • kSIGN accepts a floating-point or Int32 tensor.
  • kNOT requires a Bool tensor.

The input is a shape tensor if the output is a shape tensor.

Returns
The new unary layer, or nullptr if it could not be created

◆ destroy()

TRT_DEPRECATED void nvinfer1::INetworkDefinition::destroy ( )
inlinenoexcept

Destroy this INetworkDefinition object.

Deprecated:
Deprecated in TensorRT 8.0. Superseded by delete.
Warning
Calling destroy on a managed pointer will result in a double-free error.

◆ getBuilder()

virtual IBuilder & nvinfer1::INetworkDefinition::getBuilder ( ) const
inlinevirtualnoexcept

Return the builder from which this INetworkDefinition was created.

See also
IBuilder::createNetworkV2
Returns
the builder

◆ getErrorRecorder()

IErrorRecorder * nvinfer1::INetworkDefinition::getErrorRecorder ( ) const
inlinenoexcept

get the ErrorRecorder assigned to this interface.

Retrieves the assigned error recorder object for the given class. A nullptr will be returned if setErrorRecorder has not been called.

Returns
A pointer to the IErrorRecorder object that has been registered.
See also
setErrorRecorder()

◆ getInput()

ITensor * nvinfer1::INetworkDefinition::getInput ( int32_t  index) const
inlinenoexcept

Get the input tensor specified by the given index.

Parameters
indexThe index of the input tensor.
Returns
The input tensor, or nullptr if the index is out of range.
Note
adding inputs invalidates indexing here
See also
getNbInputs()

◆ getLayer()

ILayer * nvinfer1::INetworkDefinition::getLayer ( int32_t  index) const
inlinenoexcept

Get the layer specified by the given index.

Parameters
indexThe index of the layer.
Returns
The layer, or nullptr if the index is out of range.
See also
getNbLayers()

◆ getName()

char const * nvinfer1::INetworkDefinition::getName ( ) const
inlinenoexcept

Returns the name associated with the network.

The memory pointed to by getName() is owned by the INetworkDefinition object.

See also
INetworkDefinition::setName()
Returns
A null-terminated C-style string representing the name of the network.

◆ getNbInputs()

int32_t nvinfer1::INetworkDefinition::getNbInputs ( ) const
inlinenoexcept

Get the number of inputs in the network.

Returns
The number of inputs in the network.
See also
getInput()

◆ getNbLayers()

int32_t nvinfer1::INetworkDefinition::getNbLayers ( ) const
inlinenoexcept

Get the number of layers in the network.

Returns
The number of layers in the network.
See also
getLayer()

◆ getNbOutputs()

int32_t nvinfer1::INetworkDefinition::getNbOutputs ( ) const
inlinenoexcept

Get the number of outputs in the network.

The outputs include those marked by markOutput or markOutputForShapes.

Returns
The number of outputs in the network.
See also
getOutput()

◆ getOutput()

ITensor * nvinfer1::INetworkDefinition::getOutput ( int32_t  index) const
inlinenoexcept

Get the output tensor specified by the given index.

Parameters
indexThe index of the output tensor.
Returns
The output tensor, or nullptr if the index is out of range.
Note
adding inputs invalidates indexing here
See also
getNbOutputs()

◆ hasExplicitPrecision()

TRT_DEPRECATED bool nvinfer1::INetworkDefinition::hasExplicitPrecision ( ) const
inlinenoexcept

True if network is an explicit precision network.

Deprecated:
Deprecated in TensorRT 8.0.
See also
createNetworkV2
Returns
True if network has explicit precision, false otherwise.

◆ hasImplicitBatchDimension()

bool nvinfer1::INetworkDefinition::hasImplicitBatchDimension ( ) const
inlinenoexcept

Query whether the network was created with an implicit batch dimension.

Returns
True if tensors have implicit batch dimension, false otherwise.

This is a network-wide property. Either all tensors in the network have an implicit batch dimension or none of them do.

hasImplicitBatchDimension() is true if and only if this INetworkDefinition was created with createNetworkV2() without NetworkDefinitionCreationFlag::kEXPLICIT_BATCH flag.

See also
createNetworkV2

◆ markOutput()

void nvinfer1::INetworkDefinition::markOutput ( ITensor tensor)
inlinenoexcept

Mark a tensor as a network output.

Parameters
tensorThe tensor to mark as an output tensor.
Warning
It is an error to mark a network input as an output.
It is an error to mark a tensor inside an ILoop or an IIfConditional as an output.

◆ markOutputForShapes()

bool nvinfer1::INetworkDefinition::markOutputForShapes ( ITensor tensor)
inlinenoexcept

Enable tensor's value to be computed by IExecutionContext::getShapeBinding.

Returns
True if successful, false if tensor is already marked as an output.

The tensor must be of type DataType::kINT32 and have no more than one dimension.

Warning
The tensor must have dimensions that can be determined to be constants at build time.
It is an error to mark a network input as a shape output.
See also
isShapeBinding(), getShapeBinding()

◆ removeTensor()

void nvinfer1::INetworkDefinition::removeTensor ( ITensor tensor)
inlinenoexcept

remove a tensor from the network definition.

Parameters
tensorthe tensor to remove

It is illegal to remove a tensor that is the input or output of a layer. if this method is called with such a tensor, a warning will be emitted on the log and the call will be ignored. Its intended use is to remove detached tensors after e.g. concatenating two networks with Layer::setInput().

◆ setErrorRecorder()

void nvinfer1::INetworkDefinition::setErrorRecorder ( IErrorRecorder recorder)
inlinenoexcept

Set the ErrorRecorder for this interface.

Assigns the ErrorRecorder to this interface. The ErrorRecorder will track all errors during execution. This function will call incRefCount of the registered ErrorRecorder at least once. Setting recorder to nullptr unregisters the recorder with the interface, resulting in a call to decRefCount if a recorder has been registered.

If an error recorder is not set, messages will be sent to the global log stream.

Parameters
recorderThe error recorder to register with this interface.
See also
getErrorRecorder()

◆ setName()

void nvinfer1::INetworkDefinition::setName ( char const *  name)
inlinenoexcept

Sets the name of the network.

Parameters
nameThe name to assign to this network.

Set the name of the network so that it can be associated with a built engine. The name must be a null-terminated C-style string. TensorRT makes no use of this string except storing it as part of the engine so that it may be retrieved at runtime. A name unique to the builder will be generated by default.

This method copies the name string.

Warning
The string name must be null-terminated, and be at most 4096 bytes including the terminator.
See also
INetworkDefinition::getName(), ISafeCudaEngine::getName()
Returns
none

◆ setWeightsName()

bool nvinfer1::INetworkDefinition::setWeightsName ( Weights  weights,
char const *  name 
)
inlinenoexcept

Associate a name with all current uses of the given weights.

The name must be set after the Weights are used in the network. Lookup is associative. The name applies to all Weights with matching type, value pointer, and count. If Weights with a matching value pointer, but different type or count exists in the network, an error message is issued, the name is rejected, and return false. If the name has already been used for other weights, return false. A nullptr causes the weights to become unnamed, i.e. clears any previous name.

Parameters
weightsThe weights to be named.
nameThe name to associate with the weights.
Returns
true on success.
Warning
The string name must be null-terminated, and be at most 4096 bytes including the terminator.

◆ unmarkOutput()

void nvinfer1::INetworkDefinition::unmarkOutput ( ITensor tensor)
inlinenoexcept

unmark a tensor as a network output.

Parameters
tensorThe tensor to unmark as an output tensor.

see markOutput()

◆ unmarkOutputForShapes()

bool nvinfer1::INetworkDefinition::unmarkOutputForShapes ( ITensor tensor)
inlinenoexcept

Undo markOutputForShapes.

Warning
inputs to addShape cannot contain wildcard dimension values.
Returns
True if successful, false if tensor is not marked as an output.

Member Data Documentation

◆ mImpl

apiv::VNetworkDefinition* nvinfer1::INetworkDefinition::mImpl
protected

The documentation for this class was generated from the following file: