TensorRT 10.0.1
nvinfer1::IExprBuilder Class Reference

Object for constructing IDimensionExpr. More...

#include <NvInferRuntime.h>

Inheritance diagram for nvinfer1::IExprBuilder:
nvinfer1::INoCopy

Public Member Functions

IDimensionExpr const * constant (int64_t value) noexcept
 Return pointer to IDimensionExp for given value. More...
 
IDimensionExpr const * operation (DimensionOperation op, IDimensionExpr const &first, IDimensionExpr const &second) noexcept
 Get the operation. More...
 
IDimensionExpr const * declareSizeTensor (int32_t outputIndex, IDimensionExpr const &opt, IDimensionExpr const &upper)
 Declare a size tensor at the given output index, with the specified auto-tuning formula and upper bound. More...
 

Protected Member Functions

virtual ~IExprBuilder () noexcept=default
 
- Protected Member Functions inherited from nvinfer1::INoCopy
 INoCopy ()=default
 
virtual ~INoCopy ()=default
 
 INoCopy (INoCopy const &other)=delete
 
INoCopyoperator= (INoCopy const &other)=delete
 
 INoCopy (INoCopy &&other)=delete
 
INoCopyoperator= (INoCopy &&other)=delete
 

Protected Attributes

apiv::VExprBuilder * mImpl
 

Detailed Description

Object for constructing IDimensionExpr.

There is no public way to construct an IExprBuilder. It appears as an argument to method IPluginV2DynamicExt::getOutputDimensions() and IPluginV3OneBuild::getOutputShapes(). Overrides of that method can use that IExprBuilder argument to construct expressions that define output dimensions in terms of input dimensions.

Clients should assume that any values constructed by the IExprBuilder are destroyed after IPluginV2DynamicExt::getOutputDimensions() or IPluginV3OneBuild::getOutputShapes() returns.

Warning
Do not inherit from this class, as doing so will break forward-compatibility of the API and ABI.
See also
IDimensionExpr

Constructor & Destructor Documentation

◆ ~IExprBuilder()

virtual nvinfer1::IExprBuilder::~IExprBuilder ( )
protectedvirtualdefaultnoexcept

Member Function Documentation

◆ constant()

IDimensionExpr const * nvinfer1::IExprBuilder::constant ( int64_t  value)
inlinenoexcept

Return pointer to IDimensionExp for given value.

◆ declareSizeTensor()

IDimensionExpr const * nvinfer1::IExprBuilder::declareSizeTensor ( int32_t  outputIndex,
IDimensionExpr const &  opt,
IDimensionExpr const &  upper 
)
inline

Declare a size tensor at the given output index, with the specified auto-tuning formula and upper bound.

A size tensor allows a plugin to have output dimensions that cannot be computed solely from input dimensions. For example, suppose a plugin implements the equivalent of INonZeroLayer for 2D input. The plugin can have one output for the indices of non-zero elements, and a second output containing the number of non-zero elements. Suppose the input has size [M,N] and has K non-zero elements. The plugin can write K to the second output. When telling TensorRT that the first output has shape [2,K], plugin uses IExprBuilder::constant() and IExprBuilder::declareSizeTensor(1,...) to create the IDimensionExpr that respectively denote 2 and K.

TensorRT also needs to know the value of K to use for auto-tuning and an upper bound on K so that it can allocate memory for the output tensor. In the example, supposed typically half of the plugin's input elements are non-zero, and all the elements might be nonzero. then using M*N/2 might be a good expression for the opt parameter, and M*N for the upper bound. IDimensionsExpr for these expressions can be constructed from IDimensionsExpr for the input dimensions.

Parameters
outputIndexindex of a plugin output that is a size tensor.
optformula for computing auto-tuning value. Must not depend on a size tensor.
upperUpper bound on the size tensor.
Returns
IDimensionExpr denoting the value of the size tensor.
See also
IPluginV3OneBuild::getOutputShapes()

◆ operation()

IDimensionExpr const * nvinfer1::IExprBuilder::operation ( DimensionOperation  op,
IDimensionExpr const &  first,
IDimensionExpr const &  second 
)
inlinenoexcept

Get the operation.

Return pointer to IDimensionExp that represents the given operation applied to first and second. Returns nullptr if op is not a valid DimensionOperation.

Member Data Documentation

◆ mImpl

apiv::VExprBuilder* nvinfer1::IExprBuilder::mImpl
protected

The documentation for this class was generated from the following file:

  Copyright © 2024 NVIDIA Corporation
  Privacy Policy | Manage My Privacy | Do Not Sell or Share My Data | Terms of Service | Accessibility | Corporate Policies | Product Security | Contact