TensorRT  7.1.3.0
NvInfer.h File Reference
#include "NvInferRuntime.h"

Go to the source code of this file.

Classes

class  nvinfer1::Dims2
 Descriptor for two-dimensional data. More...
 
class  nvinfer1::DimsHW
 Descriptor for two-dimensional spatial data. More...
 
class  nvinfer1::Dims3
 Descriptor for three-dimensional data. More...
 
class  nvinfer1::DimsCHW
 Descriptor for data with one channel dimension and two spatial dimensions. More...
 
class  nvinfer1::Dims4
 Descriptor for four-dimensional data. More...
 
class  nvinfer1::DimsNCHW
 Descriptor for data with one index dimension, one channel dimension and two spatial dimensions. More...
 
class  nvinfer1::ITensor
 A tensor in a network definition. More...
 
class  nvinfer1::ILayer
 Base class for all layer classes in a network definition. More...
 
class  nvinfer1::IConvolutionLayer
 A convolution layer in a network definition. More...
 
class  nvinfer1::IFullyConnectedLayer
 A fully connected layer in a network definition. This layer expects an input tensor of three or more non-batch dimensions. The input is automatically reshaped into an MxV tensor X, where V is a product of the last three dimensions and M is a product of the remaining dimensions (where the product over 0 dimensions is defined as 1). For example: More...
 
class  nvinfer1::IActivationLayer
 An Activation layer in a network definition. More...
 
class  nvinfer1::IPoolingLayer
 A Pooling layer in a network definition. More...
 
class  nvinfer1::ILRNLayer
 A LRN layer in a network definition. More...
 
class  nvinfer1::IScaleLayer
 A Scale layer in a network definition. More...
 
class  nvinfer1::ISoftMaxLayer
 A Softmax layer in a network definition. More...
 
class  nvinfer1::IConcatenationLayer
 A concatenation layer in a network definition. More...
 
class  nvinfer1::IDeconvolutionLayer
 A deconvolution layer in a network definition. More...
 
class  nvinfer1::IElementWiseLayer
 A elementwise layer in a network definition. More...
 
class  nvinfer1::IGatherLayer
 
class  nvinfer1::IRNNLayer
 A RNN layer in a network definition. More...
 
class  nvinfer1::IRNNv2Layer
 An RNN layer in a network definition, version 2. More...
 
class  nvinfer1::IOutputDimensionsFormula
 Application-implemented interface to compute layer output sizes. More...
 
class  nvinfer1::IPluginLayer
 Layer type for plugins. More...
 
class  nvinfer1::IPluginV2Layer
 Layer type for pluginV2. More...
 
class  nvinfer1::IUnaryLayer
 Layer that represents an unary operation. More...
 
class  nvinfer1::IReduceLayer
 Layer that represents a reduction operator across Shape, Int32, Float, and Half tensors. More...
 
class  nvinfer1::IPaddingLayer
 Layer that represents a padding operation. More...
 
struct  nvinfer1::Permutation
 
class  nvinfer1::IShuffleLayer
 Layer type for shuffling data. More...
 
class  nvinfer1::ISliceLayer
 Slices an input tensor into an output tensor based on the offset and strides. More...
 
class  nvinfer1::IShapeLayer
 Layer type for getting shape of a tensor. More...
 
class  nvinfer1::ITopKLayer
 Layer that represents a TopK reduction. More...
 
class  nvinfer1::IMatrixMultiplyLayer
 Layer that represents a Matrix Multiplication. More...
 
class  nvinfer1::IRaggedSoftMaxLayer
 A RaggedSoftmax layer in a network definition. More...
 
class  nvinfer1::IIdentityLayer
 A layer that represents the identity function. More...
 
class  nvinfer1::IConstantLayer
 Layer that represents a constant value. More...
 
class  nvinfer1::IParametricReLULayer
 Layer that represents a parametric ReLU operation. More...
 
class  nvinfer1::IResizeLayer
 A resize layer in a network definition. More...
 
class  nvinfer1::ILoopBoundaryLayer
 
class  nvinfer1::IRecurrenceLayer
 
class  nvinfer1::ILoopOutputLayer
 
class  nvinfer1::ITripLimitLayer
 
class  nvinfer1::IIteratorLayer
 
class  nvinfer1::ILoop
 
class  nvinfer1::ISelectLayer
 
class  nvinfer1::IFillLayer
 Generate an output tensor with specified mode. More...
 
class  nvinfer1::INetworkDefinition
 A network definition for input to the builder. More...
 
class  nvinfer1::IInt8Calibrator
 Application-implemented interface for calibration. More...
 
class  nvinfer1::IInt8EntropyCalibrator
 
class  nvinfer1::IInt8EntropyCalibrator2
 
class  nvinfer1::IInt8MinMaxCalibrator
 
class  nvinfer1::IInt8LegacyCalibrator
 
class  nvinfer1::IAlgorithmIOInfo
 Carries information about input or output of the algorithm. IAlgorithmIOInfo for all the input and output along with IAlgorithmVariant denotes the variation of algorithm and can be used to select or reproduce an algorithm using IAlgorithmSelector::selectAlgorithms(). More...
 
class  nvinfer1::IAlgorithmVariant
 provides a unique 128-bit identifier, which along with the input and output information denotes the variation of algorithm and can be used to select or reproduce an algorithm, using IAlgorithmSelector::selectAlgorithms() More...
 
class  nvinfer1::IAlgorithmContext
 Describes the context and requirements, that could be fulfilled by one or more instances of IAlgorithm. More...
 
class  nvinfer1::IAlgorithm
 Describes a variation of execution of a layer. An algorithm is represented by IAlgorithmVariant and the IAlgorithmIOInfo for each of its inputs and outputs. An algorithm can be selected or reproduced using AlgorithmSelector::selectAlgorithms().". More...
 
class  nvinfer1::IAlgorithmSelector
 Interface implemented by application for selecting and reporting algorithms of a layer provided by the builder. More...
 
class  nvinfer1::IBuilderConfig
 Holds properties for configuring a builder to produce an engine. More...
 
class  nvinfer1::IBuilder
 Builds an engine from a network definition. More...
 

Namespaces

 nvinfer1
 The TensorRT API version 1 namespace.
 

Typedefs

typedef uint32_t nvinfer1::QuantizationFlags
 Represents a collection of one or more QuantizationFlag values using binary OR operations. More...
 
typedef uint32_t nvinfer1::BuilderFlags
 Represents a collection of one or more QuantizationFlag values using binary OR operations, e.g., 1U << BuilderFlag::kFP16 | 1U << BuilderFlag::kDEBUG. More...
 
typedef uint32_t nvinfer1::NetworkDefinitionCreationFlags
 This bitset is capable of representing one or more NetworkDefinitionCreationFlag flags constructed with binary OR operations. e.g., 1U << NetworkDefinitionCreationFlag::kEXPLICIT_BATCH. More...
 

Enumerations

enum  nvinfer1::LayerType : int {
  nvinfer1::LayerType::kCONVOLUTION = 0, nvinfer1::LayerType::kFULLY_CONNECTED = 1, nvinfer1::LayerType::kACTIVATION = 2, nvinfer1::LayerType::kPOOLING = 3,
  nvinfer1::LayerType::kLRN = 4, nvinfer1::LayerType::kSCALE = 5, nvinfer1::LayerType::kSOFTMAX = 6, nvinfer1::LayerType::kDECONVOLUTION = 7,
  nvinfer1::LayerType::kCONCATENATION = 8, nvinfer1::LayerType::kELEMENTWISE = 9, nvinfer1::LayerType::kPLUGIN = 10, nvinfer1::LayerType::kRNN = 11,
  nvinfer1::LayerType::kUNARY = 12, nvinfer1::LayerType::kPADDING = 13, nvinfer1::LayerType::kSHUFFLE = 14, nvinfer1::LayerType::kREDUCE = 15,
  nvinfer1::LayerType::kTOPK = 16, nvinfer1::LayerType::kGATHER = 17, nvinfer1::LayerType::kMATRIX_MULTIPLY = 18, nvinfer1::LayerType::kRAGGED_SOFTMAX = 19,
  nvinfer1::LayerType::kCONSTANT = 20, nvinfer1::LayerType::kRNN_V2 = 21, nvinfer1::LayerType::kIDENTITY = 22, nvinfer1::LayerType::kPLUGIN_V2 = 23,
  nvinfer1::LayerType::kSLICE = 24, nvinfer1::LayerType::kSHAPE = 25, nvinfer1::LayerType::kPARAMETRIC_RELU = 26, nvinfer1::LayerType::kRESIZE = 27,
  nvinfer1::LayerType::kTRIP_LIMIT = 28, nvinfer1::LayerType::kRECURRENCE = 29, nvinfer1::LayerType::kITERATOR = 30, nvinfer1::LayerType::kLOOP_OUTPUT = 31,
  nvinfer1::LayerType::kSELECT = 32, nvinfer1::LayerType::kFILL = 33
}
 The type values of layer classes. More...
 
enum  nvinfer1::PaddingMode : int {
  nvinfer1::PaddingMode::kEXPLICIT_ROUND_DOWN = 0, nvinfer1::PaddingMode::kEXPLICIT_ROUND_UP = 1, nvinfer1::PaddingMode::kSAME_UPPER = 2, nvinfer1::PaddingMode::kSAME_LOWER = 3,
  nvinfer1::PaddingMode::kCAFFE_ROUND_DOWN = 4, nvinfer1::PaddingMode::kCAFFE_ROUND_UP = 5
}
 Enumerates the modes of padding to perform in convolution, deconvolution and pooling layer, padding mode takes precedence if setPaddingMode() and setPrePadding() are also used. More...
 
enum  nvinfer1::PoolingType : int { kMAX = 0, kAVERAGE = 1, kMAX_AVERAGE_BLEND = 2 }
 The type of pooling to perform in a pooling layer.
 
enum  nvinfer1::ScaleMode : int { nvinfer1::ScaleMode::kUNIFORM = 0, nvinfer1::ScaleMode::kCHANNEL = 1, nvinfer1::ScaleMode::kELEMENTWISE = 2 }
 Controls how shift, scale and power are applied in a Scale layer. More...
 
enum  nvinfer1::ElementWiseOperation : int {
  nvinfer1::ElementWiseOperation::kSUM = 0, nvinfer1::ElementWiseOperation::kPROD = 1, nvinfer1::ElementWiseOperation::kMAX = 2, nvinfer1::ElementWiseOperation::kMIN = 3,
  nvinfer1::ElementWiseOperation::kSUB = 4, nvinfer1::ElementWiseOperation::kDIV = 5, nvinfer1::ElementWiseOperation::kPOW = 6, nvinfer1::ElementWiseOperation::kFLOOR_DIV = 7,
  nvinfer1::ElementWiseOperation::kAND = 8, nvinfer1::ElementWiseOperation::kOR = 9, nvinfer1::ElementWiseOperation::kXOR = 10, nvinfer1::ElementWiseOperation::kEQUAL = 11,
  nvinfer1::ElementWiseOperation::kGREATER = 12, nvinfer1::ElementWiseOperation::kLESS = 13
}
 Enumerates the binary operations that may be performed by an ElementWise layer. More...
 
enum  nvinfer1::RNNOperation : int { nvinfer1::RNNOperation::kRELU = 0, nvinfer1::RNNOperation::kTANH = 1, nvinfer1::RNNOperation::kLSTM = 2, nvinfer1::RNNOperation::kGRU = 3 }
 Enumerates the RNN operations that may be performed by an RNN layer. More...
 
enum  nvinfer1::RNNDirection : int { nvinfer1::RNNDirection::kUNIDIRECTION = 0, nvinfer1::RNNDirection::kBIDIRECTION = 1 }
 Enumerates the RNN direction that may be performed by an RNN layer. More...
 
enum  nvinfer1::RNNInputMode : int { nvinfer1::RNNInputMode::kLINEAR = 0, nvinfer1::RNNInputMode::kSKIP = 1 }
 Enumerates the RNN input modes that may occur with an RNN layer. More...
 
enum  nvinfer1::RNNGateType : int {
  nvinfer1::RNNGateType::kINPUT = 0, nvinfer1::RNNGateType::kOUTPUT = 1, nvinfer1::RNNGateType::kFORGET = 2, nvinfer1::RNNGateType::kUPDATE = 3,
  nvinfer1::RNNGateType::kRESET = 4, nvinfer1::RNNGateType::kCELL = 5, nvinfer1::RNNGateType::kHIDDEN = 6
}
 Identifies an individual gate within an RNN cell. More...
 
enum  nvinfer1::UnaryOperation : int {
  nvinfer1::UnaryOperation::kEXP = 0, nvinfer1::UnaryOperation::kLOG = 1, nvinfer1::UnaryOperation::kSQRT = 2, nvinfer1::UnaryOperation::kRECIP = 3,
  nvinfer1::UnaryOperation::kABS = 4, nvinfer1::UnaryOperation::kNEG = 5, nvinfer1::UnaryOperation::kSIN = 6, nvinfer1::UnaryOperation::kCOS = 7,
  nvinfer1::UnaryOperation::kTAN = 8, nvinfer1::UnaryOperation::kSINH = 9, nvinfer1::UnaryOperation::kCOSH = 10, nvinfer1::UnaryOperation::kASIN = 11,
  nvinfer1::UnaryOperation::kACOS = 12, nvinfer1::UnaryOperation::kATAN = 13, nvinfer1::UnaryOperation::kASINH = 14, nvinfer1::UnaryOperation::kACOSH = 15,
  nvinfer1::UnaryOperation::kATANH = 16, nvinfer1::UnaryOperation::kCEIL = 17, nvinfer1::UnaryOperation::kFLOOR = 18, nvinfer1::UnaryOperation::kERF = 19,
  nvinfer1::UnaryOperation::kNOT = 20
}
 Enumerates the unary operations that may be performed by a Unary layer. More...
 
enum  nvinfer1::ReduceOperation : int {
  kSUM = 0, kPROD = 1, kMAX = 2, kMIN = 3,
  kAVG = 4
}
 Enumerates the reduce operations that may be performed by a Reduce layer. More...
 
enum  nvinfer1::SliceMode : int { nvinfer1::SliceMode::kDEFAULT = 0, nvinfer1::SliceMode::kWRAP = 1 }
 Controls how ISliceLayer handles out of bounds coordinates. More...
 
enum  nvinfer1::TopKOperation : int { nvinfer1::TopKOperation::kMAX = 0, nvinfer1::TopKOperation::kMIN = 1 }
 Enumerates the operations that may be performed by a TopK layer. More...
 
enum  nvinfer1::MatrixOperation : int { nvinfer1::MatrixOperation::kNONE, nvinfer1::MatrixOperation::kTRANSPOSE, nvinfer1::MatrixOperation::kVECTOR }
 Enumerates the operations that may be performed on a tensor by IMatrixMultiplyLayer before multiplication. More...
 
enum  nvinfer1::ResizeMode : int { kNEAREST = 0, kLINEAR = 1 }
 Enumerates various modes of resize in the resize layer. Resize mode set using setResizeMode().
 
enum  nvinfer1::LoopOutput : int { nvinfer1::LoopOutput::kLAST_VALUE = 0, nvinfer1::LoopOutput::kCONCATENATE = 1, nvinfer1::LoopOutput::kREVERSE = 2 }
 Enum that describes kinds of loop outputs. More...
 
enum  nvinfer1::TripLimit : int { kCOUNT = 0, kWHILE = 1 }
 Enum that describes kinds of trip limits.
 
enum  nvinfer1::FillOperation : int { nvinfer1::FillOperation::kLINSPACE = 0, nvinfer1::FillOperation::kRANDOM_UNIFORM = 1 }
 Enumerates the tensor fill operations that may performed by a fill layer. More...
 
enum  nvinfer1::CalibrationAlgoType : int { kLEGACY_CALIBRATION = 0, kENTROPY_CALIBRATION = 1, kENTROPY_CALIBRATION_2 = 2, kMINMAX_CALIBRATION = 3 }
 Version of calibration algorithm to use. More...
 
enum  nvinfer1::QuantizationFlag : int32_t { nvinfer1::QuantizationFlag::kCALIBRATE_BEFORE_FUSION = 0 }
 List of valid flags for quantizing the network to int8. More...
 
enum  nvinfer1::BuilderFlag : int32_t {
  nvinfer1::BuilderFlag::kFP16 = 0, nvinfer1::BuilderFlag::kINT8 = 1, nvinfer1::BuilderFlag::kDEBUG = 2, nvinfer1::BuilderFlag::kGPU_FALLBACK = 3,
  nvinfer1::BuilderFlag::kSTRICT_TYPES = 4, nvinfer1::BuilderFlag::kREFIT = 5, nvinfer1::BuilderFlag::kDISABLE_TIMING_CACHE = 6, nvinfer1::BuilderFlag::kTF32 = 7
}
 List of valid modes that the builder can enable when creating an engine from a network definition. More...
 
enum  nvinfer1::ProfilingVerbosity : int32_t { nvinfer1::ProfilingVerbosity::kDEFAULT = 0, nvinfer1::ProfilingVerbosity::kNONE = 1, nvinfer1::ProfilingVerbosity::kVERBOSE = 2 }
 List of verbosity levels of layer information exposed in NVTX annotations. More...
 
enum  nvinfer1::NetworkDefinitionCreationFlag : int { nvinfer1::NetworkDefinitionCreationFlag::kEXPLICIT_BATCH = 0, nvinfer1::NetworkDefinitionCreationFlag::kEXPLICIT_PRECISION = 1 }
 List of immutable network properties expressed at network creation time. NetworkDefinitionCreationFlag is used with createNetworkV2 to specify immutable properties of the network. The createNetwork() function always had an implicit batch dimension being specified by the maxBatchSize builder parameter. createNetworkV2 with kDEFAULT flag mimics that behaviour. More...
 

Functions

template<>
constexpr int nvinfer1::EnumMax< LayerType > ()
 Maximum number of elements in LayerType enum. More...
 
template<>
constexpr int nvinfer1::EnumMax< PaddingMode > ()
 Maximum number of elements in PaddingMode enum. More...
 
template<>
constexpr int nvinfer1::EnumMax< PoolingType > ()
 Maximum number of elements in PoolingType enum. More...
 
template<>
constexpr int nvinfer1::EnumMax< ScaleMode > ()
 Maximum number of elements in ScaleMode enum. More...
 
template<>
constexpr int nvinfer1::EnumMax< ElementWiseOperation > ()
 Maximum number of elements in ElementWiseOperation enum. More...
 
template<>
constexpr int nvinfer1::EnumMax< RNNOperation > ()
 Maximum number of elements in RNNOperation enum. More...
 
template<>
constexpr int nvinfer1::EnumMax< RNNDirection > ()
 Maximum number of elements in RNNDirection enum. More...
 
template<>
constexpr int nvinfer1::EnumMax< RNNInputMode > ()
 Maximum number of elements in RNNInputMode enum. More...
 
template<>
constexpr int nvinfer1::EnumMax< RNNGateType > ()
 Maximum number of elements in RNNGateType enum. More...
 
template<>
constexpr int nvinfer1::EnumMax< UnaryOperation > ()
 Maximum number of elements in UnaryOperation enum. More...
 
template<>
constexpr int nvinfer1::EnumMax< ReduceOperation > ()
 Maximum number of elements in ReduceOperation enum. More...
 
template<>
constexpr int nvinfer1::EnumMax< SliceMode > ()
 Maximum number of elements in SliceMode enum. More...
 
template<>
constexpr int nvinfer1::EnumMax< TopKOperation > ()
 Maximum number of elements in TopKOperation enum. More...
 
template<>
constexpr int nvinfer1::EnumMax< MatrixOperation > ()
 Maximum number of elements in MatrixOperation enum. More...
 
template<>
constexpr int nvinfer1::EnumMax< ResizeMode > ()
 Maximum number of elements in ResizeMode enum. More...
 
template<>
constexpr int nvinfer1::EnumMax< LoopOutput > ()
 Maximum number of elements in LoopOutput enum. More...
 
template<>
constexpr int nvinfer1::EnumMax< TripLimit > ()
 Maximum number of elements in TripLimit enum. More...
 
template<>
constexpr int nvinfer1::EnumMax< FillOperation > ()
 Maximum number of elements in FillOperation enum. More...
 
template<>
constexpr int nvinfer1::EnumMax< CalibrationAlgoType > ()
 Maximum number of elements in CalibrationAlgoType enum. More...
 
template<>
constexpr int nvinfer1::EnumMax< QuantizationFlag > ()
 Maximum number of quantization flags in QuantizationFlag enum. More...
 
template<>
constexpr int nvinfer1::EnumMax< BuilderFlag > ()
 Maximum number of builder flags in BuilderFlag enum. More...
 
template<>
constexpr int32_t nvinfer1::EnumMax< ProfilingVerbosity > ()
 Maximum number of profile verbosity levels in ProfilingVerbosity enum. More...
 
template<>
constexpr int nvinfer1::EnumMax< NetworkDefinitionCreationFlag > ()
 
IBuilder * nvinfer1::anonymous_namespace{NvInfer.h}::createInferBuilder (ILogger &logger)
 Create an instance of an IBuilder class. More...
 

Detailed Description

TensorRT Versioning follows Semantic Versioning Guidelines specified here: https://semver.org/

This is the top-level API file for TensorRT.