TensorRT  5.1.3.4
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Macros Pages
NvInfer.h File Reference
#include <cstddef>
#include <cstdint>

Go to the source code of this file.

Classes

class  nvinfer1::Dims
 Structure to define the dimensions of a tensor. More...
 
class  nvinfer1::Dims2
 Descriptor for two-dimensional data. More...
 
class  nvinfer1::DimsHW
 Descriptor for two-dimensional spatial data. More...
 
class  nvinfer1::Dims3
 Descriptor for three-dimensional data. More...
 
class  nvinfer1::DimsCHW
 Descriptor for data with one channel dimension and two spatial dimensions. More...
 
class  nvinfer1::Dims4
 Descriptor for four-dimensional data. More...
 
class  nvinfer1::DimsNCHW
 Descriptor for data with one index dimension, one channel dimension and two spatial dimensions. More...
 
class  nvinfer1::Weights
 An array of weights used as a layer parameter. More...
 
class  nvinfer1::IHostMemory
 Class to handle library allocated memory that is accessible to the user. More...
 
class  nvinfer1::ITensor
 A tensor in a network definition. More...
 
class  nvinfer1::ILayer
 Base class for all layer classes in a network definition. More...
 
class  nvinfer1::IConvolutionLayer
 A convolution layer in a network definition. More...
 
class  nvinfer1::IFullyConnectedLayer
 A fully connected layer in a network definition. This layer expects an input tensor of three or more non-batch dimensions. The input is automatically reshaped into an MxV tensor X, where V is a product of the last three dimensions and M is a product of the remaining dimensions (where the product over 0 dimensions is defined as 1). For example: More...
 
class  nvinfer1::IActivationLayer
 An Activation layer in a network definition. More...
 
class  nvinfer1::IPoolingLayer
 A Pooling layer in a network definition. More...
 
class  nvinfer1::ILRNLayer
 A LRN layer in a network definition. More...
 
class  nvinfer1::IScaleLayer
 A Scale layer in a network definition. More...
 
class  nvinfer1::ISoftMaxLayer
 A Softmax layer in a network definition. More...
 
class  nvinfer1::IConcatenationLayer
 A concatenation layer in a network definition. More...
 
class  nvinfer1::IDeconvolutionLayer
 A deconvolution layer in a network definition. More...
 
class  nvinfer1::IElementWiseLayer
 A elementwise layer in a network definition. More...
 
class  nvinfer1::IGatherLayer
 
class  nvinfer1::IRNNLayer
 A RNN layer in a network definition. More...
 
class  nvinfer1::IRNNv2Layer
 An RNN layer in a network definition, version 2. More...
 
class  nvinfer1::IOutputDimensionsFormula
 Application-implemented interface to compute layer output sizes. More...
 
class  nvinfer1::IPlugin
 Plugin class for user-implemented layers. More...
 
class  nvinfer1::IPluginExt
 Plugin class for user-implemented layers. More...
 
class  nvinfer1::IPluginV2
 Plugin class for user-implemented layers. More...
 
class  nvinfer1::IPluginV2Ext
 Plugin class for user-implemented layers. More...
 
class  nvinfer1::IPluginLayer
 Layer type for plugins. More...
 
class  nvinfer1::IPluginV2Layer
 Layer type for pluginV2. More...
 
class  nvinfer1::PluginField
 Structure containing plugin attribute field names and associated data This information can be parsed to decode necessary plugin metadata. More...
 
struct  nvinfer1::PluginFieldCollection
 
class  nvinfer1::IPluginCreator
 Plugin creator class for user implemented layers. More...
 
class  nvinfer1::IPluginRegistry
 Single registration point for all plugins in an application. It is used to find plugin implementations during engine deserialization. Internally, the plugin registry is considered to be a singleton so all plugins in an application are part of the same global registry. Note that the plugin registry is only supported for plugins of type IPluginV2 and should also have a corresponding IPluginCreator implementation. More...
 
class  nvinfer1::IUnaryLayer
 Layer that represents an unary operation. More...
 
class  nvinfer1::IReduceLayer
 Layer that represents a reduction operator. More...
 
class  nvinfer1::IPaddingLayer
 Layer that represents a padding operation. More...
 
struct  nvinfer1::Permutation
 
class  nvinfer1::IShuffleLayer
 Layer type for shuffling data. More...
 
class  nvinfer1::ISliceLayer
 
class  nvinfer1::ITopKLayer
 Layer that represents a TopK reduction. More...
 
class  nvinfer1::IMatrixMultiplyLayer
 Layer that represents a Matrix Multiplication. More...
 
class  nvinfer1::IRaggedSoftMaxLayer
 A RaggedSoftmax layer in a network definition. More...
 
class  nvinfer1::IIdentityLayer
 A layer that represents the identity function. More...
 
class  nvinfer1::IConstantLayer
 Layer that represents a constant value. More...
 
class  nvinfer1::INetworkDefinition
 A network definition for input to the builder. More...
 
class  nvinfer1::IProfiler
 Application-implemented interface for profiling. More...
 
class  nvinfer1::IExecutionContext
 Context for executing inference using an engine. More...
 
class  nvinfer1::ICudaEngine
 An engine for executing inference on a built network. More...
 
class  nvinfer1::IInt8Calibrator
 Application-implemented interface for calibration. More...
 
class  nvinfer1::IInt8EntropyCalibrator
 
class  nvinfer1::IInt8EntropyCalibrator2
 
class  nvinfer1::IInt8LegacyCalibrator
 
class  nvinfer1::IGpuAllocator
 Application-implemented class for controlling allocation on the GPU. More...
 
class  nvinfer1::IBuilder
 Builds an engine from a network definition. More...
 
class  nvinfer1::IRefitter
 Updates weights in an engine. More...
 
class  nvinfer1::IPluginFactory
 Plugin factory for deserialization. More...
 
class  nvinfer1::IRuntime
 Allows a serialized engine to be deserialized. More...
 
class  nvinfer1::ILogger
 Application-implemented logging interface for the builder, engine and runtime. More...
 
class  nvinfer1::PluginRegistrar< T >
 Register the plugin creator to the registry The static registry object will be instantiated when the plugin library is loaded. This static object will register all creators available in the library to the registry. More...
 

Namespaces

 nvinfer1
 The TensorRT API version 1 namespace.
 

Macros

#define NV_TENSORRT_MAJOR   5
 TensorRT major version.
 
#define NV_TENSORRT_MINOR   1
 TensorRT minor version.
 
#define NV_TENSORRT_PATCH   3
 TensorRT patch version.
 
#define NV_TENSORRT_BUILD   4
 TensorRT build number.
 
#define NV_TENSORRT_SONAME_MAJOR   5
 Shared object library major version number.
 
#define NV_TENSORRT_SONAME_MINOR   1
 Shared object library minor version number.
 
#define NV_TENSORRT_SONAME_PATCH   3
 Shared object library patch version number.
 
#define _TENSORRT_FINAL
 
#define _TENSORRT_OVERRIDE
 Defines which symbols are exported.
 
#define TENSORRTAPI
 
#define REGISTER_TENSORRT_PLUGIN(name)   static nvinfer1::PluginRegistrar<name> pluginRegistrar##name {}
 

Typedefs

typedef struct CUstream_st * cudaStream_t
 Forward declaration of cudaStream_t.
 
typedef struct CUevent_st * cudaEvent_t
 Forward declaration of cudaEvent_t.
 

Enumerations

enum  nvinfer1::DataType : int { nvinfer1::DataType::kFLOAT = 0, nvinfer1::DataType::kHALF = 1, nvinfer1::DataType::kINT8 = 2, nvinfer1::DataType::kINT32 = 3 }
 The type of weights and tensors. More...
 
enum  nvinfer1::DeviceType : int { nvinfer1::DeviceType::kGPU, nvinfer1::DeviceType::kDLA }
 The device that this layer/network will execute on. More...
 
enum  nvinfer1::DimensionType : int { nvinfer1::DimensionType::kSPATIAL = 0, nvinfer1::DimensionType::kCHANNEL = 1, nvinfer1::DimensionType::kINDEX = 2, nvinfer1::DimensionType::kSEQUENCE = 3 }
 The type of data encoded across this dimension. More...
 
enum  nvinfer1::LayerType : int {
  nvinfer1::LayerType::kCONVOLUTION = 0, nvinfer1::LayerType::kFULLY_CONNECTED = 1, nvinfer1::LayerType::kACTIVATION = 2, nvinfer1::LayerType::kPOOLING = 3,
  nvinfer1::LayerType::kLRN = 4, nvinfer1::LayerType::kSCALE = 5, nvinfer1::LayerType::kSOFTMAX = 6, nvinfer1::LayerType::kDECONVOLUTION = 7,
  nvinfer1::LayerType::kCONCATENATION = 8, nvinfer1::LayerType::kELEMENTWISE = 9, nvinfer1::LayerType::kPLUGIN = 10, nvinfer1::LayerType::kRNN = 11,
  nvinfer1::LayerType::kUNARY = 12, nvinfer1::LayerType::kPADDING = 13, nvinfer1::LayerType::kSHUFFLE = 14, nvinfer1::LayerType::kREDUCE = 15,
  nvinfer1::LayerType::kTOPK = 16, nvinfer1::LayerType::kGATHER = 17, nvinfer1::LayerType::kMATRIX_MULTIPLY = 18, nvinfer1::LayerType::kRAGGED_SOFTMAX = 19,
  nvinfer1::LayerType::kCONSTANT = 20, nvinfer1::LayerType::kRNN_V2 = 21, nvinfer1::LayerType::kIDENTITY = 22, nvinfer1::LayerType::kPLUGIN_V2 = 23,
  nvinfer1::LayerType::kSLICE = 24
}
 The type values of layer classes. More...
 
enum  nvinfer1::TensorLocation : int { nvinfer1::TensorLocation::kDEVICE = 0, nvinfer1::TensorLocation::kHOST = 1 }
 The location for tensor data storage, device or host. More...
 
enum  nvinfer1::PaddingMode : int {
  nvinfer1::PaddingMode::kEXPLICIT_ROUND_DOWN = 0, nvinfer1::PaddingMode::kEXPLICIT_ROUND_UP = 1, nvinfer1::PaddingMode::kSAME_UPPER = 2, nvinfer1::PaddingMode::kSAME_LOWER = 3,
  nvinfer1::PaddingMode::kCAFFE_ROUND_DOWN = 4, nvinfer1::PaddingMode::kCAFFE_ROUND_UP = 5
}
 Enumerates the modes of padding to perform in convolution, deconvolution and pooling layer, padding mode gets precedence if setPaddingMode() and setPrePadding() are also used. More...
 
enum  nvinfer1::ActivationType : int {
  nvinfer1::ActivationType::kRELU = 0, nvinfer1::ActivationType::kSIGMOID = 1, nvinfer1::ActivationType::kTANH = 2, nvinfer1::ActivationType::kLEAKY_RELU = 3,
  nvinfer1::ActivationType::kELU = 4, nvinfer1::ActivationType::kSELU = 5, nvinfer1::ActivationType::kSOFTSIGN = 6, nvinfer1::ActivationType::kSOFTPLUS = 7,
  nvinfer1::ActivationType::kCLIP = 8, nvinfer1::ActivationType::kHARD_SIGMOID = 9, nvinfer1::ActivationType::kSCALED_TANH = 10, nvinfer1::ActivationType::kTHRESHOLDED_RELU = 11
}
 Enumerates the types of activation to perform in an activation layer. More...
 
enum  nvinfer1::PoolingType : int { kMAX = 0, kAVERAGE = 1, kMAX_AVERAGE_BLEND = 2 }
 The type of pooling to perform in a pooling layer.
 
enum  nvinfer1::ScaleMode : int { nvinfer1::ScaleMode::kUNIFORM = 0, nvinfer1::ScaleMode::kCHANNEL = 1, nvinfer1::ScaleMode::kELEMENTWISE = 2 }
 Controls how shift, scale and power are applied in a Scale layer. More...
 
enum  nvinfer1::ElementWiseOperation : int {
  nvinfer1::ElementWiseOperation::kSUM = 0, nvinfer1::ElementWiseOperation::kPROD = 1, nvinfer1::ElementWiseOperation::kMAX = 2, nvinfer1::ElementWiseOperation::kMIN = 3,
  nvinfer1::ElementWiseOperation::kSUB = 4, nvinfer1::ElementWiseOperation::kDIV = 5, nvinfer1::ElementWiseOperation::kPOW = 6
}
 Enumerates the binary operations that may be performed by an ElementWise layer. More...
 
enum  nvinfer1::RNNOperation : int { nvinfer1::RNNOperation::kRELU = 0, nvinfer1::RNNOperation::kTANH = 1, nvinfer1::RNNOperation::kLSTM = 2, nvinfer1::RNNOperation::kGRU = 3 }
 Enumerates the RNN operations that may be performed by an RNN layer. More...
 
enum  nvinfer1::RNNDirection : int { nvinfer1::RNNDirection::kUNIDIRECTION = 0, nvinfer1::RNNDirection::kBIDIRECTION = 1 }
 Enumerates the RNN direction that may be performed by an RNN layer. More...
 
enum  nvinfer1::RNNInputMode : int { nvinfer1::RNNInputMode::kLINEAR = 0, nvinfer1::RNNInputMode::kSKIP = 1 }
 Enumerates the RNN input modes that may occur with an RNN layer. More...
 
enum  nvinfer1::RNNGateType : int {
  nvinfer1::RNNGateType::kINPUT = 0, nvinfer1::RNNGateType::kOUTPUT = 1, nvinfer1::RNNGateType::kFORGET = 2, nvinfer1::RNNGateType::kUPDATE = 3,
  nvinfer1::RNNGateType::kRESET = 4, nvinfer1::RNNGateType::kCELL = 5, nvinfer1::RNNGateType::kHIDDEN = 6
}
 Identifies an individual gate within an RNN cell. More...
 
enum  nvinfer1::PluginFormat : uint8_t { nvinfer1::PluginFormat::kNCHW = 0, nvinfer1::PluginFormat::kNC2HW2 = 1, nvinfer1::PluginFormat::kNHWC8 = 2 }
 
enum  nvinfer1::PluginFieldType : int {
  nvinfer1::PluginFieldType::kFLOAT16 = 0, nvinfer1::PluginFieldType::kFLOAT32 = 1, nvinfer1::PluginFieldType::kFLOAT64 = 2, nvinfer1::PluginFieldType::kINT8 = 3,
  nvinfer1::PluginFieldType::kINT16 = 4, nvinfer1::PluginFieldType::kINT32 = 5, nvinfer1::PluginFieldType::kCHAR = 6, nvinfer1::PluginFieldType::kDIMS = 7,
  kUNKNOWN = 8
}
 
enum  nvinfer1::UnaryOperation : int {
  nvinfer1::UnaryOperation::kEXP = 0, nvinfer1::UnaryOperation::kLOG = 1, nvinfer1::UnaryOperation::kSQRT = 2, nvinfer1::UnaryOperation::kRECIP = 3,
  nvinfer1::UnaryOperation::kABS = 4, nvinfer1::UnaryOperation::kNEG = 5, nvinfer1::UnaryOperation::kSIN = 6, nvinfer1::UnaryOperation::kCOS = 7,
  nvinfer1::UnaryOperation::kTAN = 8, nvinfer1::UnaryOperation::kSINH = 9, nvinfer1::UnaryOperation::kCOSH = 10, nvinfer1::UnaryOperation::kASIN = 11,
  nvinfer1::UnaryOperation::kACOS = 12, nvinfer1::UnaryOperation::kATAN = 13, nvinfer1::UnaryOperation::kASINH = 14, nvinfer1::UnaryOperation::kACOSH = 15,
  nvinfer1::UnaryOperation::kATANH = 16, nvinfer1::UnaryOperation::kCEIL = 17, nvinfer1::UnaryOperation::kFLOOR = 18
}
 Enumerates the unary operations that may be performed by a Unary layer. More...
 
enum  nvinfer1::ReduceOperation : int {
  kSUM = 0, kPROD = 1, kMAX = 2, kMIN = 3,
  kAVG = 4
}
 Enumerates the reduce operations that may be performed by a Reduce layer.
 
enum  nvinfer1::TopKOperation : int { nvinfer1::TopKOperation::kMAX = 0, nvinfer1::TopKOperation::kMIN = 1 }
 Enumerates the operations that may be performed by a TopK layer. More...
 
enum  nvinfer1::MatrixOperation : int { nvinfer1::MatrixOperation::kNONE, nvinfer1::MatrixOperation::kTRANSPOSE, nvinfer1::MatrixOperation::kVECTOR }
 Enumerates the operations that may be performed on a tensor by IMatrixMultiplyLayer before multiplication. More...
 
enum  nvinfer1::CalibrationAlgoType : int { kLEGACY_CALIBRATION = 0, kENTROPY_CALIBRATION = 1, kENTROPY_CALIBRATION_2 = 2 }
 Version of calibration algorithm to use. More...
 
enum  nvinfer1::EngineCapability { nvinfer1::EngineCapability::kDEFAULT = 0, nvinfer1::EngineCapability::kSAFE_GPU = 1, nvinfer1::EngineCapability::kSAFE_DLA = 2 }
 List of supported engine capability flows. More...
 
enum  nvinfer1::WeightsRole : int {
  nvinfer1::WeightsRole::kKERNEL = 0, nvinfer1::WeightsRole::kBIAS = 1, nvinfer1::WeightsRole::kSHIFT = 2, nvinfer1::WeightsRole::kSCALE = 3,
  nvinfer1::WeightsRole::kCONSTANT = 4
}
 How a layer uses particular Weights. More...
 

Functions

template<typename T >
int nvinfer1::EnumMax ()
 Maximum number of elements in an enumeration type.
 
template<>
int nvinfer1::EnumMax< DataType > ()
 Maximum number of elements in DataType enum. More...
 
template<>
int nvinfer1::EnumMax< DeviceType > ()
 Maximum number of elements in DeviceType enum. More...
 
template<>
int nvinfer1::EnumMax< DimensionType > ()
 Maximum number of elements in DimensionType enum. More...
 
template<>
int nvinfer1::EnumMax< LayerType > ()
 Maximum number of elements in LayerType enum. More...
 
template<>
int nvinfer1::EnumMax< TensorLocation > ()
 Maximum number of elements in TensorLocation enum. More...
 
template<>
int nvinfer1::EnumMax< PaddingMode > ()
 Maximum number of elements in PaddingMode enum. More...
 
template<>
int nvinfer1::EnumMax< ActivationType > ()
 Maximum number of elements in ActivationType enum. More...
 
template<>
int nvinfer1::EnumMax< PoolingType > ()
 Maximum number of elements in PoolingType enum. More...
 
template<>
int nvinfer1::EnumMax< ScaleMode > ()
 Maximum number of elements in ScaleMode enum. More...
 
template<>
int nvinfer1::EnumMax< ElementWiseOperation > ()
 Maximum number of elements in ElementWiseOperation enum. More...
 
template<>
int nvinfer1::EnumMax< RNNOperation > ()
 Maximum number of elements in RNNOperation enum. More...
 
template<>
int nvinfer1::EnumMax< RNNDirection > ()
 Maximum number of elements in RNNDirection enum. More...
 
template<>
int nvinfer1::EnumMax< RNNInputMode > ()
 Maximum number of elements in RNNInputMode enum. More...
 
template<>
int nvinfer1::EnumMax< RNNGateType > ()
 Maximum number of elements in RNNGateType enum. More...
 
template<>
int nvinfer1::EnumMax< PluginFormat > ()
 Maximum number of elements in PluginFormat enum. More...
 
template<>
int nvinfer1::EnumMax< UnaryOperation > ()
 Maximum number of elements in UnaryOperation enum. More...
 
template<>
int nvinfer1::EnumMax< ReduceOperation > ()
 Maximum number of elements in ReduceOperation enum. More...
 
template<>
int nvinfer1::EnumMax< TopKOperation > ()
 Maximum number of elements in TopKOperation enum. More...
 
template<>
int nvinfer1::EnumMax< MatrixOperation > ()
 Maximum number of elements in MatrixOperation enum. More...
 
template<>
int nvinfer1::EnumMax< CalibrationAlgoType > ()
 Maximum number of elements in CalibrationAlgoType enum. More...
 
template<>
int nvinfer1::EnumMax< EngineCapability > ()
 Maximum number of elements in EngineCapability enum. More...
 
template<>
int nvinfer1::EnumMax< WeightsRole > ()
 Maximum number of elements in WeightsRole enum. More...
 
template<>
int nvinfer1::EnumMax< ILogger::Severity > ()
 Maximum number of elements in ILogger::Severity enum. More...
 
TENSORRTAPI void * createInferBuilder_INTERNAL (void *logger, int version)
 Internal C entry point for creating IBuilder.
 
TENSORRTAPI void * createInferRefitter_INTERNAL (void *engine, void *logger, int version)
 Internal C entry point for creating IRefitter.
 
TENSORRTAPI void * createInferRuntime_INTERNAL (void *logger, int version)
 Internal C entry point for creating IRuntime.
 
TENSORRTAPI nvinfer1::ILoggergetLogger ()
 Return the logger object.
 
TENSORRTAPI int getInferLibVersion ()
 Return the library version number. More...
 
TENSORRTAPI
nvinfer1::IPluginRegistry
getPluginRegistry ()
 Return the plugin registry.
 

Detailed Description

This is the top-level API file for TensorRT.

Function Documentation

TENSORRTAPI int getInferLibVersion ( )

Return the library version number.

The format is as for TENSORRT_VERSION: (TENSORRT_MAJOR * 1000) + (TENSORRT_MINOR * 100) + TENSOR_PATCH.