TensorRT
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Macros Pages
NvInfer.h File Reference
#include <cstddef>
#include <cstdint>

Go to the source code of this file.

Classes

class  nvinfer1::Dims
 Structure to define the dimensions of a tensor. More...
 
class  nvinfer1::Dims2
 Descriptor for two-dimensional data. More...
 
class  nvinfer1::DimsHW
 Descriptor for two-dimensional spatial data. More...
 
class  nvinfer1::Dims3
 Descriptor for three-dimensional data. More...
 
class  nvinfer1::DimsCHW
 Descriptor for data with one channel dimension and two spatial dimensions. More...
 
class  nvinfer1::Dims4
 Descriptor for four-dimensional data. More...
 
class  nvinfer1::DimsNCHW
 Descriptor for data with one index dimension, one channel dimension and two spatial dimensions. More...
 
class  nvinfer1::Weights
 An array of weights used as a layer parameter. More...
 
class  nvinfer1::IHostMemory
 Class to handle library allocated memory that is accessible to the user. More...
 
class  nvinfer1::ITensor
 A tensor in a network definition. More...
 
class  nvinfer1::ILayer
 Base class for all layer classes in a network definition. More...
 
class  nvinfer1::IConvolutionLayer
 A convolution layer in a network definition. More...
 
class  nvinfer1::IFullyConnectedLayer
 A fully connected layer in a network definition. This layer expects an input tensor of three or more non-batch dimensions. The input is automatically reshaped into an MxV tensor X, where V is a product of the last three dimensions and M is a product of the remaining dimensions (where the product over 0 dimensions is defined as 1). For example: More...
 
class  nvinfer1::IActivationLayer
 An Activation layer in a network definition. More...
 
class  nvinfer1::IPoolingLayer
 A Pooling layer in a network definition. More...
 
class  nvinfer1::ILRNLayer
 A LRN layer in a network definition. More...
 
class  nvinfer1::IScaleLayer
 A Scale layer in a network definition. More...
 
class  nvinfer1::ISoftMaxLayer
 A Softmax layer in a network definition. More...
 
class  nvinfer1::IConcatenationLayer
 A concatenation layer in a network definition. More...
 
class  nvinfer1::IDeconvolutionLayer
 A deconvolution layer in a network definition. More...
 
class  nvinfer1::IElementWiseLayer
 A elementwise layer in a network definition. More...
 
class  nvinfer1::IGatherLayer
 
class  nvinfer1::IRNNLayer
 A RNN layer in a network definition. More...
 
class  nvinfer1::IRNNv2Layer
 An RNN layer in a network definition, version 2. More...
 
class  nvinfer1::IOutputDimensionsFormula
 Application-implemented interface to compute layer output sizes. More...
 
class  nvinfer1::IPlugin
 Plugin class for user-implemented layers. More...
 
class  nvinfer1::IPluginExt
 Plugin class for user-implemented layers. More...
 
class  nvinfer1::IPluginLayer
 Layer type for plugins. More...
 
class  nvinfer1::IUnaryLayer
 Layer that represents an unary operation. More...
 
class  nvinfer1::IReduceLayer
 Layer that represents a reduction operator. More...
 
class  nvinfer1::IPaddingLayer
 Layer that represents a padding operation. More...
 
struct  nvinfer1::Permutation
 
class  nvinfer1::IShuffleLayer
 Layer type for shuffling data. More...
 
class  nvinfer1::ITopKLayer
 Layer that represents a TopK reduction. More...
 
class  nvinfer1::IMatrixMultiplyLayer
 Layer that represents a Matrix Multiplication. More...
 
class  nvinfer1::IRaggedSoftMaxLayer
 A RaggedSoftmax layer in a network definition. More...
 
class  nvinfer1::IConstantLayer
 Layer that represents a constant value. More...
 
class  nvinfer1::INetworkDefinition
 A network definition for input to the builder. More...
 
class  nvinfer1::IProfiler
 Application-implemented interface for profiling. More...
 
class  nvinfer1::IExecutionContext
 Context for executing inference using an engine. More...
 
class  nvinfer1::ICudaEngine
 An engine for executing inference on a built network. More...
 
class  nvinfer1::IInt8Calibrator
 Application-implemented interface for calibration. More...
 
class  nvinfer1::IInt8EntropyCalibrator
 
class  nvinfer1::IInt8LegacyCalibrator
 
class  nvinfer1::IGpuAllocator
 Application-implemented class for controlling allocation on the GPU. More...
 
class  nvinfer1::IBuilder
 Builds an engine from a network definition. More...
 
class  nvinfer1::IPluginFactory
 Plugin factory for deserialization. More...
 
class  nvinfer1::IRuntime
 Allows a serialized engine to be deserialized. More...
 
class  nvinfer1::ILogger
 Application-implemented logging interface for the builder, engine and runtime. More...
 

Namespaces

 nvinfer1
 The TensorRT API version 1 namespace.
 

Macros

#define NV_TENSORRT_MAJOR   4
 TensorRT major version.
 
#define NV_TENSORRT_MINOR   0
 TensorRT minor version.
 
#define NV_TENSORRT_PATCH   1
 TensorRT patch version.
 
#define NV_TENSORRT_BUILD   3
 TensorRT build number.
 
#define NV_TENSORRT_SONAME_MAJOR   4
 Shared object library major version number.
 
#define NV_TENSORRT_SONAME_MINOR   1
 Shared object library minor version number.
 
#define NV_TENSORRT_SONAME_PATCH   2
 Shared object library patch version number.
 
#define _TENSORRT_FINAL
 Defines which symbols are exported.
 
#define TENSORRTAPI
 

Typedefs

typedef struct CUstream_st * cudaStream_t
 Forward declaration of cudaStream_t.
 
typedef struct CUevent_st * cudaEvent_t
 Forward declaration of cudaEvent_t.
 

Enumerations

enum  nvinfer1::DataType : int { nvinfer1::DataType::kFLOAT = 0, nvinfer1::DataType::kHALF = 1, nvinfer1::DataType::kINT8 = 2, nvinfer1::DataType::kINT32 = 3 }
 The type of weights and tensors. More...
 
enum  nvinfer1::DimensionType : int { nvinfer1::DimensionType::kSPATIAL = 0, nvinfer1::DimensionType::kCHANNEL = 1, nvinfer1::DimensionType::kINDEX = 2, nvinfer1::DimensionType::kSEQUENCE = 3 }
 The type of data encoded across this dimension. More...
 
enum  nvinfer1::LayerType : int {
  nvinfer1::LayerType::kCONVOLUTION = 0, nvinfer1::LayerType::kFULLY_CONNECTED = 1, nvinfer1::LayerType::kACTIVATION = 2, nvinfer1::LayerType::kPOOLING = 3,
  nvinfer1::LayerType::kLRN = 4, nvinfer1::LayerType::kSCALE = 5, nvinfer1::LayerType::kSOFTMAX = 6, nvinfer1::LayerType::kDECONVOLUTION = 7,
  nvinfer1::LayerType::kCONCATENATION = 8, nvinfer1::LayerType::kELEMENTWISE = 9, nvinfer1::LayerType::kPLUGIN = 10, nvinfer1::LayerType::kRNN = 11,
  nvinfer1::LayerType::kUNARY = 12, nvinfer1::LayerType::kPADDING = 13, nvinfer1::LayerType::kSHUFFLE = 14, nvinfer1::LayerType::kREDUCE = 15,
  nvinfer1::LayerType::kTOPK = 16, nvinfer1::LayerType::kGATHER = 17, nvinfer1::LayerType::kMATRIX_MULTIPLY = 18, nvinfer1::LayerType::kRAGGED_SOFTMAX = 19,
  nvinfer1::LayerType::kCONSTANT = 20, nvinfer1::LayerType::kRNN_V2 = 21
}
 The type values of layer classes. More...
 
enum  nvinfer1::TensorLocation : int { nvinfer1::TensorLocation::kDEVICE = 0, nvinfer1::TensorLocation::kHOST = 1 }
 The location for tensor data storage, device or host. More...
 
enum  nvinfer1::ActivationType : int { nvinfer1::ActivationType::kRELU = 0, nvinfer1::ActivationType::kSIGMOID = 1, nvinfer1::ActivationType::kTANH = 2 }
 Enumerates the types of activation to perform in an activation layer. More...
 
enum  nvinfer1::PoolingType : int { kMAX = 0, kAVERAGE = 1, kMAX_AVERAGE_BLEND = 2 }
 The type of pooling to perform in a pooling layer.
 
enum  nvinfer1::ScaleMode : int { nvinfer1::ScaleMode::kUNIFORM = 0, nvinfer1::ScaleMode::kCHANNEL = 1, nvinfer1::ScaleMode::kELEMENTWISE = 2 }
 Controls how scale is applied in a Scale layer. More...
 
enum  nvinfer1::ElementWiseOperation : int {
  nvinfer1::ElementWiseOperation::kSUM = 0, nvinfer1::ElementWiseOperation::kPROD = 1, nvinfer1::ElementWiseOperation::kMAX = 2, nvinfer1::ElementWiseOperation::kMIN = 3,
  nvinfer1::ElementWiseOperation::kSUB = 4, nvinfer1::ElementWiseOperation::kDIV = 5, nvinfer1::ElementWiseOperation::kPOW = 6
}
 Enumerates the binary operations that may be performed by an ElementWise layer. More...
 
enum  nvinfer1::RNNOperation : int { nvinfer1::RNNOperation::kRELU = 0, nvinfer1::RNNOperation::kTANH = 1, nvinfer1::RNNOperation::kLSTM = 2, nvinfer1::RNNOperation::kGRU = 3 }
 Enumerates the RNN operations that may be performed by an RNN layer. More...
 
enum  nvinfer1::RNNDirection : int { nvinfer1::RNNDirection::kUNIDIRECTION = 0, nvinfer1::RNNDirection::kBIDIRECTION = 1 }
 Enumerates the RNN direction that may be performed by an RNN layer. More...
 
enum  nvinfer1::RNNInputMode : int { nvinfer1::RNNInputMode::kLINEAR = 0, nvinfer1::RNNInputMode::kSKIP = 1 }
 Enumerates the RNN input modes that may occur with an RNN layer. More...
 
enum  nvinfer1::RNNGateType : int {
  nvinfer1::RNNGateType::kINPUT = 0, nvinfer1::RNNGateType::kOUTPUT = 1, nvinfer1::RNNGateType::kFORGET = 2, nvinfer1::RNNGateType::kUPDATE = 3,
  nvinfer1::RNNGateType::kRESET = 4, nvinfer1::RNNGateType::kCELL = 5, nvinfer1::RNNGateType::kHIDDEN = 6
}
 Identifies an individual gate within an RNN cell. More...
 
enum  nvinfer1::PluginFormat : uint8_t { nvinfer1::PluginFormat::kNCHW = 0, nvinfer1::PluginFormat::kNC2HW2 = 1, nvinfer1::PluginFormat::kNHWC8 = 2 }
 
enum  nvinfer1::UnaryOperation : int {
  nvinfer1::UnaryOperation::kEXP = 0, nvinfer1::UnaryOperation::kLOG = 1, nvinfer1::UnaryOperation::kSQRT = 2, nvinfer1::UnaryOperation::kRECIP = 3,
  nvinfer1::UnaryOperation::kABS = 4, nvinfer1::UnaryOperation::kNEG = 5
}
 Enumerates the unary operations that may be performed by a Unary layer. More...
 
enum  nvinfer1::ReduceOperation : int {
  kSUM = 0, kPROD = 1, kMAX = 2, kMIN = 3,
  kAVG = 4
}
 Enumerates the reduce operations that may be performed by a Reduce layer.
 
enum  nvinfer1::TopKOperation : int { nvinfer1::TopKOperation::kMAX = 0, nvinfer1::TopKOperation::kMIN = 1 }
 Enumerates the operations that may be performed by a TopK layer. More...
 
enum  nvinfer1::CalibrationAlgoType : int { kLEGACY_CALIBRATION = 0, kENTROPY_CALIBRATION = 1 }
 Version of calibration algorithm to use. More...
 

Functions

template<typename T >
int nvinfer1::EnumMax ()
 Maximum number of elements in an enumeration type.
 
template<>
int nvinfer1::EnumMax< DataType > ()
 Maximum number of elements in DataType enum. More...
 
template<>
int nvinfer1::EnumMax< DimensionType > ()
 Maximum number of elements in DimensionType enum. More...
 
template<>
int nvinfer1::EnumMax< LayerType > ()
 Maximum number of elements in LayerType enum. More...
 
template<>
int nvinfer1::EnumMax< TensorLocation > ()
 Maximum number of elements in TensorLocation enum. More...
 
template<>
int nvinfer1::EnumMax< ActivationType > ()
 Maximum number of elements in ActivationType enum. More...
 
template<>
int nvinfer1::EnumMax< PoolingType > ()
 Maximum number of elements in PoolingType enum. More...
 
template<>
int nvinfer1::EnumMax< ScaleMode > ()
 Maximum number of elements in ScaleMode enum. More...
 
template<>
int nvinfer1::EnumMax< ElementWiseOperation > ()
 Maximum number of elements in ElementWiseOperation enum. More...
 
template<>
int nvinfer1::EnumMax< RNNOperation > ()
 Maximum number of elements in RNNOperation enum. More...
 
template<>
int nvinfer1::EnumMax< RNNDirection > ()
 Maximum number of elements in RNNDirection enum. More...
 
template<>
int nvinfer1::EnumMax< RNNInputMode > ()
 Maximum number of elements in RNNInputMode enum. More...
 
template<>
int nvinfer1::EnumMax< RNNGateType > ()
 Maximum number of elements in RNNGateType enum. More...
 
template<>
int nvinfer1::EnumMax< PluginFormat > ()
 Maximum number of elements in PluginFormat enum. More...
 
template<>
int nvinfer1::EnumMax< UnaryOperation > ()
 Maximum number of elements in UnaryOperation enum. More...
 
template<>
int nvinfer1::EnumMax< ReduceOperation > ()
 Maximum number of elements in ReduceOperation enum. More...
 
template<>
int nvinfer1::EnumMax< TopKOperation > ()
 Maximum number of elements in TopKOperation enum. More...
 
template<>
int nvinfer1::EnumMax< CalibrationAlgoType > ()
 Maximum number of elements in CalibrationAlgoType enum. More...
 
template<>
int nvinfer1::EnumMax< ILogger::Severity > ()
 Maximum number of elements in ILogger::Severity enum. More...
 
TENSORRTAPI void * createInferBuilder_INTERNAL (void *logger, int version)
 Internal C entry point for creating IBuilder.
 
TENSORRTAPI void * createInferRuntime_INTERNAL (void *logger, int version)
 Internal C entry point for creating IRuntime.
 
TENSORRTAPI nvinfer1::ILoggergetLogger ()
 Return the logger object.
 
TENSORRTAPI int getInferLibVersion ()
 Return the library version number. More...
 

Detailed Description

This is the top-level API file for TensorRT.

Function Documentation

TENSORRTAPI int getInferLibVersion ( )

Return the library version number.

The format is as for TENSORRT_VERSION: (TENSORRT_MAJOR * 1000) + (TENSORRT_MINOR * 100) + TENSOR_PATCH.