TensorRT
8.0.0
|
Builds an engine from a network definition. More...
#include <NvInfer.h>
Public Member Functions | |
void | setMaxBatchSize (int32_t batchSize) noexcept |
Set the maximum batch size. More... | |
int32_t | getMaxBatchSize () const noexcept |
Get the maximum batch size. More... | |
bool | platformHasFastFp16 () const noexcept |
Determine whether the platform has fast native fp16. | |
bool | platformHasFastInt8 () const noexcept |
Determine whether the platform has fast native int8. | |
TRT_DEPRECATED void | destroy () noexcept |
Destroy this object. More... | |
int32_t | getMaxDLABatchSize () const noexcept |
Get the maximum batch size DLA can support. For any tensor the total volume of index dimensions combined(dimensions other than CHW) with the requested batch size should not exceed the value returned by this function. More... | |
int32_t | getNbDLACores () const noexcept |
Return the number of DLA engines available to this builder. | |
void | setGpuAllocator (IGpuAllocator *allocator) noexcept |
Set the GPU allocator. More... | |
nvinfer1::IBuilderConfig * | createBuilderConfig () noexcept |
Create a builder configuration object. More... | |
TRT_DEPRECATED nvinfer1::ICudaEngine * | buildEngineWithConfig (INetworkDefinition &network, IBuilderConfig &config) noexcept |
Builds an engine for the given INetworkDefinition and given IBuilderConfig. More... | |
nvinfer1::INetworkDefinition * | createNetworkV2 (NetworkDefinitionCreationFlags flags) noexcept |
Create a network definition object. More... | |
nvinfer1::IOptimizationProfile * | createOptimizationProfile () noexcept |
Create a new optimization profile. More... | |
void | setErrorRecorder (IErrorRecorder *recorder) noexcept |
Set the ErrorRecorder for this interface. More... | |
IErrorRecorder * | getErrorRecorder () const noexcept |
get the ErrorRecorder assigned to this interface. More... | |
void | reset () noexcept |
Resets the builder state to default values. | |
bool | platformHasTf32 () const noexcept |
Determine whether the platform has TF32 support. | |
nvinfer1::IHostMemory * | buildSerializedNetwork (INetworkDefinition &network, IBuilderConfig &config) noexcept |
Builds and serializes a network for the given INetworkDefinition and IBuilderConfig. More... | |
Protected Attributes | |
apiv::VBuilder * | mImpl |
Additional Inherited Members | |
Protected Member Functions inherited from nvinfer1::INoCopy | |
INoCopy (const INoCopy &other)=delete | |
INoCopy & | operator= (const INoCopy &other)=delete |
INoCopy (INoCopy &&other)=delete | |
INoCopy & | operator= (INoCopy &&other)=delete |
Builds an engine from a network definition.
|
inlinenoexcept |
Builds an engine for the given INetworkDefinition and given IBuilderConfig.
It enables the builder to build multiple engines based on the same network definition, but with different builder configurations.
config.getProfileStream()
before returning.
|
inlinenoexcept |
Builds and serializes a network for the given INetworkDefinition and IBuilderConfig.
This function allows building and serialization of a network without creating an engine.
network | Network definition. |
config | Builder configuration. |
config.getProfileStream()
before returning.
|
inlinenoexcept |
Create a builder configuration object.
|
inlinenoexcept |
Create a network definition object.
Creates a network definition object with immutable properties specified using the flags parameter. Providing the kDEFAULT flag as parameter mimics the behaviour of createNetwork(). CreateNetworkV2 supports dynamic shapes and explicit batch dimensions when used with NetworkDefinitionCreationFlag::kEXPLICIT_BATCH flag.
flags | Bitset of NetworkDefinitionCreationFlags specifying network properties combined with bitwise OR. e.g., 1U << NetworkDefinitionCreationFlag::kEXPLICIT_BATCH |
|
inlinenoexcept |
Create a new optimization profile.
If the network has any dynamic input tensors, the appropriate calls to setDimensions() must be made. Likewise, if there are any shape input tensors, the appropriate calls to setShapeValues() are required. The builder retains ownership of the created optimization profile and returns a raw pointer, i.e. the users must not attempt to delete the returned pointer.
|
inlinenoexcept |
Destroy this object.
|
inlinenoexcept |
get the ErrorRecorder assigned to this interface.
Retrieves the assigned error recorder object for the given class. A nullptr will be returned if setErrorRecorder has not been called.
|
inlinenoexcept |
Get the maximum batch size.
|
inlinenoexcept |
Get the maximum batch size DLA can support. For any tensor the total volume of index dimensions combined(dimensions other than CHW) with the requested batch size should not exceed the value returned by this function.
|
inlinenoexcept |
Set the ErrorRecorder for this interface.
Assigns the ErrorRecorder to this interface. The ErrorRecorder will track all errors during execution. This function will call incRefCount of the registered ErrorRecorder at least once. Setting recorder to nullptr unregisters the recorder with the interface, resulting in a call to decRefCount if a recorder has been registered.
If an error recorder is not set, messages will be sent to the global log stream.
recorder | The error recorder to register with this interface. |
|
inlinenoexcept |
Set the GPU allocator.
allocator | Set the GPU allocator to be used by the builder. All GPU memory acquired will use this allocator. If NULL is passed, the default allocator will be used. |
Default: uses cudaMalloc/cudaFree.
|
inlinenoexcept |
Set the maximum batch size.
batchSize | The maximum batch size which can be used at execution time, and also the batch size for which the engine will be optimized. |