TensorRT
6.0.1.5
|
Holds properties for configuring a builder to produce an engine. More...
#include <NvInfer.h>
Public Member Functions | |
virtual void | setMinTimingIterations (int minTiming)=0 |
Set the number of minimization iterations used when timing layers. More... | |
virtual int | getMinTimingIterations () const =0 |
Query the number of minimization iterations. More... | |
virtual void | setAvgTimingIterations (int avgTiming)=0 |
Set the number of averaging iterations used when timing layers. More... | |
virtual int | getAvgTimingIterations () const =0 |
Query the number of averaging iterations. More... | |
virtual void | setEngineCapability (EngineCapability capability)=0 |
Configure the builder to target specified EngineCapability flow. More... | |
virtual EngineCapability | getEngineCapability () const =0 |
Query EngineCapability flow configured for the builder. More... | |
virtual void | setInt8Calibrator (IInt8Calibrator *calibrator)=0 |
Set Int8 Calibration interface. More... | |
virtual IInt8Calibrator * | getInt8Calibrator () const =0 |
Get Int8 Calibration interface. | |
virtual void | setMaxWorkspaceSize (std::size_t workspaceSize)=0 |
Set the maximum workspace size. More... | |
virtual std::size_t | getMaxWorkspaceSize () const =0 |
Get the maximum workspace size. More... | |
virtual void | setFlags (BuilderFlags builderFlags)=0 |
Set the build mode flags to turn on builder options for this network. More... | |
virtual BuilderFlags | getFlags () const =0 |
Get the set of build mode flags for this builder config. Defaults to BuildMode::kDEFAULT. More... | |
virtual void | clearFlag (BuilderFlag builderFlag)=0 |
clear a single build mode flag. More... | |
virtual void | setFlag (BuilderFlag builderFlag)=0 |
Set a single build mode flag. More... | |
virtual bool | getFlag (BuilderFlag builderFlag) const =0 |
returns true if the build mode flag is set More... | |
virtual void | setDeviceType (const ILayer *layer, DeviceType deviceType)=0 |
Set the device that this layer must execute on. More... | |
virtual DeviceType | getDeviceType (const ILayer *layer) const =0 |
Get the device that this layer executes on. More... | |
virtual bool | isDeviceTypeSet (const ILayer *layer) const =0 |
whether the DeviceType has been explicitly set for this layer More... | |
virtual void | resetDeviceType (const ILayer *layer)=0 |
reset the DeviceType for this layer More... | |
virtual bool | canRunOnDLA (const ILayer *layer) const =0 |
Checks if a layer can run on DLA. More... | |
virtual void | setDLACore (int dlaCore)=0 |
Sets the DLA core used by the network. More... | |
virtual int | getDLACore () const =0 |
Get the DLA core that the engine executes on. More... | |
virtual void | setDefaultDeviceType (DeviceType deviceType)=0 |
Sets the default DeviceType to be used by the builder. It ensures that all the layers that can run on this device will run on it, unless setDeviceType is used to override the default DeviceType for a layer. More... | |
virtual DeviceType | getDefaultDeviceType () const =0 |
Get the default DeviceType which was set by setDefaultDeviceType. More... | |
virtual void | reset ()=0 |
Resets the builder configuration to defaults. More... | |
virtual void | destroy ()=0 |
De-allocates any internally allocated memory. More... | |
virtual void | setProfileStream (const cudaStream_t stream)=0 |
Set the cudaStream that is used to profile this network. More... | |
virtual cudaStream_t | getProfileStream () const =0 |
Get the cudaStream that is used to profile this network. More... | |
virtual int | addOptimizationProfile (const IOptimizationProfile *profile) noexcept=0 |
Add an optimization profile. More... | |
virtual int | getNbOptimizationProfiles () const noexcept=0 |
Get number of optimization profiles. More... | |
Holds properties for configuring a builder to produce an engine.
|
pure virtualnoexcept |
Add an optimization profile.
This function must be called at least once if the network has dynamic or shape input tensors. This function may be called at most once when building a refittable engine, as more than a single optimization profile are not supported for refittable engines.
profile | The new optimization profile, which must satisfy profile->isValid() == true |
|
pure virtual |
Checks if a layer can run on DLA.
|
pure virtual |
clear a single build mode flag.
clears the builder flag from the set of enabled flags.
|
pure virtual |
De-allocates any internally allocated memory.
When destroying a builder config object, we can call this function.
|
pure virtual |
Query the number of averaging iterations.
By default the number of averaging iterations is 1.
|
pure virtual |
Get the default DeviceType which was set by setDefaultDeviceType.
By default it returns DeviceType::kGPU.
|
pure virtual |
Get the device that this layer executes on.
|
pure virtual |
Get the DLA core that the engine executes on.
|
pure virtual |
Query EngineCapability flow configured for the builder.
By default it returns EngineCapability::kDEFAULT.
|
pure virtual |
returns true if the build mode flag is set
Check if a build mode flag is set.
|
pure virtual |
Get the set of build mode flags for this builder config. Defaults to BuildMode::kDEFAULT.
|
pure virtual |
Get the maximum workspace size.
By default the workspace size is 0, which means there is no temporary memory.
|
pure virtual |
Query the number of minimization iterations.
By default the minimum number of iterations is 2.
|
pure virtualnoexcept |
Get number of optimization profiles.
This is one higher than the index of the last optimization profile that has be defined (or zero, if none has been defined yet).
|
pure virtual |
Get the cudaStream that is used to profile this network.
|
pure virtual |
whether the DeviceType has been explicitly set for this layer
|
pure virtual |
Resets the builder configuration to defaults.
When initializing a builder config object, we can call this function.
|
pure virtual |
reset the DeviceType for this layer
|
pure virtual |
Set the number of averaging iterations used when timing layers.
When timing layers, the builder minimizes over a set of average times for layer execution. This parameter controls the number of iterations used in averaging.
|
pure virtual |
Sets the default DeviceType to be used by the builder. It ensures that all the layers that can run on this device will run on it, unless setDeviceType is used to override the default DeviceType for a layer.
|
pure virtual |
Set the device that this layer must execute on.
DeviceType | that this layer must execute on. If DeviceType is not set or is reset, TensorRT will use the default DeviceType set in the builder. |
|
pure virtual |
Sets the DLA core used by the network.
dlaCore | The DLA core to execute the engine on (0 to N-1). Default value is 0. |
It can be used to specify which DLA core to use via indexing, if multiple DLA cores are available.
|
pure virtual |
Configure the builder to target specified EngineCapability flow.
The flow means a sequence of API calls that allow an application to set up a runtime, engine, and execution context in order to run inference.
The supported flows are specified in the EngineCapability enum.
|
pure virtual |
Set a single build mode flag.
Sets the build mode flags on top of the flags already specified.
|
pure virtual |
Set the build mode flags to turn on builder options for this network.
The flags are listed in the BuilderFlags enum. The flags set configuration options to build the network.
builderFlags | The build option for an engine. |
|
pure virtual |
Set Int8 Calibration interface.
The calibrator is to minimize the information loss during the INT8 quantization process.
|
pure virtual |
Set the maximum workspace size.
workspaceSize | The maximum GPU temporary memory which the engine can use at execution time. |
|
pure virtual |
Set the number of minimization iterations used when timing layers.
When timing layers, the builder minimizes over a set of average times for layer execution. This parameter controls the number of iterations used in minimization. The builder may sometimes run layers for more iterations to improve timing accuracy if this parameter is set to a small value and the runtime of the layer is short.
|
pure virtual |
Set the cudaStream that is used to profile this network.
stream | The cuda stream used for profiling by the builder. |