TensorRT 10.0.0
nvinfer1::IBuilderConfig Class Reference

Holds properties for configuring a builder to produce an engine. More...

#include <NvInfer.h>

Inheritance diagram for nvinfer1::IBuilderConfig:
nvinfer1::INoCopy

Public Member Functions

virtual ~IBuilderConfig () noexcept=default
 
virtual void setAvgTimingIterations (int32_t avgTiming) noexcept
 Set the number of averaging iterations used when timing layers. More...
 
int32_t getAvgTimingIterations () const noexcept
 Query the number of averaging iterations. More...
 
void setEngineCapability (EngineCapability capability) noexcept
 Configure the builder to target specified EngineCapability flow. More...
 
EngineCapability getEngineCapability () const noexcept
 Query EngineCapability flow configured for the builder. More...
 
void setInt8Calibrator (IInt8Calibrator *calibrator) noexcept
 Set Int8 Calibration interface. More...
 
IInt8CalibratorgetInt8Calibrator () const noexcept
 Get Int8 Calibration interface. More...
 
void setFlags (BuilderFlags builderFlags) noexcept
 Set the build mode flags to turn on builder options for this network. More...
 
BuilderFlags getFlags () const noexcept
 Get the build mode flags for this builder config. Defaults to 0. More...
 
void clearFlag (BuilderFlag builderFlag) noexcept
 clear a single build mode flag. More...
 
void setFlag (BuilderFlag builderFlag) noexcept
 Set a single build mode flag. More...
 
bool getFlag (BuilderFlag builderFlag) const noexcept
 Returns true if the build mode flag is set. More...
 
void setDeviceType (ILayer const *layer, DeviceType deviceType) noexcept
 Set the device that this layer must execute on. More...
 
DeviceType getDeviceType (ILayer const *layer) const noexcept
 Get the device that this layer executes on. More...
 
bool isDeviceTypeSet (ILayer const *layer) const noexcept
 whether the DeviceType has been explicitly set for this layer More...
 
void resetDeviceType (ILayer const *layer) noexcept
 reset the DeviceType for this layer More...
 
bool canRunOnDLA (ILayer const *layer) const noexcept
 Checks if a layer can run on DLA. More...
 
void setDLACore (int32_t dlaCore) noexcept
 Sets the DLA core used by the network. Defaults to -1. More...
 
int32_t getDLACore () const noexcept
 Get the DLA core that the engine executes on. More...
 
void setDefaultDeviceType (DeviceType deviceType) noexcept
 Sets the default DeviceType to be used by the builder. It ensures that all the layers that can run on this device will run on it, unless setDeviceType is used to override the default DeviceType for a layer. More...
 
DeviceType getDefaultDeviceType () const noexcept
 Get the default DeviceType which was set by setDefaultDeviceType. More...
 
void reset () noexcept
 Resets the builder configuration to defaults. More...
 
void setProfileStream (const cudaStream_t stream) noexcept
 Set the cuda stream that is used to profile this network. More...
 
cudaStream_t getProfileStream () const noexcept
 Get the cuda stream that is used to profile this network. More...
 
int32_t addOptimizationProfile (IOptimizationProfile const *profile) noexcept
 Add an optimization profile. More...
 
int32_t getNbOptimizationProfiles () const noexcept
 Get number of optimization profiles. More...
 
void setProfilingVerbosity (ProfilingVerbosity verbosity) noexcept
 Set verbosity level of layer information exposed in NVTX annotations and IEngineInspector. More...
 
ProfilingVerbosity getProfilingVerbosity () const noexcept
 Get verbosity level of layer information exposed in NVTX annotations and IEngineInspector. More...
 
void setAlgorithmSelector (IAlgorithmSelector *selector) noexcept
 Set Algorithm Selector. More...
 
IAlgorithmSelectorgetAlgorithmSelector () const noexcept
 Get Algorithm Selector. More...
 
bool setCalibrationProfile (IOptimizationProfile const *profile) noexcept
 Add a calibration profile. More...
 
IOptimizationProfile const * getCalibrationProfile () noexcept
 Get the current calibration profile. More...
 
void setQuantizationFlags (QuantizationFlags flags) noexcept
 Set the quantization flags. More...
 
QuantizationFlags getQuantizationFlags () const noexcept
 Get the quantization flags. More...
 
void clearQuantizationFlag (QuantizationFlag flag) noexcept
 clear a quantization flag. More...
 
void setQuantizationFlag (QuantizationFlag flag) noexcept
 Set a single quantization flag. More...
 
bool getQuantizationFlag (QuantizationFlag flag) const noexcept
 Returns true if the quantization flag is set. More...
 
bool setTacticSources (TacticSources tacticSources) noexcept
 Set tactic sources. More...
 
TacticSources getTacticSources () const noexcept
 Get tactic sources. More...
 
nvinfer1::ITimingCachecreateTimingCache (void const *blob, std::size_t size) const noexcept
 Create timing cache. More...
 
bool setTimingCache (ITimingCache const &cache, bool ignoreMismatch) noexcept
 Attach a timing cache to IBuilderConfig. More...
 
nvinfer1::ITimingCache const * getTimingCache () const noexcept
 Get the pointer to the timing cache from current IBuilderConfig. More...
 
void setMemoryPoolLimit (MemoryPoolType pool, std::size_t poolSize) noexcept
 Set the memory size for the memory pool. More...
 
std::size_t getMemoryPoolLimit (MemoryPoolType pool) const noexcept
 Get the memory size limit of the memory pool. More...
 
void setPreviewFeature (PreviewFeature feature, bool enable) noexcept
 Enable or disable a specific preview feature. More...
 
bool getPreviewFeature (PreviewFeature feature) const noexcept
 Get status of preview feature. More...
 
void setBuilderOptimizationLevel (int32_t level) noexcept
 Set builder optimization level. More...
 
int32_t getBuilderOptimizationLevel () noexcept
 Get builder optimization level. More...
 
void setHardwareCompatibilityLevel (HardwareCompatibilityLevel hardwareCompatibilityLevel) noexcept
 Set the hardware compatibility level. More...
 
HardwareCompatibilityLevel getHardwareCompatibilityLevel () const noexcept
 Get the hardware compatibility level. More...
 
void setPluginsToSerialize (char const *const *paths, int32_t nbPaths) noexcept
 Set the plugin libraries to be serialized with version-compatible engines. More...
 
char const * getPluginToSerialize (int32_t index) const noexcept
 Get the plugin library path to be serialized with version-compatible engines. More...
 
int32_t getNbPluginsToSerialize () const noexcept
 Get the number of plugin library paths to be serialized with version-compatible engines. More...
 
void setMaxAuxStreams (int32_t nbStreams) noexcept
 Set the maximum number of auxiliary streams that TRT is allowed to use. More...
 
int32_t getMaxAuxStreams () const noexcept
 Get the maximum number of auxiliary streams that TRT is allowed to use. More...
 
void setProgressMonitor (IProgressMonitor *monitor) noexcept
 Sets the progress monitor for building a network. More...
 
IProgressMonitorgetProgressMonitor () const noexcept
 

Protected Attributes

apiv::VBuilderConfig * mImpl
 

Additional Inherited Members

- Protected Member Functions inherited from nvinfer1::INoCopy
 INoCopy ()=default
 
virtual ~INoCopy ()=default
 
 INoCopy (INoCopy const &other)=delete
 
INoCopyoperator= (INoCopy const &other)=delete
 
 INoCopy (INoCopy &&other)=delete
 
INoCopyoperator= (INoCopy &&other)=delete
 

Detailed Description

Holds properties for configuring a builder to produce an engine.

See also
BuilderFlags

Constructor & Destructor Documentation

◆ ~IBuilderConfig()

virtual nvinfer1::IBuilderConfig::~IBuilderConfig ( )
virtualdefaultnoexcept

Member Function Documentation

◆ addOptimizationProfile()

int32_t nvinfer1::IBuilderConfig::addOptimizationProfile ( IOptimizationProfile const *  profile)
inlinenoexcept

Add an optimization profile.

This function must be called at least once if the network has dynamic or shape input tensors. This function may be called at most once when building a refittable engine, as more than a single optimization profile are not supported for refittable engines.

Parameters
profileThe new optimization profile, which must satisfy profile->isValid() == true
Returns
The index of the optimization profile (starting from 0) if the input is valid, or -1 if the input is not valid.

◆ canRunOnDLA()

bool nvinfer1::IBuilderConfig::canRunOnDLA ( ILayer const *  layer) const
inlinenoexcept

Checks if a layer can run on DLA.

Returns
status true if the layer can on DLA else returns false.

◆ clearFlag()

void nvinfer1::IBuilderConfig::clearFlag ( BuilderFlag  builderFlag)
inlinenoexcept

clear a single build mode flag.

clears the builder mode flag from the enabled flags.

See also
setFlags()

◆ clearQuantizationFlag()

void nvinfer1::IBuilderConfig::clearQuantizationFlag ( QuantizationFlag  flag)
inlinenoexcept

clear a quantization flag.

Clears the quantization flag from the enabled quantization flags.

See also
setQuantizationFlags()

◆ createTimingCache()

nvinfer1::ITimingCache * nvinfer1::IBuilderConfig::createTimingCache ( void const *  blob,
std::size_t  size 
) const
inlinenoexcept

Create timing cache.

Create ITimingCache instance from serialized raw data. The created timing cache doesn’t belong to a specific IBuilderConfig. It can be shared by multiple builder instances. Call setTimingCache() before launching a builder to attach cache to builder instance.

Parameters
blobA pointer to the raw data that contains serialized timing cache
sizeThe size in bytes of the serialized timing cache. Size 0 means create a new cache from scratch
See also
setTimingCache
Returns
the pointer to ITimingCache created

◆ getAlgorithmSelector()

IAlgorithmSelector * nvinfer1::IBuilderConfig::getAlgorithmSelector ( ) const
inlinenoexcept

Get Algorithm Selector.

◆ getAvgTimingIterations()

int32_t nvinfer1::IBuilderConfig::getAvgTimingIterations ( ) const
inlinenoexcept

Query the number of averaging iterations.

By default the number of averaging iterations is 1.

See also
setAvgTimingIterations()

◆ getBuilderOptimizationLevel()

int32_t nvinfer1::IBuilderConfig::getBuilderOptimizationLevel ( )
inlinenoexcept

Get builder optimization level.

Returns
the current builder optimization level
See also
setBuilderOptimizationLevel

◆ getCalibrationProfile()

IOptimizationProfile const * nvinfer1::IBuilderConfig::getCalibrationProfile ( )
inlinenoexcept

Get the current calibration profile.

Returns
A pointer to the current calibration profile or nullptr if calibration profile is unset.

◆ getDefaultDeviceType()

DeviceType nvinfer1::IBuilderConfig::getDefaultDeviceType ( ) const
inlinenoexcept

Get the default DeviceType which was set by setDefaultDeviceType.

By default it returns DeviceType::kGPU.

◆ getDeviceType()

DeviceType nvinfer1::IBuilderConfig::getDeviceType ( ILayer const *  layer) const
inlinenoexcept

Get the device that this layer executes on.

Returns
Returns DeviceType of the layer.

◆ getDLACore()

int32_t nvinfer1::IBuilderConfig::getDLACore ( ) const
inlinenoexcept

Get the DLA core that the engine executes on.

Returns
assigned DLA core or -1 for DLA not present or unset.

◆ getEngineCapability()

EngineCapability nvinfer1::IBuilderConfig::getEngineCapability ( ) const
inlinenoexcept

Query EngineCapability flow configured for the builder.

By default it returns EngineCapability::kSTANDARD.

See also
setEngineCapability()

◆ getFlag()

bool nvinfer1::IBuilderConfig::getFlag ( BuilderFlag  builderFlag) const
inlinenoexcept

Returns true if the build mode flag is set.

See also
getFlags()
Returns
True if flag is set, false if unset.

◆ getFlags()

BuilderFlags nvinfer1::IBuilderConfig::getFlags ( ) const
inlinenoexcept

Get the build mode flags for this builder config. Defaults to 0.

Returns
The build options as a bitmask.
See also
setFlags()

◆ getHardwareCompatibilityLevel()

HardwareCompatibilityLevel nvinfer1::IBuilderConfig::getHardwareCompatibilityLevel ( ) const
inlinenoexcept

Get the hardware compatibility level.

Returns
hardwareCompatibilityLevel The level of hardware compatibility.
See also
setHardwareCompatiblityLevel()

◆ getInt8Calibrator()

IInt8Calibrator * nvinfer1::IBuilderConfig::getInt8Calibrator ( ) const
inlinenoexcept

Get Int8 Calibration interface.

◆ getMaxAuxStreams()

int32_t nvinfer1::IBuilderConfig::getMaxAuxStreams ( ) const
inlinenoexcept

Get the maximum number of auxiliary streams that TRT is allowed to use.

See also
setMaxAuxStreams()

◆ getMemoryPoolLimit()

std::size_t nvinfer1::IBuilderConfig::getMemoryPoolLimit ( MemoryPoolType  pool) const
inlinenoexcept

Get the memory size limit of the memory pool.

Retrieve the memory size limit of the corresponding pool in bytes. If setMemoryPoolLimit for the pool has not been called, this returns the default value used by TensorRT. This default value is not necessarily the maximum possible value for that configuration.

Parameters
poolThe memory pool to get the limit for.
Returns
The size of the memory limit, in bytes, for the corresponding pool.
See also
setMemoryPoolLimit

◆ getNbOptimizationProfiles()

int32_t nvinfer1::IBuilderConfig::getNbOptimizationProfiles ( ) const
inlinenoexcept

Get number of optimization profiles.

This is one higher than the index of the last optimization profile that has be defined (or zero, if none has been defined yet).

Returns
The number of the optimization profiles.

◆ getNbPluginsToSerialize()

int32_t nvinfer1::IBuilderConfig::getNbPluginsToSerialize ( ) const
inlinenoexcept

Get the number of plugin library paths to be serialized with version-compatible engines.

Returns
The number of paths.

◆ getPluginToSerialize()

char const * nvinfer1::IBuilderConfig::getPluginToSerialize ( int32_t  index) const
inlinenoexcept

Get the plugin library path to be serialized with version-compatible engines.

Parameters
indexIndex of the plugin library path in the list. Should be in the range [0, getNbPluginsToSerialize()).
Returns
The path to the plugin library.

◆ getPreviewFeature()

bool nvinfer1::IBuilderConfig::getPreviewFeature ( PreviewFeature  feature) const
inlinenoexcept

Get status of preview feature.

Parameters
featurethe feature to query
Returns
true if the feature is enabled, false otherwise
See also
PreviewFeature, setPreviewFeature

◆ getProfileStream()

cudaStream_t nvinfer1::IBuilderConfig::getProfileStream ( ) const
inlinenoexcept

Get the cuda stream that is used to profile this network.

Returns
The cuda stream set by setProfileStream, nullptr if setProfileStream has not been called.
See also
setProfileStream()

◆ getProfilingVerbosity()

ProfilingVerbosity nvinfer1::IBuilderConfig::getProfilingVerbosity ( ) const
inlinenoexcept

Get verbosity level of layer information exposed in NVTX annotations and IEngineInspector.

Get the current setting of verbosity level of layer information exposed in NVTX annotations and IEngineInspector. Default value is ProfilingVerbosity::kLAYER_NAMES_ONLY.

See also
ProfilingVerbosity, setProfilingVerbosity(), IEngineInspector

◆ getProgressMonitor()

IProgressMonitor * nvinfer1::IBuilderConfig::getProgressMonitor ( ) const
inlinenoexcept
Returns
The progress monitor set by the application or nullptr.
See also
IBuilderConfig::setProgressMonitor

◆ getQuantizationFlag()

bool nvinfer1::IBuilderConfig::getQuantizationFlag ( QuantizationFlag  flag) const
inlinenoexcept

Returns true if the quantization flag is set.

See also
getQuantizationFlags()
Returns
True if quantization flag is set, false if unset.

◆ getQuantizationFlags()

QuantizationFlags nvinfer1::IBuilderConfig::getQuantizationFlags ( ) const
inlinenoexcept

Get the quantization flags.

Returns
The quantization flags as a bitmask.
See also
setQuantizationFlag()

◆ getTacticSources()

TacticSources nvinfer1::IBuilderConfig::getTacticSources ( ) const
inlinenoexcept

Get tactic sources.

Get the tactic sources currently set in the engine build configuration.

See also
setTacticSources()
Returns
tactic sources

◆ getTimingCache()

nvinfer1::ITimingCache const * nvinfer1::IBuilderConfig::getTimingCache ( ) const
inlinenoexcept

Get the pointer to the timing cache from current IBuilderConfig.

Returns
pointer to the timing cache used in current IBuilderConfig

◆ isDeviceTypeSet()

bool nvinfer1::IBuilderConfig::isDeviceTypeSet ( ILayer const *  layer) const
inlinenoexcept

whether the DeviceType has been explicitly set for this layer

Returns
true if device type is not default
See also
setDeviceType() getDeviceType() resetDeviceType()

◆ reset()

void nvinfer1::IBuilderConfig::reset ( )
inlinenoexcept

Resets the builder configuration to defaults.

Useful for initializing a builder config object to its original state.

◆ resetDeviceType()

void nvinfer1::IBuilderConfig::resetDeviceType ( ILayer const *  layer)
inlinenoexcept

reset the DeviceType for this layer

See also
setDeviceType() getDeviceType() isDeviceTypeSet()

◆ setAlgorithmSelector()

void nvinfer1::IBuilderConfig::setAlgorithmSelector ( IAlgorithmSelector selector)
inlinenoexcept

Set Algorithm Selector.

Parameters
selectorThe algorithm selector to be set in the build config.

◆ setAvgTimingIterations()

virtual void nvinfer1::IBuilderConfig::setAvgTimingIterations ( int32_t  avgTiming)
inlinevirtualnoexcept

Set the number of averaging iterations used when timing layers.

When timing layers, the builder minimizes over a set of average times for layer execution. This parameter controls the number of iterations used in averaging.

See also
getAvgTimingIterations()

◆ setBuilderOptimizationLevel()

void nvinfer1::IBuilderConfig::setBuilderOptimizationLevel ( int32_t  level)
inlinenoexcept

Set builder optimization level.

Set the builder optimization level. Setting a higher optimization level allows the optimizer to spend more time searching for optimization opportunities. The resulting engine may have better performance compared to an engine built with a lower optimization level.

The default optimization level is 3. Valid values include integers from 0 to the maximum optimization level, which is currently 5. Setting it to greater than the maximum level results in behavior identical to the maximum level.

Below are the descriptions about each builder optimization level:

  • Level 0: This enables the fastest compilation by disabling dynamic kernel generation and selecting the first tactic that succeeds in execution. This will also not respect a timing cache.
  • Level 1: Available tactics are sorted by heuristics, but only the top are tested to select the best. If a dynamic kernel is generated its compile optimization is low.
  • Level 2: Available tactics are sorted by heuristics, but only the fastest tactics are tested to select the best.
  • Level 3: Apply heuristics to see if a static precompiled kernel is applicable or if a new one has to be compiled dynamically.
  • Level 4: Always compiles a dynamic kernel.
  • Level 5: Always compiles a dynamic kernel and compares it to static kernels.
Parameters
levelThe optimization level to set to. Must be non-negative.
See also
getBuilderOptimizationLevel

◆ setCalibrationProfile()

bool nvinfer1::IBuilderConfig::setCalibrationProfile ( IOptimizationProfile const *  profile)
inlinenoexcept

Add a calibration profile.

Calibration optimization profile must be set if int8 calibration is used to set scales for a network with runtime dimensions.

Parameters
profileThe new calibration profile, which must satisfy profile->isValid() == true or be nullptr. MIN and MAX values will be overwritten by kOPT.
Returns
True if the calibration profile was set correctly.

◆ setDefaultDeviceType()

void nvinfer1::IBuilderConfig::setDefaultDeviceType ( DeviceType  deviceType)
inlinenoexcept

Sets the default DeviceType to be used by the builder. It ensures that all the layers that can run on this device will run on it, unless setDeviceType is used to override the default DeviceType for a layer.

See also
getDefaultDeviceType()

◆ setDeviceType()

void nvinfer1::IBuilderConfig::setDeviceType ( ILayer const *  layer,
DeviceType  deviceType 
)
inlinenoexcept

Set the device that this layer must execute on.

Parameters
layerwhich layer to execute.
deviceTypethat this layer must execute on. If DeviceType is not set or is reset, TensorRT will use the default DeviceType set in the builder.
Note
The device type for a layer must be compatible with the safety flow (if specified). For example a layer cannot be marked for DLA execution while the builder is configured for kSAFETY.
See also
getDeviceType()

◆ setDLACore()

void nvinfer1::IBuilderConfig::setDLACore ( int32_t  dlaCore)
inlinenoexcept

Sets the DLA core used by the network. Defaults to -1.

Parameters
dlaCoreThe DLA core to execute the engine on, in the range [0,getNbDlaCores()).

This function is used to specify which DLA core to use via indexing, if multiple DLA cores are available.

Warning
if getNbDLACores() returns 0, then this function does nothing.
See also
IRuntime::setDLACore() getDLACore()

◆ setEngineCapability()

void nvinfer1::IBuilderConfig::setEngineCapability ( EngineCapability  capability)
inlinenoexcept

Configure the builder to target specified EngineCapability flow.

The flow means a sequence of API calls that allow an application to set up a runtime, engine, and execution context in order to run inference.

The supported flows are specified in the EngineCapability enum.

◆ setFlag()

void nvinfer1::IBuilderConfig::setFlag ( BuilderFlag  builderFlag)
inlinenoexcept

Set a single build mode flag.

Add the input builder mode flag to the already enabled flags.

See also
setFlags()

◆ setFlags()

void nvinfer1::IBuilderConfig::setFlags ( BuilderFlags  builderFlags)
inlinenoexcept

Set the build mode flags to turn on builder options for this network.

The flags are listed in the BuilderFlags enum. The flags set configuration options to build the network.

Parameters
builderFlagsThe build option for an engine.
Note
This function will override the previous set flags, rather than bitwise ORing the new flag.
See also
getFlags()

◆ setHardwareCompatibilityLevel()

void nvinfer1::IBuilderConfig::setHardwareCompatibilityLevel ( HardwareCompatibilityLevel  hardwareCompatibilityLevel)
inlinenoexcept

Set the hardware compatibility level.

Hardware compatibility allows an engine to run on GPU architectures other than that of the GPU where the engine was built.

The default hardware compatibility level is HardwareCompatibilityLevel::kNONE.

Parameters
hardwareCompatibilityLevelThe level of hardware compatibility.

◆ setInt8Calibrator()

void nvinfer1::IBuilderConfig::setInt8Calibrator ( IInt8Calibrator calibrator)
inlinenoexcept

Set Int8 Calibration interface.

The calibrator is to minimize the information loss during the INT8 quantization process.

◆ setMaxAuxStreams()

void nvinfer1::IBuilderConfig::setMaxAuxStreams ( int32_t  nbStreams)
inlinenoexcept

Set the maximum number of auxiliary streams that TRT is allowed to use.

If the network contains operators that can run in parallel, TRT can execute them using auxiliary streams in addition to the one provided to the IExecutionContext::enqueueV3() call.

The default maximum number of auxiliary streams is determined by the heuristics in TensorRT on whether enabling multi-stream would improve the performance. This behavior can be overridden by calling this API to set the maximum number of auxiliary streams explicitly. Set this to 0 to enforce single-stream inference.

The resulting engine may use fewer auxiliary streams than the maximum if the network does not contain enough parallelism or if TensorRT determines that using more auxiliary streams does not help improve the performance.

Note
Allowing more auxiliary streams does not always give better performance since there will be synchronizations overhead between streams. Using CUDA graphs at runtime can help reduce the overhead caused by cross-stream synchronizations.
Using more auxiliary leads to more memory usage at runtime since some activation memory blocks will not be able to be reused.
Parameters
nbStreamsThe maximum number of auxiliary streams that TRT is allowed to use.
See also
getMaxAuxStreams(), ICudaEngine::getNbAuxStreams(), IExecutionContext::setAuxStreams()

◆ setMemoryPoolLimit()

void nvinfer1::IBuilderConfig::setMemoryPoolLimit ( MemoryPoolType  pool,
std::size_t  poolSize 
)
inlinenoexcept

Set the memory size for the memory pool.

TensorRT layers access different memory pools depending on the operation. This function sets in the IBuilderConfig the size limit, specified by poolSize, for the corresponding memory pool, specified by pool. TensorRT will build a plan file that is constrained by these limits or report which constraint caused the failure.

If the size of the pool, specified by poolSize, fails to meet the size requirements for the pool, this function does nothing and emits the recoverable error, ErrorCode::kINVALID_ARGUMENT, to the registered IErrorRecorder.

If the size of the pool is larger than the maximum possible value for the configuration, this function does nothing and emits ErrorCode::kUNSUPPORTED_STATE.

If the pool does not exist on the requested device type when building the network, a warning is emitted to the logger, and the memory pool value is ignored.

Refer to MemoryPoolType to see the size requirements for each pool.

Parameters
poolThe memory pool to limit the available memory for.
poolSizeThe size of the pool in bytes.
See also
getMemoryPoolLimit, MemoryPoolType

◆ setPluginsToSerialize()

void nvinfer1::IBuilderConfig::setPluginsToSerialize ( char const *const *  paths,
int32_t  nbPaths 
)
inlinenoexcept

Set the plugin libraries to be serialized with version-compatible engines.

Each entry in the list of libraries must be unique.

Parameters
pathsThe paths of plugin libraries.
nbPathsThe number of paths.

◆ setPreviewFeature()

void nvinfer1::IBuilderConfig::setPreviewFeature ( PreviewFeature  feature,
bool  enable 
)
inlinenoexcept

Enable or disable a specific preview feature.

Allows enabling or disabling experimental features, which are not enabled by default in the current release.

Refer to PreviewFeature for additional information, and a list of the available features.

Parameters
featurethe feature to enable / disable
enabletrue for enable, false for disable
See also
PreviewFeature, getPreviewFeature

◆ setProfileStream()

void nvinfer1::IBuilderConfig::setProfileStream ( const cudaStream_t  stream)
inlinenoexcept

Set the cuda stream that is used to profile this network.

Parameters
streamThe cuda stream used for profiling by the builder.
See also
getProfileStream()

◆ setProfilingVerbosity()

void nvinfer1::IBuilderConfig::setProfilingVerbosity ( ProfilingVerbosity  verbosity)
inlinenoexcept

Set verbosity level of layer information exposed in NVTX annotations and IEngineInspector.

Control how much layer information will be exposed in NVTX annotations and IEngineInspector.

See also
ProfilingVerbosity, getProfilingVerbosity(), IEngineInspector

◆ setProgressMonitor()

void nvinfer1::IBuilderConfig::setProgressMonitor ( IProgressMonitor monitor)
inlinenoexcept

Sets the progress monitor for building a network.

Parameters
monitorThe progress monitor to assign to the IBuilderConfig.

The progress monitor signals to the application when different phases of the compiler are being executed. Setting to nullptr unsets the monitor so that the application is not signaled.

See also
IBuilderConfig::getProgressMonitor

◆ setQuantizationFlag()

void nvinfer1::IBuilderConfig::setQuantizationFlag ( QuantizationFlag  flag)
inlinenoexcept

Set a single quantization flag.

Add the input quantization flag to the already enabled quantization flags.

See also
setQuantizationFlags()

◆ setQuantizationFlags()

void nvinfer1::IBuilderConfig::setQuantizationFlags ( QuantizationFlags  flags)
inlinenoexcept

Set the quantization flags.

The flags are listed in the QuantizationFlag enum. The flags set configuration options to quantize the network in int8.

Parameters
flagsThe quantization flags.
Note
This function will override the previous set flags, rather than bitwise ORing the new flag.
See also
getQuantizationFlags()

◆ setTacticSources()

bool nvinfer1::IBuilderConfig::setTacticSources ( TacticSources  tacticSources)
inlinenoexcept

Set tactic sources.

This bitset controls which tactic sources TensorRT is allowed to use for tactic selection.

Multiple tactic sources may be combined with a bitwise OR operation. For example, to enable cublas and cublasLt as tactic sources, use a value of:

1U << static_cast<uint32_t>(TacticSource::kCUBLAS) | 1U << static_cast<uint32_t>(TacticSource::kCUBLAS_LT)

See also
getTacticSources
Returns
true if the tactic sources in the build configuration were updated. The tactic sources in the build configuration will not be updated if the provided value is invalid.

◆ setTimingCache()

bool nvinfer1::IBuilderConfig::setTimingCache ( ITimingCache const &  cache,
bool  ignoreMismatch 
)
inlinenoexcept

Attach a timing cache to IBuilderConfig.

The timing cache has verification header to make sure the provided cache can be used in current environment. A failure will be reported if the CUDA device property in the provided cache is different from current environment. ignoreMismatch = true skips strict verification and allows loading cache created from a different device.

The cache must not be destroyed until after the engine is built.

Parameters
cachethe timing cache to be used
ignoreMismatchwhether or not allow using a cache that contains different CUDA device property
Returns
true if set successfully, false otherwise
Warning
Using cache generated from devices with different CUDA device properties may lead to functional/performance bugs.

Member Data Documentation

◆ mImpl

apiv::VBuilderConfig* nvinfer1::IBuilderConfig::mImpl
protected

The documentation for this class was generated from the following file: