Builds an engine from a network definition. More...

#include <NvInfer.h>

Inheritance diagram for nvinfer1::IBuilder:

Public Member Functions
virtual	~IBuilder () noexcept=default

TRT_DEPRECATED void	setMaxBatchSize (int32_t batchSize) noexcept
	Set the maximum batch size. This has no effect for networks created with explicit batch dimension mode. More...

TRT_DEPRECATED int32_t	getMaxBatchSize () const noexcept
	Get the maximum batch size. More...

bool	platformHasFastFp16 () const noexcept
	Determine whether the platform has fast native fp16. More...

bool	platformHasFastInt8 () const noexcept
	Determine whether the platform has fast native int8. More...

TRT_DEPRECATED void	destroy () noexcept
	Destroy this object. More...

int32_t	getMaxDLABatchSize () const noexcept
	Get the maximum batch size DLA can support. For any tensor the total volume of index dimensions combined(dimensions other than CHW) with the requested batch size should not exceed the value returned by this function. More...

int32_t	getNbDLACores () const noexcept
	Return the number of DLA engines available to this builder. More...

void	setGpuAllocator (IGpuAllocator *allocator) noexcept
	Set the GPU allocator. More...

nvinfer1::IBuilderConfig *	createBuilderConfig () noexcept
	Create a builder configuration object. More...

TRT_DEPRECATED nvinfer1::ICudaEngine *	buildEngineWithConfig (INetworkDefinition &network, IBuilderConfig &config) noexcept
	Builds an engine for the given INetworkDefinition and given IBuilderConfig. More...

nvinfer1::INetworkDefinition *	createNetworkV2 (NetworkDefinitionCreationFlags flags) noexcept
	Create a network definition object. More...

nvinfer1::IOptimizationProfile *	createOptimizationProfile () noexcept
	Create a new optimization profile. More...

void	setErrorRecorder (IErrorRecorder *recorder) noexcept
	Set the ErrorRecorder for this interface. More...

IErrorRecorder *	getErrorRecorder () const noexcept
	get the ErrorRecorder assigned to this interface. More...

void	reset () noexcept
	Resets the builder state to default values. More...

bool	platformHasTf32 () const noexcept
	Determine whether the platform has TF32 support. More...

nvinfer1::IHostMemory *	buildSerializedNetwork (INetworkDefinition &network, IBuilderConfig &config) noexcept
	Builds and serializes a network for the given INetworkDefinition and IBuilderConfig. More...

bool	isNetworkSupported (INetworkDefinition const &network, IBuilderConfig const &config) const noexcept
	Checks that a network is within the scope of the IBuilderConfig settings. More...

ILogger *	getLogger () const noexcept
	get the logger with which the builder was created More...

bool	setMaxThreads (int32_t maxThreads) noexcept
	Set the maximum number of threads. More...

int32_t	getMaxThreads () const noexcept
	get the maximum number of threads that can be used by the builder. More...

IPluginRegistry &	getPluginRegistry () noexcept
	get the local plugin registry that can be used by the builder. More...

Protected Attributes
apiv::VBuilder *	mImpl

Additional Inherited Members
Protected Member Functions inherited from nvinfer1::INoCopy
	INoCopy ()=default

virtual	~INoCopy ()=default

	INoCopy (INoCopy const &other)=delete

INoCopy &	operator= (INoCopy const &other)=delete

	INoCopy (INoCopy &&other)=delete

INoCopy &	operator= (INoCopy &&other)=delete

Detailed Description

Builds an engine from a network definition.

Warning: Do not inherit from this class, as doing so will break forward-compatibility of the API and ABI.

Constructor & Destructor Documentation

◆ ~IBuilder()

virtual nvinfer1::IBuilder::~IBuilder ( )

virtualdefaultnoexcept

Member Function Documentation

◆ buildEngineWithConfig()

TRT_DEPRECATED nvinfer1::ICudaEngine * nvinfer1::IBuilder::buildEngineWithConfig	(	INetworkDefinition &	network,
		IBuilderConfig &	config
	)

inlinenoexcept

Builds an engine for the given INetworkDefinition and given IBuilderConfig.

It enables the builder to build multiple engines based on the same network definition, but with different builder configurations.

Note: This function will synchronize the cuda stream returned by config.getProfileStream() before returning.

Deprecated:: Deprecated in TensorRT 8.0. Superseded by IBuilder::buildSerializedNetwork().

◆ buildSerializedNetwork()

nvinfer1::IHostMemory * nvinfer1::IBuilder::buildSerializedNetwork	(	INetworkDefinition &	network,
		IBuilderConfig &	config
	)

inlinenoexcept

Builds and serializes a network for the given INetworkDefinition and IBuilderConfig.

This function allows building and serialization of a network without creating an engine.

Parameters

network	Network definition.
config	Builder configuration.

Returns: A pointer to a IHostMemory object that contains a serialized network.

Note: This function will synchronize the cuda stream returned by config.getProfileStream() before returning.

See also: INetworkDefinition, IBuilderConfig, IHostMemory

◆ createBuilderConfig()

nvinfer1::IBuilderConfig * nvinfer1::IBuilder::createBuilderConfig ( )

inlinenoexcept

Create a builder configuration object.

See also: IBuilderConfig

◆ createNetworkV2()

nvinfer1::INetworkDefinition * nvinfer1::IBuilder::createNetworkV2 ( NetworkDefinitionCreationFlags flags )

inlinenoexcept

Create a network definition object.

Creates a network definition object with immutable properties specified using the flags parameter. CreateNetworkV2 supports dynamic shapes and explicit batch dimensions when used with NetworkDefinitionCreationFlag::kEXPLICIT_BATCH flag. Creating a network without NetworkDefinitionCreationFlag::kEXPLICIT_BATCH flag has been deprecated.

Parameters

flags Bitset of NetworkDefinitionCreationFlags specifying network properties combined with bitwise OR. e.g., 1U << NetworkDefinitionCreationFlag::kEXPLICIT_BATCH

See also: INetworkDefinition, NetworkDefinitionCreationFlags

◆ createOptimizationProfile()

nvinfer1::IOptimizationProfile * nvinfer1::IBuilder::createOptimizationProfile ( )

inlinenoexcept

Create a new optimization profile.

If the network has any dynamic input tensors, the appropriate calls to setDimensions() must be made. Likewise, if there are any shape input tensors, the appropriate calls to setShapeValues() are required. The builder retains ownership of the created optimization profile and returns a raw pointer, i.e. the users must not attempt to delete the returned pointer.

See also: IOptimizationProfile

◆ destroy()

TRT_DEPRECATED void nvinfer1::IBuilder::destroy ( )

inlinenoexcept

Destroy this object.

Deprecated:: Deprecated in TensorRT 8.0. Superseded by delete.

Warning: Calling destroy on a managed pointer will result in a double-free error.

◆ getErrorRecorder()

IErrorRecorder * nvinfer1::IBuilder::getErrorRecorder ( ) const

inlinenoexcept

get the ErrorRecorder assigned to this interface.

Retrieves the assigned error recorder object for the given class. A nullptr will be returned if setErrorRecorder has not been called.

Returns: A pointer to the IErrorRecorder object that has been registered.

See also: setErrorRecorder()

◆ getLogger()

ILogger * nvinfer1::IBuilder::getLogger ( ) const

inlinenoexcept

get the logger with which the builder was created

Returns: the logger

◆ getMaxBatchSize()

TRT_DEPRECATED int32_t nvinfer1::IBuilder::getMaxBatchSize ( ) const

inlinenoexcept

Get the maximum batch size.

Returns: The maximum batch size.

Deprecated:: Deprecated in TensorRT 8.4.

See also: setMaxBatchSize(); getMaxDLABatchSize()

◆ getMaxDLABatchSize()

int32_t nvinfer1::IBuilder::getMaxDLABatchSize ( ) const

inlinenoexcept

Get the maximum batch size DLA can support. For any tensor the total volume of index dimensions combined(dimensions other than CHW) with the requested batch size should not exceed the value returned by this function.

Warning: getMaxDLABatchSize does not work with dynamic shapes.

◆ getMaxThreads()

int32_t nvinfer1::IBuilder::getMaxThreads ( ) const

inlinenoexcept

get the maximum number of threads that can be used by the builder.

Retrieves the maximum number of threads that can be used by the builder.

Returns: The maximum number of threads that can be used by the builder.

See also: setMaxThreads()

◆ getNbDLACores()

int32_t nvinfer1::IBuilder::getNbDLACores ( ) const

inlinenoexcept

Return the number of DLA engines available to this builder.

◆ getPluginRegistry()

IPluginRegistry & nvinfer1::IBuilder::getPluginRegistry ( )

inlinenoexcept

get the local plugin registry that can be used by the builder.

Returns: The local plugin registry that can be used by the builder.

◆ isNetworkSupported()

bool nvinfer1::IBuilder::isNetworkSupported	(	INetworkDefinition const &	network,
		IBuilderConfig const &	config
	)		const

inlinenoexcept

Checks that a network is within the scope of the IBuilderConfig settings.

Parameters

network	The network definition to check for configuration compliance.
config	The configuration of the builder to use when checking `network`.

Given an INetworkDefinition, network, and an IBuilderConfig, config, check if the network falls within the constraints of the builder configuration based on the EngineCapability, BuilderFlag, and DeviceType. If the network is within the constraints, then the function returns true, and false if a violation occurs. This function reports the conditions that are violated to the registered ErrorRecorder.

Returns: True if network is within the scope of the restrictions specified by the builder config, false otherwise.

Note: This function will synchronize the cuda stream returned by config.getProfileStream() before returning.

This function is only supported in NVIDIA Drive(R) products.

◆ platformHasFastFp16()

bool nvinfer1::IBuilder::platformHasFastFp16 ( ) const

inlinenoexcept

Determine whether the platform has fast native fp16.

◆ platformHasFastInt8()

bool nvinfer1::IBuilder::platformHasFastInt8 ( ) const

inlinenoexcept

Determine whether the platform has fast native int8.

◆ platformHasTf32()

bool nvinfer1::IBuilder::platformHasTf32 ( ) const

inlinenoexcept

Determine whether the platform has TF32 support.

◆ reset()

void nvinfer1::IBuilder::reset ( )

inlinenoexcept

Resets the builder state to default values.

◆ setErrorRecorder()

void nvinfer1::IBuilder::setErrorRecorder ( IErrorRecorder * recorder )

inlinenoexcept

Set the ErrorRecorder for this interface.

Assigns the ErrorRecorder to this interface. The ErrorRecorder will track all errors during execution. This function will call incRefCount of the registered ErrorRecorder at least once. Setting recorder to nullptr unregisters the recorder with the interface, resulting in a call to decRefCount if a recorder has been registered.

If an error recorder is not set, messages will be sent to the global log stream.

Parameters

recorder The error recorder to register with this interface.

See also: getErrorRecorder()

◆ setGpuAllocator()

void nvinfer1::IBuilder::setGpuAllocator ( IGpuAllocator * allocator )

inlinenoexcept

Set the GPU allocator.

Parameters

allocator Set the GPU allocator to be used by the builder. All GPU memory acquired will use this allocator. If NULL is passed, the default allocator will be used.

Default: uses cudaMalloc/cudaFree.

Note: This allocator will be passed to any engines created via the builder; thus the lifetime of the allocator must span the lifetime of those engines as well as that of the builder. If nullptr is passed, the default allocator will be used.

◆ setMaxBatchSize()

TRT_DEPRECATED void nvinfer1::IBuilder::setMaxBatchSize ( int32_t batchSize )

inlinenoexcept

Set the maximum batch size. This has no effect for networks created with explicit batch dimension mode.

Parameters

batchSize The maximum batch size which can be used at execution time, and also the batch size for which the engine will be optimized.

Deprecated:: Deprecated in TensorRT 8.4.

See also: getMaxBatchSize()

◆ setMaxThreads()

bool nvinfer1::IBuilder::setMaxThreads ( int32_t maxThreads )

inlinenoexcept

Set the maximum number of threads.

Parameters

maxThreads The maximum number of threads that can be used by the builder.

Returns: True if successful, false otherwise.

The default value is 1 and includes the current thread. A value greater than 1 permits TensorRT to use multi-threaded algorithms. A value less than 1 triggers a kINVALID_ARGUMENT error.

Member Data Documentation

◆ mImpl

apiv::VBuilder* nvinfer1::IBuilder::mImpl

protected

The documentation for this class was generated from the following file:

NvInfer.h

Public Member Functions

Protected Attributes

Additional Inherited Members

Detailed Description

Constructor & Destructor Documentation

◆ ~IBuilder()

Member Function Documentation

◆ buildEngineWithConfig()

◆ buildSerializedNetwork()

◆ createBuilderConfig()

◆ createNetworkV2()

◆ createOptimizationProfile()

◆ destroy()

◆ getErrorRecorder()

◆ getLogger()

◆ getMaxBatchSize()

◆ getMaxDLABatchSize()

◆ getMaxThreads()

◆ getNbDLACores()

◆ getPluginRegistry()

◆ isNetworkSupported()

◆ platformHasFastFp16()

◆ platformHasFastInt8()

◆ platformHasTf32()

◆ reset()

◆ setErrorRecorder()

◆ setGpuAllocator()

◆ setMaxBatchSize()

◆ setMaxThreads()

Member Data Documentation

◆ mImpl