TensorRT 10.0.0
nvinfer1::IBuilder Class Reference

Builds an engine from a network definition. More...

#include <NvInfer.h>

Inheritance diagram for nvinfer1::IBuilder:
nvinfer1::INoCopy

Public Member Functions

virtual ~IBuilder () noexcept=default
 
bool platformHasFastFp16 () const noexcept
 Determine whether the platform has fast native fp16. More...
 
bool platformHasFastInt8 () const noexcept
 Determine whether the platform has fast native int8. More...
 
int32_t getMaxDLABatchSize () const noexcept
 Get the maximum batch size DLA can support. For any tensor the total volume of index dimensions combined(dimensions other than CHW) with the requested batch size should not exceed the value returned by this function. More...
 
int32_t getNbDLACores () const noexcept
 Return the number of DLA engines available to this builder. More...
 
void setGpuAllocator (IGpuAllocator *allocator) noexcept
 Set the GPU allocator. More...
 
nvinfer1::IBuilderConfigcreateBuilderConfig () noexcept
 Create a builder configuration object. More...
 
nvinfer1::INetworkDefinitioncreateNetworkV2 (NetworkDefinitionCreationFlags flags) noexcept
 Create a network definition object. More...
 
nvinfer1::IOptimizationProfilecreateOptimizationProfile () noexcept
 Create a new optimization profile. More...
 
void setErrorRecorder (IErrorRecorder *recorder) noexcept
 Set the ErrorRecorder for this interface. More...
 
IErrorRecordergetErrorRecorder () const noexcept
 get the ErrorRecorder assigned to this interface. More...
 
void reset () noexcept
 Resets the builder state to default values. More...
 
bool platformHasTf32 () const noexcept
 Determine whether the platform has TF32 support. More...
 
nvinfer1::IHostMemorybuildSerializedNetwork (INetworkDefinition &network, IBuilderConfig &config) noexcept
 Builds and serializes a network for the given INetworkDefinition and IBuilderConfig. More...
 
bool isNetworkSupported (INetworkDefinition const &network, IBuilderConfig const &config) const noexcept
 Checks that a network is within the scope of the IBuilderConfig settings. More...
 
ILoggergetLogger () const noexcept
 get the logger with which the builder was created More...
 
bool setMaxThreads (int32_t maxThreads) noexcept
 Set the maximum number of threads. More...
 
int32_t getMaxThreads () const noexcept
 get the maximum number of threads that can be used by the builder. More...
 
IPluginRegistrygetPluginRegistry () noexcept
 get the local plugin registry that can be used by the builder. More...
 

Protected Attributes

apiv::VBuilder * mImpl
 

Additional Inherited Members

- Protected Member Functions inherited from nvinfer1::INoCopy
 INoCopy ()=default
 
virtual ~INoCopy ()=default
 
 INoCopy (INoCopy const &other)=delete
 
INoCopyoperator= (INoCopy const &other)=delete
 
 INoCopy (INoCopy &&other)=delete
 
INoCopyoperator= (INoCopy &&other)=delete
 

Detailed Description

Builds an engine from a network definition.

Warning
Do not inherit from this class, as doing so will break forward-compatibility of the API and ABI.

Constructor & Destructor Documentation

◆ ~IBuilder()

virtual nvinfer1::IBuilder::~IBuilder ( )
virtualdefaultnoexcept

Member Function Documentation

◆ buildSerializedNetwork()

nvinfer1::IHostMemory * nvinfer1::IBuilder::buildSerializedNetwork ( INetworkDefinition network,
IBuilderConfig config 
)
inlinenoexcept

Builds and serializes a network for the given INetworkDefinition and IBuilderConfig.

This function allows building and serialization of a network without creating an engine.

Parameters
networkNetwork definition.
configBuilder configuration.
Returns
A pointer to a IHostMemory object that contains a serialized network.
Note
This function will synchronize the cuda stream returned by config.getProfileStream() before returning.
See also
INetworkDefinition, IBuilderConfig, IHostMemory

◆ createBuilderConfig()

nvinfer1::IBuilderConfig * nvinfer1::IBuilder::createBuilderConfig ( )
inlinenoexcept

Create a builder configuration object.

See also
IBuilderConfig

◆ createNetworkV2()

nvinfer1::INetworkDefinition * nvinfer1::IBuilder::createNetworkV2 ( NetworkDefinitionCreationFlags  flags)
inlinenoexcept

Create a network definition object.

Creates a network definition object with immutable properties specified using the flags parameter.

createNetworkV2 supports creating network with properties from NetworkDefinitionCreationFlags.

CreateNetworkV2 supports dynamic shapes and explicit batch dimensions by default.

createNetworkV2 with NetworkDefinitionCreationFlag::kSTRONGLY_TYPED flag supports creating a strongly typed plan where tensor data types are inferred from network input types and operator type specification.

Parameters
flagsBitset of NetworkDefinitionCreationFlags specifying network properties combined with bitwise OR. e.g., 1U << NetworkDefinitionCreationFlag::kSTRONGLY_TYPED
See also
INetworkDefinition, NetworkDefinitionCreationFlags

◆ createOptimizationProfile()

nvinfer1::IOptimizationProfile * nvinfer1::IBuilder::createOptimizationProfile ( )
inlinenoexcept

Create a new optimization profile.

If the network has any dynamic input tensors, the appropriate calls to setDimensions() must be made. Likewise, if there are any shape input tensors, the appropriate calls to setShapeValues() are required. The builder retains ownership of the created optimization profile and returns a raw pointer, i.e. the users must not attempt to delete the returned pointer.

See also
IOptimizationProfile

◆ getErrorRecorder()

IErrorRecorder * nvinfer1::IBuilder::getErrorRecorder ( ) const
inlinenoexcept

get the ErrorRecorder assigned to this interface.

Retrieves the assigned error recorder object for the given class. A nullptr will be returned if setErrorRecorder has not been called.

Returns
A pointer to the IErrorRecorder object that has been registered.
See also
setErrorRecorder()

◆ getLogger()

ILogger * nvinfer1::IBuilder::getLogger ( ) const
inlinenoexcept

get the logger with which the builder was created

Returns
the logger

◆ getMaxDLABatchSize()

int32_t nvinfer1::IBuilder::getMaxDLABatchSize ( ) const
inlinenoexcept

Get the maximum batch size DLA can support. For any tensor the total volume of index dimensions combined(dimensions other than CHW) with the requested batch size should not exceed the value returned by this function.

Warning
getMaxDLABatchSize does not work with dynamic shapes.

◆ getMaxThreads()

int32_t nvinfer1::IBuilder::getMaxThreads ( ) const
inlinenoexcept

get the maximum number of threads that can be used by the builder.

Retrieves the maximum number of threads that can be used by the builder.

Returns
The maximum number of threads that can be used by the builder.
See also
setMaxThreads()

◆ getNbDLACores()

int32_t nvinfer1::IBuilder::getNbDLACores ( ) const
inlinenoexcept

Return the number of DLA engines available to this builder.

◆ getPluginRegistry()

IPluginRegistry & nvinfer1::IBuilder::getPluginRegistry ( )
inlinenoexcept

get the local plugin registry that can be used by the builder.

Returns
The local plugin registry that can be used by the builder.

◆ isNetworkSupported()

bool nvinfer1::IBuilder::isNetworkSupported ( INetworkDefinition const &  network,
IBuilderConfig const &  config 
) const
inlinenoexcept

Checks that a network is within the scope of the IBuilderConfig settings.

Parameters
networkThe network definition to check for configuration compliance.
configThe configuration of the builder to use when checking network.

Given an INetworkDefinition, network, and an IBuilderConfig, config, check if the network falls within the constraints of the builder configuration based on the EngineCapability, BuilderFlag, and DeviceType. If the network is within the constraints, then the function returns true, and false if a violation occurs. This function reports the conditions that are violated to the registered ErrorRecorder.

Returns
True if network is within the scope of the restrictions specified by the builder config, false otherwise.
Note
This function will synchronize the cuda stream returned by config.getProfileStream() before returning.

◆ platformHasFastFp16()

bool nvinfer1::IBuilder::platformHasFastFp16 ( ) const
inlinenoexcept

Determine whether the platform has fast native fp16.

◆ platformHasFastInt8()

bool nvinfer1::IBuilder::platformHasFastInt8 ( ) const
inlinenoexcept

Determine whether the platform has fast native int8.

◆ platformHasTf32()

bool nvinfer1::IBuilder::platformHasTf32 ( ) const
inlinenoexcept

Determine whether the platform has TF32 support.

◆ reset()

void nvinfer1::IBuilder::reset ( )
inlinenoexcept

Resets the builder state to default values.

◆ setErrorRecorder()

void nvinfer1::IBuilder::setErrorRecorder ( IErrorRecorder recorder)
inlinenoexcept

Set the ErrorRecorder for this interface.

Assigns the ErrorRecorder to this interface. The ErrorRecorder will track all errors during execution. This function will call incRefCount of the registered ErrorRecorder at least once. Setting recorder to nullptr unregisters the recorder with the interface, resulting in a call to decRefCount if a recorder has been registered.

If an error recorder is not set, messages will be sent to the global log stream.

Parameters
recorderThe error recorder to register with this interface.
See also
getErrorRecorder()

◆ setGpuAllocator()

void nvinfer1::IBuilder::setGpuAllocator ( IGpuAllocator allocator)
inlinenoexcept

Set the GPU allocator.

Parameters
allocatorSet the GPU allocator to be used by the builder. All GPU memory acquired will use this allocator. If NULL is passed, the default allocator will be used.

Default: uses cudaMalloc/cudaFree.

Note
This allocator will be passed to any engines created via the builder; thus the lifetime of the allocator must span the lifetime of those engines as well as that of the builder. If nullptr is passed, the default allocator will be used.

◆ setMaxThreads()

bool nvinfer1::IBuilder::setMaxThreads ( int32_t  maxThreads)
inlinenoexcept

Set the maximum number of threads.

Parameters
maxThreadsThe maximum number of threads that can be used by the builder.
Returns
True if successful, false otherwise.

The default value is 1 and includes the current thread. A value greater than 1 permits TensorRT to use multi-threaded algorithms. A value less than 1 triggers a kINVALID_ARGUMENT error.

Member Data Documentation

◆ mImpl

apiv::VBuilder* nvinfer1::IBuilder::mImpl
protected

The documentation for this class was generated from the following file: