API Migration Guide

This section highlights the TensorRT API modifications. If you are unfamiliar with these changes, refer to our sample code for clarification.

Python

Python API Changes

Allocating Buffers and Using a Name-Based Engine API

TensorRT 8.x

def allocate_buffers(self, engine):
    '''
    Allocates all buffers required for an engine, i.e., host/device inputs/outputs.
    '''
    inputs = []
    outputs = []
    bindings = []
    stream = cuda.Stream()

    # binding is the name of input/output
    for binding in the engine:
        size = trt.volume(engine.get_binding_shape(binding)) * engine.max_batch_size
        dtype = trt.nptype(engine.get_binding_dtype(binding))

        # Allocate host and device buffers
        host_mem = cuda.pagelocked_empty(size, dtype) # page-locked memory buffer (won't swap to disk)
        device_mem = cuda.mem_alloc(host_mem.nbytes)

        # Append the device buffer address to device bindings.
        # When cast to int, it's a linear index into the context's memory (like memory address).
        bindings.append(int(device_mem))

        # Append to the appropriate input/output list.
        if engine.binding_is_input(binding):
            inputs.append(self.HostDeviceMem(host_mem, device_mem))
        else:
            outputs.append(self.HostDeviceMem(host_mem, device_mem))

    return inputs, outputs, bindings, stream

TensorRT 10.0

def allocate_buffers(self, engine):
    '''
    Allocates all buffers required for an engine, i.e., host/device inputs/outputs.
    '''
    inputs = []
    outputs = []
    bindings = []
    stream = cuda.Stream()

    for i in range(engine.num_io_tensors):
        tensor_name = engine.get_tensor_name(i)
        size = trt.volume(engine.get_tensor_shape(tensor_name))
        dtype = trt.nptype(engine.get_tensor_dtype(tensor_name))

        # Allocate host and device buffers
        host_mem = cuda.pagelocked_empty(size, dtype) # page-locked memory buffer (won't swap to disk)
        device_mem = cuda.mem_alloc(host_mem.nbytes)

        # Append the device buffer address to device bindings.
        # When cast to int, it's a linear index into the context's memory (like memory address).
        bindings.append(int(device_mem))

        # Append to the appropriate input/output list.
        if engine.get_tensor_mode(tensor_name) == trt.TensorIOMode.INPUT:
            inputs.append(self.HostDeviceMem(host_mem, device_mem))
        else:
            outputs.append(self.HostDeviceMem(host_mem, device_mem))

    return inputs, outputs, bindings, stream

Transition from enqueueV2 to enqueueV3 for Python

TensorRT 8.x

# Allocate device memory for inputs.
d_inputs = [cuda.mem_alloc(input_nbytes) for binding in range(input_num)]

# Allocate device memory for outputs.
h_output = cuda.pagelocked_empty(output_nbytes, dtype=np.float32)
d_output = cuda.mem_alloc(h_output.nbytes)

# Transfer data from host to device.
cuda.memcpy_htod_async(d_inputs[0], input_a, stream)
cuda.memcpy_htod_async(d_inputs[1], input_b, stream)
cuda.memcpy_htod_async(d_inputs[2], input_c, stream)

# Run inference
context.execute_async_v2(bindings=[int(d_inp) for d_inp in d_inputs] + [int(d_output)], stream_handle=stream.handle)

# Synchronize the stream
stream.synchronize()

TensorRT 10.0

# Allocate device memory for inputs.
d_inputs = [cuda.mem_alloc(input_nbytes) for binding in range(input_num)]

# Allocate device memory for outputs.
h_output = cuda.pagelocked_empty(output_nbytes, dtype=np.float32)
d_output = cuda.mem_alloc(h_output.nbytes)

# Transfer data from host to device.
cuda.memcpy_htod_async(d_inputs[0], input_a, stream)
cuda.memcpy_htod_async(d_inputs[1], input_b, stream)
cuda.memcpy_htod_async(d_inputs[2], input_c, stream)

# Setup tensor address
bindings = [int(d_inputs[i]) for i in range(3)] + [int(d_output)]

for i in range(engine.num_io_tensors):
    context.set_tensor_address(engine.get_tensor_name(i), bindings[i])

# Run inference
context.execute_async_v3(stream_handle=stream.handle)

# Synchronize the stream
stream.synchronize()

Engine Building, use only build_serialized_network

TensorRT 8.x

engine_bytes = None
try:
    engine_bytes = self.builder.build_serialized_network(self.network, self.config)
except AttributeError:
    engine = self.builder.build_engine(self.network, self.config)
    engine_bytes = engine.serialize()
    del engine
assert engine_bytes

TensorRT 10.0

engine_bytes = self.builder.build_serialized_network(self.network, self.config)
if engine_bytes is None:
    log.error("Failed to create engine")
    sys.exit(1)

Added Python APIs

Types

• APILanguage

• ExecutionContextAllocationStrategy

• IGpuAsyncAllocator

• InterfaceInfo

• IPluginResource

• IPluginV3

• IStreamReader

• IVersionedInterface

Methods and Properties

• ICudaEngine.is_debug_tensor()

• ICudaEngine.minimum_weight_streaming_budget

• ICudaEngine.streamable_weights_size

• ICudaEngine.weight_streaming_budget

• IExecutionContext.get_debug_listener()

• IExecutionContext.get_debug_state()

• IExecutionContext.set_all_tensors_debug_state()

• IExecutionContext.set_debug_listener()

• IExecutionContext.set_tensor_debug_state()

• IExecutionContext.update_device_memory_size_for_shapes()

• IGpuAllocator.allocate_async()

• IGpuAllocator.deallocate_async()

• INetworkDefinition.add_plugin_v3()

• INetworkDefinition.is_debug_tensor()

• INetworkDefinition.mark_debug()

• INetworkDefinition.unmark_debug()

• IPluginRegistry.acquire_plugin_resource()

• IPluginRegistry.all_creators

• IPluginRegistry.deregister_creator()

• IPluginRegistry.get_creator()

• IPluginRegistry.register_creator()

• IPluginRegistry.release_plugin_resource()

Removed Python APIs

Removed Python APIs and their Suggested Superseded API

Python API	Superseded API
BuilderFlag.ENABLE_TACTIC_HEURISTIC	Builder optimization level 2
BuilderFlag.STRICT_TYPES	Use all three flags: BuilderFlag.DIRECT_IO BuilderFlag.PREFER_PRECISION_CONSTRAINTS BuilderFlag.REJECT_EMPTY_ALGORITHMS
EngineCapability.DEFAULT EngineCapability.kSAFE_DLA EngineCapability.SAFE_GPU	EngineCapability.STANDARD EngineCapability.DLA_STANDALONE EngineCapability.SAFETY
IAlgorithmIOInfo.tensor_format	The strides, data type, and vectorization information are sufficient to identify tensor formats uniquely.
IBuilder.max_batch_size	Implicit batch is no longer supported.
IBuilderConfig.max_workspace_size	IBuilderConfig.set_memory_pool_limit() with MemoryPoolType.WORKSPACE IBuilderConfig.get_memory_pool_limit() with MemoryPoolType.WORKSPACE
IBuilderConfig.min_timing_iterations	IBuilderConfig.avg_timing_iterations
ICudaEngine.binding_is_input() ICudaEngine.get_binding_bytes_per_component() ICudaEngine.get_binding_components_per_element() ICudaEngine.get_binding_dtype() ICudaEngine.get_binding_format() ICudaEngine.get_binding_format_desc() ICudaEngine.get_binding_index() ICudaEngine.get_binding_name() ICudaEngine.get_binding_shape() ICudaEngine.get_binding_vectorized_dim() ICudaEngine.get_location() ICudaEngine.get_profile_shape() ICudaEngine.get_profile_shape_input() ICudaEngine.has_implicit_batch_dimension() ICudaEngine.is_execution_binding() ICudaEngine.is_shape_binding() ICudaEngine.max_batch_size() ICudaEngine.num_bindings()	ICudaEngine.get_tensor_mode() ICudaEngine.get_tensor_bytes_per_component() ICudaEngine.get_tensor_components_per_element() ICudaEngine.get_tensor_dtype() ICudaEngine.get_tensor_format() ICudaEngine.get_tensor_format_desc() No name-based equivalent replacement No name-based equivalent replacement ICudaEngine.get_tensor_shape() ICudaEngine.get_tensor_vectorized_dim() ITensor.location ICudaEngine.get_tensor_profile_shape() ICudaEngine.get_tensor_profile_values() Implicit batch is no longer supported No name-based equivalent replacement ICudaEngine.is_shape_inference_io() Implicit batch is no longer supported ICudaEngine.num_io_tensors()
IExecutionContext.get_binding_shape() IExecutionContext.get_strides() IExecutionContext.set_binding_shape()	IExecutionContext.get_tensor_shape() IExecutionContext.get_tensor_strides() IExecutionContext.set_input_shape()
IFullyConnectedLayer	IMatrixMultiplyLayer
INetworkDefinition.add_convolution() INetworkDefinition.add_deconvolution() INetworkDefinition.add_fully_connected() INetworkDefinition.add_padding() INetworkDefinition.add_pooling() INetworkDefinition.add_rnn_v2() INetworkDefinition.has_explicit_precision INetworkDefinition.has_implicit_batch_dimension	INetworkDefinition.add_convolution_nd() INetworkDefinition.add_deconvolution_nd() INetworkDefinition.add_matrix_multiply() INetworkDefinition.add_padding_nd() INetworkDefinition.add_pooling_nd() INetworkDefinition.add_loop() Explicit precision support is removed in 10.0 Implicit batch is no longer supported
IRNNv2Layer	ILoop
NetworkDefinitionCreationFlag.EXPLICIT_BATCH NetworkDefinitionCreationFlag.EXPLICIT_PRECISION	Support is removed in 10.0
PaddingMode.CAFFE_ROUND_DOWN PaddingMode.CAFFE_ROUND_UP	Caffe is not supported since 9.0
PreviewFeature.DISABLE_EXTERNAL_TACTIC_SOURCES_FOR_CORE_0805 PreviewFeature.FASTER_DYNAMIC_SHAPES_0805	External tactics are always disabled for core code This flag is on by default
ProfilingVerbosity.DEFAULT ProfilingVerbosity.VERBOSE	ProfilingVerbosity.LAYER_NAMES_ONLY ProfilingVerbosity.DETAILED
ResizeMode	Use InterpolationMode. Alias was removed.
SampleMode.DEFAULT	SampleMode.STRICT_BOUNDS
SliceMode	Use SampleMode. Alias was removed.

C++

C++ API Changes

Transition from enqueueV2 to enqueueV3 for C++

TensorRT 8.x

// Create RAII buffer manager object.
samplesCommon::BufferManager buffers(mEngine);

auto context = SampleUniquePtr<nvinfer1::IExecutionContext>(mEngine->createExecutionContext());
if (!context)
{
    return false;
}

// Pick a random digit to try to infer.
srand(time(NULL));
int32_t const digit = rand() % 10;

// Read the input data into the managed buffers.
// There should be just 1 input tensor.
ASSERT(mParams.inputTensorNames.size() == 1);

if (!processInput(buffers, mParams.inputTensorNames[0], digit))
{
    return false;
}
// Create a CUDA stream to execute this inference.
cudaStream_t stream;
CHECK(cudaStreamCreate(&stream));

// Asynchronously copy data from host input buffers to device input
buffers.copyInputToDeviceAsync(stream);

// Asynchronously enqueue the inference work
if (!context->enqueueV2(buffers.getDeviceBindings().data(), stream, nullptr))
{
    return false;
}
// Asynchronously copy data from device output buffers to host output buffers.
buffers.copyOutputToHostAsync(stream);

// Wait for the work in the stream to complete.
CHECK(cudaStreamSynchronize(stream));

// Release stream.
CHECK(cudaStreamDestroy(stream));

TensorRT 10.0

// Create RAII buffer manager object.
samplesCommon::BufferManager buffers(mEngine);

auto context = SampleUniquePtr<nvinfer1::IExecutionContext>(mEngine->createExecutionContext());
if (!context)
{
    return false;
}

for (int32_t i = 0, e = mEngine->getNbIOTensors(); i < e; i++)
{
    auto const name = mEngine->getIOTensorName(i);
    context->setTensorAddress(name, buffers.getDeviceBuffer(name));
}

// Pick a random digit to try to infer.
srand(time(NULL));
int32_t const digit = rand() % 10;

// Read the input data into the managed buffers.
// There should be just 1 input tensor.
ASSERT(mParams.inputTensorNames.size() == 1);

if (!processInput(buffers, mParams.inputTensorNames[0], digit))
{
    return false;
}
// Create a CUDA stream to execute this inference.
cudaStream_t stream;
CHECK(cudaStreamCreate(&stream));

// Asynchronously copy data from host input buffers to device input
buffers.copyInputToDeviceAsync(stream);

// Asynchronously enqueue the inference work
if (!context->enqueueV3(stream))
{
    return false;
}

// Asynchronously copy data from device output buffers to host output buffers.
buffers.copyOutputToHostAsync(stream);

// Wait for the work in the stream to complete.
CHECK(cudaStreamSynchronize(stream));

// Release stream.
CHECK(cudaStreamDestroy(stream));

64-Bit Dimension Changes

The dimensions held by Dims changed from int32_t to int64_t. However, in TensorRT 10.0, TensorRT will generally reject networks that use dimensions exceeding the range of int32_t. The tensor type returned by IShapeLayer is now DataType::kINT64. Use ICastLayer to cast the result to the tensor of type DataType::kINT32 if 32-bit dimensions are required.

Inspect code that bitwise copies to and from Dims to ensure it is correct for int64_t dimensions.

Added C++ APIs

Enums

• ActivationType::kGELU_ERF

• ActivationType::kGELU_TANH

• BuilderFlag::kREFIT_IDENTICAL

• BuilderFlag::kSTRIP_PLAN

• BuilderFlag::kWEIGHT_STREAMING

• Datatype::kINT4

• LayerType::kPLUGIN_V3

Types

• APILanguage

• Dims64

• ExecutionContextAllocationStrategy

• IGpuAsyncAllocator

• InterfaceInfo

• IPluginResource

• IPluginV3

• IStreamReader

• IVersionedInterface

Methods and Properties

• getInferLibBuildVersion

• getInferLibMajorVersion

• getInferLibMinorVersion

• getInferLibPatchVersion

• ICudaEngine::createRefitter

• IcudaEngine::getMinimumWeightStreamingBudget

• IcudaEngine::getStreamableWeightsSize

• ICudaEngine::getWeightStreamingBudget

• IcudaEngine::isDebugTensor

• ICudaEngine::setWeightStreamingBudget

• IExecutionContext::getDebugListener

• IExecutionContext::getTensorDebugState

• IExecutionContext::setAllTensorsDebugState

• IExecutionContext::setDebugListener

• IExecutionContext::setOuputTensorAddress

• IExecutionContext::setTensorDebugState

• IExecutionContext::updateDeviceMemorySizeForShapes

• IGpuAllocator::allocateAsync

• IGpuAllocator::deallocateAsync

• INetworkDefinition::addPluginV3

• INetworkDefinition::isDebugTensor

• INetworkDefinition::markDebug

• INetworkDefinition::unmarkDebug

• IPluginRegistry::acquirePluginResource

• IPluginRegistry::deregisterCreator

• IPluginRegistry::getAllCreators

• IPluginRegistry::getCreator

• IPluginRegistry::registerCreator

• IPluginRegistry::releasePluginResource

Removed C++ APIs

Removed C++ APIs and their Suggested Superseded API

C++ API	Superseded API
BuilderFlag::kENABLE_TACTIC_HEURISTIC	Builder optimization level 2
BuilderFlag::kSTRICT_TYPES	Use all three flags: kREJECT_EMPTY_ALGORITHMS kDIRECT_IO kPREFER_PRECISION_CONSTRAINTS Note When removing enum members (for all enums in this list), we will change enumeration to sequential numbers.
EngineCapability::kDEFAULT EngineCapability::kSAFE_DLA EngineCapability::kSAFE_GPU	EngineCapability::kSTANDARD EngineCapability::kDLA_STANDALONE EngineCapability::kSAFETY
IAlgorithm::getAlgorithmIOInfo()	IAlgorithm::getAlgorithmIOInfoByIndex()
IAlgorithmIOInfo::getTensorFormat()	The strides, data type, and vectorization information are sufficient to identify tensor formats uniquely.
IBuilder::buildEngineWithConfig() IBuilder::destroy() IBuilder::getMaxBatchSize() IBuilder::setMaxBatchSize()	IBuilder::buildSerializedNetwork() delete ObjectName Implicit batch is no longer supported Implicit batch is no longer supported
IBuilderConfig::destroy() IBuilderConfig::getMaxWorkspaceSize() IBuilderConfig::getMinTimingIterations() IBuilderConfig::setMaxWorkspaceSize() IBuilderConfig::setMinTimingIterations()	delete ObjectName IBuilderConfig::getMemoryPoolLimit() with MemoryPoolType::kWORKSPACE IBuilderConfig::getAvgTimingIterations() IBuilderConfig::setMemoryPoolLimit() with MemoryPoolType::kWORKSPACE IBuilderConfig::setAvgTimingIterations()
IConvolutionLayer::getDilation() IConvolutionLayer::getKernelSize() IConvolutionLayer::getPadding() IConvolutionLayer::getStride() IConvolutionLayer::setDilation() IConvolutionLayer::setKernelSize() IConvolutionLayer::setPadding() IConvolutionLayer::setStride()	IConvolutionLayer::getDilationNd() IConvolutionLayer::getKernelSizeNd() IConvolutionLayer::getPaddingNd() IConvolutionLayer::getStrideNd() IConvolutionLayer::setDilationNd() IConvolutionLayer::setKernelSizeNd() IConvolutionLayer::setPaddingNd() IConvolutionLayer::setStrideNd()
ICudaEngine::bindingIsInput() ICudaEngine::destroy() ICudaEngine::getBindingBytesPerComponent() ICudaEngine::getBindingComponentsPerElement() ICudaEngine::getBindingDataType() ICudaEngine::getBindingDimensions() ICudaEngine::getBindingFormat() ICudaEngine::getBindingFormatDesc() ICudaEngine::getBindingIndex() ICudaEngine::getBindingName() ICudaEngine::getBindingVectorizedDim() ICudaEngine::getLocation() ICudaEngine::getMaxBatchSize() ICudaEngine::getNbBindings() ICudaEngine::getProfileDimensions() ICudaEngine::getProfileShapeValues() ICudaEngine::hasImplicitBatchDimension() ICudaEngine::isExecutionBinding() ICudaEngine::isShapeBinding()	ICudaEngine::getTensorIOMode() delete ObjectName ICudaEngine::getTensorBytesPerComponent() ICudaEngine::getTensorComponentsPerElement() ICudaEngine::getTensorDataType() ICudaEngine::getTensorShape() ICudaEngine::getTensorFormat() ICudaEngine::getTensorFormatDesc() Name-based methods Name-based methods ICudaEngine::getTensorVectorizedDim() ITensor::getLocation() Implicit batch is no longer supported ICudaEngine::getNbIOTensors() ICudaEngine::getProfileShape() ICudaEngine::getShapeValues() Implicit batch is no longer supported No name-based equivalent replacement ICudaEngine::isShapeInferenceIO()
IDeconvolutionLayer::getKernelSize() IDeconvolutionLayer::getPadding() IDeconvolutionLayer::getStride() IDeconvolutionLayer::setKernelSize() IDeconvolutionLayer::setPadding() IDeconvolutionLayer::setStride()	IDeconvolutionLayer::getKernelSizeNd() IDeconvolutionLayer::getPaddingNd() IDeconvolutionLayer::getStrideNd() IDeconvolutionLayer::setKernelSizeNd() IDeconvolutionLayer::setPaddingNd() IDeconvolutionLayer::setStrideNd()
IExecutionContext::destroy() IExecutionContext::enqueue() IExecutionContext::enqueueV2() IExecutionContext::execute() IExecutionContext::getBindingDimensions() IExecutionContext::getShapeBinding() IExecutionContext::getStrides() IExecutionContext::setBindingDimensions() IExecutionContext::setInputShapeBinding() IExecutionContext::setOptimizationProfile()	delete ObjectName IExecutionContext::enqueueV3() IExecutionContext::enqueueV3() IExecutionContext::executeV2() IExecutionContext::getTensorShape() IExecutionContext::getTensorAddress() or getOutputTensorAddress() IExecutionContext::getTensorStrides() IExecutionContext::setInputShape() IExecutionContext::setInputTensorAddress() or setTensorAddress() IExecutionContext::setOptimizationProfileAsync()
IFullyConnectedLayer	IMatrixMultiplyLayer
IGpuAllocator::free()	IGpuAllocator::deallocate()
IHostMemory::destroy()	delete ObjectName
INetworkDefinition::addConvolution() INetworkDefinition::addDeconvolution() INetworkDefinition::addFullyConnected() INetworkDefinition::addPadding() INetworkDefinition::addPooling() INetworkDefinition::addRNNv2() INetworkDefinition::destroy() INetworkDefinition::hasExplicitPrecision() INetworkDefinition::hasImplicitBatchDimension()	INetworkDefinition::addConvolutionNd() INetworkDefinition::addDeconvolutionNd() INetworkDefinition::addMatrixMultiply() INetworkDefinition::addPaddingNd() INetworkDefinition::addPoolingNd() INetworkDefinition::addLoop() delete ObjectName Explicit precision support is removed in 10.0 Implicit batch support is removed
IOnnxConfig::destroy()	delete ObjectName
IPaddingLayer::getPostPadding() IPaddingLayer::getPrePadding() IPaddingLayer::setPostPadding() IPaddingLayer::setPrePadding()	IPaddingLayer::getPostPaddingNd() IPaddingLayer::getPrePaddingNd() IPaddingLayer::setPostPaddingNd() IPaddingLayer::setPrePaddingNd()
IPoolingLayer::getPadding() IPoolingLayer::getStride() IPoolingLayer::getWindowSize() IPoolingLayer::setPadding() IPoolingLayer::setStride() IPoolingLayer::setWindowSize()	IPoolingLayer::getPaddingNd() IPoolingLayer::getStrideNd() IPoolingLayer::getWindowSizeNd() IPoolingLayer::setPaddingNd() IPoolingLayer::setStrideNd() IPoolingLayer::setWindowSizeNd()
IRefitter::destroy()	delete ObjectName
IResizeLayer::getAlignCorners() IResizeLayer::setAlignCorners()	IResizeLayer::getAlignCornersNd() IResizeLayer::setAlignCornersNd()
IRuntime::deserializeCudaEngine (void const* blob, std::size_t size, IPluginFactory* pluginFactory) IRuntime::destroy()	Use deserializeCudaEngine with two parameters delete ObjectName
IRNNv2Layer	ILoop
kNV_TENSORRT_VERSION_IMPL	#define NV_TENSORRT_VERSION_INT(major, minor, patch) ((major) *10000L + (minor) *100L + (patch) *1L) Note TensorRT version encoding was changed to accommodate a two-digit minor version.
NetworkDefinitionCreationFlag::kEXPLICIT_BATCH NetworkDefinitionCreationFlag::kEXPLICIT_PRECISION	Support is removed in 10.0
NV_TENSORRT_SONAME_MAJOR NV_TENSORRT_SONAME_MINOR NV_TENSORRT_SONAME_PATCH	NV_TENSORRT_MAJOR NV_TENSORRT_MINOR NV_TENSORRT_PATCH
PaddingMode::kCAFFE_ROUND_DOWN PaddingMode::kCAFFE_ROUND_UP	Caffe is not supported since 9.0
PreviewFeature::kDISABLE_EXTERNAL_TACTIC_SOURCES_FOR_CORE_0805 PreviewFeature::kFASTER_DYNAMIC_SHAPES_0805	External tactics are always disabled for core code This flag is on by default
ProfilingVerbosity::kDEFAULT ProfilingVerbosity::kVERBOSE	ProfilingVerbosity::kLAYER_NAMES_ONLY ProfilingVerbosity::kDETAILED
ResizeMode	Use InterpolationMode. Alias was removed.
RNNDirection RNNGateType RNNInputMode RNNOperation	RNN-related data structures are removed
SampleMode::kDEFAULT	SampleMode::kSTRICT_BOUNDS
SliceMode	Use SampleMode. Alias was removed.

Removed C++ Plugins

Removed C++ Plugins and their Suggested Superseded Plugin

C++ Plugin	Superseded Plugin
createAnchorGeneratorPlugin() createBatchedNMSPlugin() createInstanceNormalizationPlugin() createNMSPlugin() createNormalizePlugin() createPriorBoxPlugin() createRegionPlugin() createReorgPlugin() createRPNROIPlugin() createSplitPlugin()	GridAnchorPluginCreator::createPlugin() BatchedNMSPluginCreator::createPlugin() InstanceNormalizationPluginCreator::createPlugin() NMSPluginCreator::createPlugin() NormalizePluginCreator::createPlugin() PriorBoxPluginCreator::createPlugin() RegionPluginCreator::createPlugin() ReorgPluginCreator::createPlugin() RPROIPluginCreator::createPlugin() INetworkDefinition::addSlice()
struct Quadruple	Related plugins are removed

Removed Safety C++ APIs

Removed Safety C++ APIs and their Suggested Superseded Safety API

Safety C++ API	Superseded Safety API
safe::ICudaEngine::bindingIsInput() safe::ICudaEngine::getBindingBytesPerComponent() safe::ICudaEngine::getBindingComponentsPerElement() safe::ICudaEngine::getBindingDataType() safe::ICudaEngine::getBindingDimensions() safe::ICudaEngine::getBindingIndex() safe::ICudaEngine::getBindingName() safe::ICudaEngine::getBindingVectorizedDim() safe::ICudaEngine::getNbBindings() safe::ICudaEngine::getTensorFormat()	safe::ICudaEngine::tensorIOMode() safe::ICudaEngine::getTensorBytesPerComponent() safe::ICudaEngine::getTensorComponentsPerElement() safe::ICudaEngine::getTensorDataType() safe::ICudaEngine::getTensorShape() safe::name-based methods safe::name-based methods safe::ICudaEngine::getTensorVectorizedDim() safe::ICudaEngine::getNbIOTensors() safe::ICudaEngine::getBindingFormat()
safe::IExecutionContext::enqueueV2() safe::IExecutionContext::getStrides()	safe::IExecutionContext::enqueueV3() safe::IExecutionContext::getTensorStrides()

trtexec

trtexec Flag Changes

Changes to flag workspace and minTiming

TensorRT 8.x

trtexec \
    --onnx=/path/to/model.onnx \
    --saveEngine=/path/to/engine.trt \
    --optShapes=input:$INPUT_SHAPE \
    --avgTiming=1 \
    --workspace=1024 \
    --minTiming=1

TensorRT 10.0

trtexec \
    --onnx=/path/to/model.onnx \
    --saveEngine=/path/to/engine.trt \
    --optShapes=input:$INPUT_SHAPE \
    --avgTiming=1 \
    --memPoolSize=workspace:1024

Removed trtexec Flags

Removed trtexec Flags and their Suggested Superseded Flag

trtexec Flag	Superseded Flag
--minTiming	avgTiming
--preview=features options: disableExternalTacticSourcesForCore0805 fasterDynamicShapes0805	N/A
--workspace=N	--memPoolSize=poolspec

Deprecated trtexec Flags

• --buildOnly

• --explicitPrecision

• --heuristic

• --nvtxMode