Algorithm Selector

Warning

Algorithm Selector is deprecated in TensorRT 10.8. Please use editable mode in ITimingCache instead.

class tensorrt.IAlgorithmIOInfo

[DEPRECATED] Deprecated in TensorRT 10.8. Please use editable mode in ITimingCache instead. This class carries information about input or output of the algorithm. IAlgorithmIOInfo for all the input and output along with IAlgorithmVariant denotes the variation of algorithm and can be used to select or reproduce an algorithm using IAlgorithmSelector.select_algorithms().

Variables:

  • dtype – DataType DataType of the input/output of algorithm.

  • strides – Dims strides of the input/output tensor of algorithm.

  • vectorized_dim – int the index of the vectorized dimension or -1 for non-vectorized formats.

  • components_per_element – int the number of components per element. This is always 1 for non-vectorized formats.

__init__(*args, **kwargs)
class tensorrt.IAlgorithmVariant

[DEPRECATED] Deprecated in TensorRT 10.8. Please use editable mode in ITimingCache instead. provides a unique 128-bit identifier, which along with the input and output information denotes the variation of algorithm and can be used to select or reproduce an algorithm, using IAlgorithmSelector.select_algorithms() see IAlgorithmIOInfo, IAlgorithm, IAlgorithmSelector.select_algorithms() note A single implementation can have multiple tactics.

Variables:

  • implementation – int implementation of the algorithm.

  • tactic – int tactic of the algorithm.

__init__(*args, **kwargs)
class tensorrt.IAlgorithmContext

[DEPRECATED] Deprecated in TensorRT 10.8. Please use editable mode in ITimingCache instead. Describes the context and requirements, that could be fulfilled by one or more instances of IAlgorithm. see IAlgorithm

Variables:

  • name – str name of the algorithm node.

  • num_inputs – int number of inputs of the algorithm.

  • num_outputs – int number of outputs of the algorithm.

__init__(*args, **kwargs)
get_shape(self: tensorrt.tensorrt.IAlgorithmContext, index: int) → List[tensorrt.tensorrt.Dims]

[DEPRECATED] Deprecated in TensorRT 10.8. Please use editable mode in ITimingCache instead. Get the minimum / optimum / maximum dimensions for a dynamic input tensor.

Parameters:

index – Index of the input or output of the algorithm. Incremental numbers assigned to indices of inputs and the outputs.

Returns:

A List[Dims] of length 3, containing the minimum, optimum, and maximum shapes, in that order. If the shapes have not been set yet, an empty list is returned.`

class tensorrt.IAlgorithm

[DEPRECATED] Deprecated in TensorRT 10.8. Please use editable mode in ITimingCache instead. Application-implemented interface for selecting and reporting the tactic selection of a layer. Tactic Selection is a step performed by the builder for deciding best algorithms for a layer.

Variables:

  • algorithm_variant – IAlgorithmVariant& the algorithm variant.

  • timing_msec – float The time in milliseconds to execute the algorithm.

  • workspace_size – int The size of the GPU temporary memory in bytes which the algorithm uses at execution time.

__init__(*args, **kwargs)
get_algorithm_io_info(self: tensorrt.tensorrt.IAlgorithm, index: int) → tensorrt.tensorrt.IAlgorithmIOInfo

[DEPRECATED] Deprecated in TensorRT 10.8. Please use editable mode in ITimingCache instead. A single call for both inputs and outputs. Incremental numbers assigned to indices of inputs and the outputs.

Parameters:

index – Index of the input or output of the algorithm. Incremental numbers assigned to indices of inputs and the outputs.

Returns:

A IAlgorithmIOInfo&

class tensorrt.IAlgorithmSelector(self: tensorrt.tensorrt.IAlgorithmSelector)

[DEPRECATED] Deprecated in TensorRT 10.8. Please use editable mode in ITimingCache instead. Interface implemented by application for selecting and reporting algorithms of a layer provided by the builder. note A layer in context of algorithm selection may be different from ILayer in INetworkDefiniton. For example, an algorithm might be implementing a conglomeration of multiple ILayers in INetworkDefinition.

To implement a custom algorithm selector, ensure that you explicitly instantiate the base class in __init__() :

class MyAlgoSelector(trt.IAlgorithmSelector):
    def __init__(self):
        trt.IAlgorithmSelector.__init__(self)
__init__(self: tensorrt.tensorrt.IAlgorithmSelector) → None
report_algorithms(self: tensorrt.tensorrt.IAlgorithmSelector, contexts: List[tensorrt.tensorrt.IAlgorithmContext], choices: List[tensorrt.tensorrt.IAlgorithm]) → None

[DEPRECATED] Deprecated in TensorRT 10.8. Please use editable mode in ITimingCache instead. Called by TensorRT to report choices it made.

Note: For a given optimization profile, this call comes after all calls to select_algorithms. choices[i] is the choice that TensorRT made for algoContexts[i], for i in [0, num_algorithms-1]

For example, a possible implementation may look like this:

def report_algorithms(self, contexts, choices):
    # Prints the time of the chosen algorithm by TRT from the
    # selection list passed in by select_algorithms
    for choice in choices:
        print(choice.timing_msec)
Parameters:

  • contexts – The list of all algorithm contexts.

  • choices – The list of algorithm choices made by TensorRT corresponding to each context.

select_algorithms(self: tensorrt.tensorrt.IAlgorithmSelector, context: tensorrt.tensorrt.IAlgorithmContext, choices: List[tensorrt.tensorrt.IAlgorithm]) → List[int]

[DEPRECATED] Deprecated in TensorRT 10.8. Please use editable mode in ITimingCache instead. Select Algorithms for a layer from the given list of algorithm choices.

Note: TRT uses its default algorithm selection to choose from the list returned by the user. If the returned list is empty, TRT’s default algorithm selection is used unless strict type constraints are set. The list of choices is valid only for this specific algorithm context.

For example, the simplest implementation looks like this:

def select_algorithms(self, context, choices):
    assert len(choices) > 0
    return list(range(len(choices)))
Parameters:

  • context – The context for which the algorithm choices are valid.

  • choices – The list of algorithm choices to select for implementation of this layer.

Returns:

A List[int] indicating the indices from the choices vector that TensorRT should choose from.