|
| InterfaceInfo | getInterfaceInfo () const noexcept override |
| | Return version information associated with this interface. Applications must not override this method. More...
|
| |
| virtual int32_t | configurePlugin (DynamicPluginTensorDesc const *in, int32_t nbInputs, DynamicPluginTensorDesc const *out, int32_t nbOutputs) noexcept=0 |
| | Configure the plugin. More...
|
| |
| virtual int32_t | getOutputDataTypes (DataType *outputTypes, int32_t nbOutputs, const DataType *inputTypes, int32_t nbInputs) const noexcept=0 |
| | Provide the data types of the plugin outputs if the input tensors have the data types provided. More...
|
| |
| virtual int32_t | getOutputShapes (DimsExprs const *inputs, int32_t nbInputs, DimsExprs const *shapeInputs, int32_t nbShapeInputs, DimsExprs *outputs, int32_t nbOutputs, IExprBuilder &exprBuilder) noexcept=0 |
| | Provide expressions for computing dimensions of the output tensors from dimensions of the input tensors. More...
|
| |
| virtual bool | supportsFormatCombination (int32_t pos, DynamicPluginTensorDesc const *inOut, int32_t nbInputs, int32_t nbOutputs) noexcept=0 |
| | Return true if plugin supports the format and datatype for the input/output indexed by pos. More...
|
| |
| virtual int32_t | getNbOutputs () const noexcept=0 |
| | Get the number of outputs from the plugin. More...
|
| |
| virtual size_t | getWorkspaceSize (DynamicPluginTensorDesc const *inputs, int32_t nbInputs, DynamicPluginTensorDesc const *outputs, int32_t nbOutputs) const noexcept |
| | Find the workspace size required by the layer. More...
|
| |
| virtual int32_t | getValidTactics (int32_t *tactics, int32_t nbTactics) noexcept |
| | Query for any custom tactics that the plugin intends to use. More...
|
| |
| virtual int32_t | getNbTactics () noexcept |
| | Query for the number of custom tactics the plugin intends to use. More...
|
| |
| virtual char const * | getTimingCacheID () noexcept |
| | Called to query the suffix to use for the timing cache ID. May be called anytime after plugin creation. More...
|
| |
| virtual int32_t | getFormatCombinationLimit () noexcept |
| | Return the maximum number of format combinations that will be timed by TensorRT during the build phase. More...
|
| |
| virtual char const * | getMetadataString () noexcept |
| | Query for a string representing the configuration of the plugin. May be called anytime after plugin creation. More...
|
| |
| virtual APILanguage | getAPILanguage () const noexcept |
| | The language used to build the implementation of this Interface. More...
|
| |
| virtual | ~IVersionedInterface () noexcept=default |
| |
Configure the plugin.
configurePlugin() can be called multiple times in the build phase during creation of an engine by IBuilder.
configurePlugin() is called when a plugin is being prepared for profiling but not for any specific input size. This provides an opportunity for the plugin to make algorithmic choices on the basis of input and output formats, along with the bound of possible dimensions. The min, opt and max value of the DynamicPluginTensorDesc correspond to the kMIN, kOPT and kMAX value of the current profile that the plugin is being profiled for, with the desc.dims field corresponding to the dimensions of plugin specified at network creation. Wildcard dimensions may exist during this phase in the desc.dims field.
- Parameters
-
| in | The input tensors attributes that are used for configuration. |
| nbInputs | Number of input tensors. |
| out | The output tensors attributes that are used for configuration. |
| nbOutputs | Number of output tensors. |
- Returns
- 0 for success, else non-zero (which will cause engine termination, if invoked by TensorRT).
| virtual int32_t nvinfer1::v_1_0::IPluginV3OneBuild::getValidTactics |
( |
int32_t * |
tactics, |
|
|
int32_t |
nbTactics |
|
) |
| |
|
inlinevirtualnoexcept |
Query for any custom tactics that the plugin intends to use.
This method queries for the set of tactics T(f) supported by the plugin for the format combination f indicated by the immediately preceding call to configurePlugin(). It is guaranteed to be called after configurePlugin().
For each format combination provided through configurePlugin(), up to a maximum of getFormatCombinationLimit(), the plugin will be timed for each tactic advertised through this method for that format combination. i.e. The plugin will be timed ![$N = \sum_{i=0}^{i<getFormatCombinationLimit()} (T(f[i]))$](form_2.png)
times. If 
, the plugin may not be timed. In pseudocode, the timing protocol appears as the following:
counter = 0 for each supported format combination ++counter if counter > getFormatCombinationLimit() goto done configurePlugin(...) for each tactic in getValidTactics(...) time tactic done:
- Parameters
-
| tactics | Pre-allocated buffer to which the tactic values should be written |
| nbTactics | The number of tactics advertised through getNbTactics() |
- Note
- The provided tactic values must be unique and non-zero. The tactic value 0 is reserved for the default tactic attached to each format combination.
- Returns
- 0 for success, else non-zero (which will cause engine termination). The returned code will be reported through the error recorder.
| virtual bool nvinfer1::v_1_0::IPluginV3OneBuild::supportsFormatCombination |
( |
int32_t |
pos, |
|
|
DynamicPluginTensorDesc const * |
inOut, |
|
|
int32_t |
nbInputs, |
|
|
int32_t |
nbOutputs |
|
) |
| |
|
pure virtualnoexcept |
Return true if plugin supports the format and datatype for the input/output indexed by pos.
For this method inputs are numbered 0.. (nbInputs - 1) and outputs are numbered nbInputs.. (nbInputs + nbOutputs
- 1). Using this numbering, pos is an index into InOut, where 0 <= pos < nbInputs + nbOutputs - 1.
TensorRT invokes this method to ask if the input/output indexed by pos supports the format/datatype specified by inOut[pos].format and inOut[pos].type. The override should return true if that format/datatype at inOut[pos] are supported by the plugin. If support is conditional on other input/output formats/datatypes, the plugin can make its result conditional on the formats/datatypes in inOut[0.. pos - 1], which will be set to values that the plugin supports. The override should not inspect inOut[pos1.. nbInputs + nbOutputs - 1], which will have invalid values. In other words, the decision for pos must be based on inOut[0..pos] only.
Some examples:
- A definition for a plugin that supports only FP16 NCHW:
return inOut.format[pos] == TensorFormat::kLINEAR && inOut.type[pos] == DataType::kHALF;
- A definition for a plugin that supports only FP16 NCHW for its two inputs, and FP32 NCHW for its single output:
return inOut.format[pos] == TensorFormat::kLINEAR && (inOut.type[pos] == pos < 2 ? DataType::kHALF :
DataType::kFLOAT);
- A definition for a "polymorphic" plugin with two inputs and one output that supports any format or type, but the inputs and output must have the same format and type:
return pos == 0 || (inOut.format[pos] == inOut.format[0] && inOut.type[pos] == inOut.type[0]);
- Warning
- TensorRT will stop querying once it finds getFormatCombinationLimit() of combinations.
- See also
- getFormatCombinationLimit
Public Member Functions inherited from nvinfer1::IVersionedInterface