The general TensorRT workflow consists of 3 steps:
tensorrt.INetworkDefinitioneither with a parser or by using the TensorRT Network API (see
tensorrt.INetworkDefinitionfor more details). The
tensorrt.Buildercan be used to generate an empty
tensorrt.Builderto build a
tensorrt.ICudaEngineusing the populated
tensorrt.ICudaEngineand use it to perform optimized inference.
Most other TensorRT classes use a logger to report errors, warnings and informative messages. TensorRT provides a basic
tensorrt.Logger implementation, but you can write your own implementation by deriving from
tensorrt.ILogger for more advanced functionality.
Parsers are used to populate a
tensorrt.INetworkDefinition from a model trained in a Deep Learning framework.
tensorrt.INetworkDefinition represents a computational graph. In order to populate the network, TensorRT provides a suite of parsers for a variety of Deep Learning frameworks. It is also possible to populate the network manually using the Network API.
tensorrt.Builder is used to build a
tensorrt.ICudaEngine . In order to do so, it must be provided a populated
Engine and Context
tensorrt.ICudaEngine is the output of the TensorRT optimizer. It is used to generate a
tensorrt.IExecutionContext that can perform inference.