TensorRT

Overview

• Getting Started with TensorRT
  • Installation
  • Samples
  • Operator Documentation
  • Installing cuda-python
• Core Concepts
  • TensorRT Workflow
  • Classes Overview
    • Logger
    • Parsers
    • Network
    • Builder
    • Engine and Context
• Writing custom operators with TensorRT Python plugins
  • Composition of a plugin
  • Example: Circular padding plugin
    • Picking the most performant plugin configuration: Autotuning
    • Adding the plugin to a TensorRT network
      • Adding the plugin using TRT Python APIs
      • Loading an ONNX model with the custom operator
  • Advanced usage
    • Example: Operators with data-dependent output shapes - Non-zero
    • Example: Implementing in-place custom ops with I/O aliasing
    • Example: Plugins with multiple backends: Using custom tactics
  • FAQ
  • Limitations
  • Additional resources

TensorRT Python API Reference

• Foundational Types
  • DataType
  • Weights
  • Dims
    • Volume
    • Dims
    • Dims2
    • DimsHW
    • Dims3
    • Dims4
  • IHostMemory
• Core
  • Logger
  • Profiler
  • IOptimizationProfile
  • IBuilderConfig
  • Builder
    • NetworkDefinitionCreationFlag
    • Builder
  • Debug Listener
    • IDebugListener
  • ICudaEngine
  • IExecutionContext
  • Runtime
  • Refitter
  • IErrorRecorder
  • ITimingCache
  • GPU Allocator
    • AllocatorFlag
    • IGpuAllocator
  • EngineInspector
  • ISerializationConfig
• Network
  • INetworkDefinition
  • Layer Base Classes
    • ITensor
    • ILayer
  • Layers
    • PaddingMode
    • ICastLayer
    • IConvolutionLayer
    • IGridSampleLayer
    • IActivationLayer
    • IPoolingLayer
    • ILRNLayer
    • IScaleLayer
    • ISoftMaxLayer
    • IConcatenationLayer
    • IDeconvolutionLayer
    • IElementWiseLayer
    • IGatherLayer
    • IPluginV2Layer
    • IPluginV3Layer
    • IUnaryLayer
    • IReduceLayer
    • IPaddingLayer
    • IParametricReLULayer
    • ISelectLayer
    • IShuffleLayer
    • ISliceLayer
    • IShapeLayer
    • ITopKLayer
    • IMatrixMultiplyLayer
    • IRaggedSoftMaxLayer
    • IIdentityLayer
    • IConstantLayer
    • IResizeLayer
    • ILoop
      • ILoopBoundaryLayer
        • ITripLimitLayer
        • IRecurrenceLayer
        • IIteratorLayer
        • ILoopOutputLayer
    • IFillLayer
    • IQuantizeLayer
    • IDequantizeLayer
    • IScatterLayer
    • IIfConditional
    • IConditionLayer
    • IIfConditionalOutputLayer
    • IIfConditionalInputLayer
    • IEinsumLayer
    • IAssertionLayer
    • IOneHotLayer
    • INonZeroLayer
    • INMSLayer
    • IReverseSequenceLayer
    • INormalizationLayer
    • ISqueezeLayer
    • IUnsqueezeLayer
• Plugin
  • IPluginCreator
  • IPluginRegistry
  • IPluginV2DynamicExt
  • IPluginV3
• tensorrt.plugin
  • tensorrt.plugin.register
  • tensorrt.plugin.impl
  • tensorrt.plugin.autotune
    • AutoTuneCombination
  • Shape
    • Shape
    • ShapeExpr
  • Tensor
    • TensorDesc
    • Tensor
  • Functions
• Int8
  • IInt8Calibrator
  • IInt8LegacyCalibrator
  • IInt8EntropyCalibrator
  • IInt8EntropyCalibrator2
  • IInt8MinMaxCalibrator
• Algorithm Selector
• Onnx Parser