Runtime
- class tensorrt.Runtime(self: tensorrt.tensorrt.Runtime, logger: tensorrt.tensorrt.ILogger) None
Allows a serialized
ICudaEngine
to be deserialized.- Variables
error_recorder –
IErrorRecorder
Application-implemented error reporting interface for TensorRT objects.gpu_allocator –
IGpuAllocator
The GPU allocator to be used by theRuntime
. All GPU memory acquired will use this allocator. If set to None, the default allocator will be used (Default: cudaMalloc/cudaFree).DLA_core –
int
The DLA core that the engine executes on. Must be between 0 and N-1 where N is the number of available DLA cores.num_DLA_cores –
int
The number of DLA engines available to this builder.logger –
ILogger
The logger provided when creating the refitter.max_threads –
int
The maximum thread that can be used by theRuntime
.temporary_directory –
str
The temporary directory to use when loading executable code for engines. If set to None (the default), TensorRT will attempt to find a suitable directory for use using platform-specific heuristics: - On UNIX/Linux platforms, TensorRT will first try the TMPDIR environment variable, then fall back to /tmp - On Windows, TensorRT will try the TEMP environment variable.tempfile_control_flags –
int
Flags which control whether TensorRT is allowed to create in-memory or temporary files. SeeTempfileControlFlag
for details.engine_host_code_allowed –
bool
Whether this runtime is allowed to deserialize engines that contain host executable code (Default: False).
- Parameters
logger – The logger to use.
- __del__(self: tensorrt.tensorrt.Runtime) None
- __exit__(exc_type, exc_value, traceback)
Context managers are deprecated and have no effect. Objects are automatically freed when the reference count reaches 0.
- __init__(self: tensorrt.tensorrt.Runtime, logger: tensorrt.tensorrt.ILogger) None
- Parameters
logger – The logger to use.
- deserialize_cuda_engine(self: tensorrt.tensorrt.Runtime, serialized_engine: buffer) tensorrt.tensorrt.ICudaEngine
Deserialize an
ICudaEngine
from a stream.- Parameters
serialized_engine – The
buffer
that holds the serializedICudaEngine
.- Returns
The
ICudaEngine
, or None if it could not be deserialized.
- get_plugin_registry(self: tensorrt.tensorrt.Runtime) tensorrt.tensorrt.IPluginRegistry
Get the local plugin registry that can be used by the runtime.
- Returns
The local plugin registry that can be used by the runtime.
- load_runtime(self: tensorrt.tensorrt.Runtime, path: str) tensorrt.tensorrt.Runtime
Load IRuntime from the file.
This method loads a runtime library from a shared library file. The runtime can then be used to execute a plan file built with BuilderFlag.VERSION_COMPATIBLE and BuilderFlag.EXCLUDE_LEAN_RUNTIME both set and built with the same version of TensorRT as the loaded runtime library.
- Variables
path – Path to the runtime lean library.
- Returns
The
IRuntime
, or None if it could not be loaded.