Flags used to control TensorRT’s behavior when creating executable temporary files.
On some platforms the TensorRT runtime may need to create files in a temporary directory or use platform-specific APIs to create files in-memory to load temporary DLLs that implement runtime code. These flags allow the application to explicitly control TensorRT’s use of these files. This will preclude the use of certain TensorRT APIs for deserializing and loading lean runtimes.
These should be treated as bit offsets, e.g. in order to allow in-memory files for a given
runtime.tempfile_control_flags |= (1 << int(TempfileControlFlag.ALLOW_IN_MEMORY_FILES))
ALLOW_IN_MEMORY_FILES : Allow creating and loading files in-memory (or unnamed files).
ALLOW_TEMPORARY_FILES : Allow creating and loading named files in a temporary directory on the filesystem.
- class tensorrt.Runtime(self: tensorrt.tensorrt.Runtime, logger: tensorrt.tensorrt.ILogger) None
Allows a serialized
ICudaEngineto be deserialized.
IErrorRecorderApplication-implemented error reporting interface for TensorRT objects.
IGpuAllocatorThe GPU allocator to be used by the
Runtime. All GPU memory acquired will use this allocator. If set to None, the default allocator will be used (Default: cudaMalloc/cudaFree).
intThe DLA core that the engine executes on. Must be between 0 and N-1 where N is the number of available DLA cores.
intThe number of DLA engines available to this builder.
ILoggerThe logger provided when creating the refitter.
intThe maximum thread that can be used by the
strThe temporary directory to use when loading executable code for engines. If set to None (the default), TensorRT will attempt to find a suitable directory for use using platform-specific heuristics: - On UNIX/Linux platforms, TensorRT will first try the TMPDIR environment variable, then fall back to /tmp - On Windows, TensorRT will try the TEMP environment variable.
intFlags which control whether TensorRT is allowed to create in-memory or temporary files. See
boolWhether this runtime is allowed to deserialize engines that contain host executable code (Default: False).
logger – The logger to use.
- __exit__(exc_type, exc_value, traceback)
Context managers are deprecated and have no effect. Objects are automatically freed when the reference count reaches 0.
- __init__(self: tensorrt.tensorrt.Runtime, logger: tensorrt.tensorrt.ILogger) None
logger – The logger to use.
- deserialize_cuda_engine(self: tensorrt.tensorrt.Runtime, serialized_engine: buffer) tensorrt.tensorrt.ICudaEngine
ICudaEnginefrom a stream.
- get_plugin_registry(self: tensorrt.tensorrt.Runtime) tensorrt.tensorrt.IPluginRegistry
Get the local plugin registry that can be used by the runtime.
The local plugin registry that can be used by the runtime.
- load_runtime(self: tensorrt.tensorrt.Runtime, path: str) tensorrt.tensorrt.Runtime
Load IRuntime from the file.
This method loads a runtime library from a shared library file. The runtime can then be used to execute a plan file built with BuilderFlag.VERSION_COMPATIBLE and BuilderFlag.EXCLUDE_LEAN_RUNTIME both set and built with the same version of TensorRT as the loaded runtime library.
path – Path to the runtime lean library.
IRuntime, or None if it could not be loaded.