Flags used to control TensorRT’s behavior when creating executable temporary files.

On some platforms the TensorRT runtime may need to create files in a temporary directory or use platform-specific APIs to create files in-memory to load temporary DLLs that implement runtime code. These flags allow the application to explicitly control TensorRT’s use of these files. This will preclude the use of certain TensorRT APIs for deserializing and loading lean runtimes.

These should be treated as bit offsets, e.g. in order to allow in-memory files for a given IRuntime:

runtime.tempfile_control_flags |= (1 << int(TempfileControlFlag.ALLOW_IN_MEMORY_FILES))


ALLOW_IN_MEMORY_FILES : Allow creating and loading files in-memory (or unnamed files).

ALLOW_TEMPORARY_FILES : Allow creating and loading named files in a temporary directory on the filesystem.

class tensorrt.Runtime(self: tensorrt.tensorrt.Runtime, logger: tensorrt.tensorrt.ILogger) None

Allows a serialized ICudaEngine to be deserialized.

  • error_recorderIErrorRecorder Application-implemented error reporting interface for TensorRT objects.

  • gpu_allocatorIGpuAllocator The GPU allocator to be used by the Runtime . All GPU memory acquired will use this allocator. If set to None, the default allocator will be used (Default: cudaMalloc/cudaFree).

  • DLA_coreint The DLA core that the engine executes on. Must be between 0 and N-1 where N is the number of available DLA cores.

  • num_DLA_coresint The number of DLA engines available to this builder.

  • loggerILogger The logger provided when creating the refitter.

  • max_threadsint The maximum thread that can be used by the Runtime.

  • temporary_directorystr The temporary directory to use when loading executable code for engines. If set to None (the default), TensorRT will attempt to find a suitable directory for use using platform-specific heuristics: - On UNIX/Linux platforms, TensorRT will first try the TMPDIR environment variable, then fall back to /tmp - On Windows, TensorRT will try the TEMP environment variable.

  • tempfile_control_flagsint Flags which control whether TensorRT is allowed to create in-memory or temporary files. See TempfileControlFlag for details.

  • engine_host_code_allowedbool Whether this runtime is allowed to deserialize engines that contain host executable code (Default: False).


logger – The logger to use.

__del__(self: tensorrt.tensorrt.Runtime) None
__exit__(exc_type, exc_value, traceback)

Context managers are deprecated and have no effect. Objects are automatically freed when the reference count reaches 0.

__init__(self: tensorrt.tensorrt.Runtime, logger: tensorrt.tensorrt.ILogger) None

logger – The logger to use.

deserialize_cuda_engine(*args, **kwargs)

Overloaded function.

  1. deserialize_cuda_engine(self: tensorrt.tensorrt.Runtime, serialized_engine: buffer) -> tensorrt.tensorrt.ICudaEngine

    Deserialize an ICudaEngine from host memory.

    arg serialized_engine

    The buffer that holds the serialized ICudaEngine.


    The ICudaEngine, or None if it could not be deserialized.

  2. deserialize_cuda_engine(self: tensorrt.tensorrt.Runtime, stream_reader: tensorrt.tensorrt.IStreamReader) -> tensorrt.tensorrt.ICudaEngine

    Deserialize an ICudaEngine from a stream reader.

    arg stream_reader

    The PyStreamReader that will read the serialized ICudaEngine. This enables deserialization from a file directly.


    The ICudaEngine, or None if it could not be deserialized.

get_plugin_registry(self: tensorrt.tensorrt.Runtime) tensorrt.tensorrt.IPluginRegistry

Get the local plugin registry that can be used by the runtime.


The local plugin registry that can be used by the runtime.

load_runtime(self: tensorrt.tensorrt.Runtime, path: str) tensorrt.tensorrt.Runtime

Load IRuntime from the file.

This method loads a runtime library from a shared library file. The runtime can then be used to execute a plan file built with BuilderFlag.VERSION_COMPATIBLE and BuilderFlag.EXCLUDE_LEAN_RUNTIME both set and built with the same version of TensorRT as the loaded runtime library.


path – Path to the runtime lean library.


The IRuntime, or None if it could not be loaded.