ITimingCache

class tensorrt.ITimingCache

Class to handle tactic timing info collected from builder.

combine(self: tensorrt.tensorrt.ITimingCache, input_cache: tensorrt.tensorrt.ITimingCache, ignore_mismatch: bool) → bool

Combine input timing cache into local instance.

Append entries in input cache to local cache. Conflicting entries will be skipped. The input cache must be generated by a TensorRT build of exact same version, otherwise combine will be skipped and return false. bool(ignore_mismatch) == True if combining a timing cache created from a different device.

Parameters:

• input_cache – The input timing cache

• ignore_mismatch – Whether or not to allow cache verification header mismatch

Returns:

A bool indicating whether the combine operation is done successfully.

query(self: tensorrt.tensorrt.ITimingCache, key: tensorrt.tensorrt.TimingCacheKey) → tensorrt.tensorrt.TimingCacheValue

Query value in a cache entry.

If the key exists, write the value out, otherwise return an invalid value.

Parameters:

key – The query key.

Cache cache:

value if the key exists, otherwise an invalid value.

:returns A TimingCacheValue object.

queryKeys(self: tensorrt.tensorrt.ITimingCache) → List[tensorrt.tensorrt.TimingCacheKey]

Query cache keys from Timing Cache.

If an error occurs, a RuntimeError will be raised.

:returns A list containing the cache keys.

reset(self: tensorrt.tensorrt.ITimingCache) → bool

Empty the timing cache

Returns:

A bool indicating whether the reset operation is done successfully.

serialize(self: tensorrt.tensorrt.ITimingCache) → tensorrt.tensorrt.IHostMemory

Serialize a timing cache to a IHostMemory object.

Returns:

An IHostMemory object that contains a serialized timing cache.

update(self: tensorrt.tensorrt.ITimingCache, key: tensorrt.tensorrt.TimingCacheKey, value: tensorrt.tensorrt.TimingCacheValue) → bool

Update values in a cache entry.

Update the value of the given cache key. If the key does not exist, return False. If the key exists and the new tactic timing is NaN, delete the cache entry and return True. If tactic timing is not NaN and the new value is valid, override the cache value and return True. False is returned when the new value is invalid. If this layer cannot use the new tactic, build errors will be reported when building the next engine.

Parameters:

• key – The key to the entry to be updated.

• value – New cache value.

:returns True if update succeeds, otherwise False.