nvinfer1::ITimingCache Class Reference

Class to handle tactic timing info collected from builder. More...

#include <NvInfer.h>

Inheritance diagram for nvinfer1::ITimingCache:

Public Member Functions
virtual	~ITimingCache () noexcept=default
nvinfer1::IHostMemory *	serialize () const noexcept
	Serialize a timing cache to IHostMemory object. More...
bool	combine (ITimingCache const &inputCache, bool ignoreMismatch) noexcept
	Combine input timing cache into local instance. More...
bool	reset () noexcept
	Empty the timing cache. More...
int64_t	queryKeys (TimingCacheKey *keyBuffer, int64_t capacity) const noexcept
	Query cache keys from Timing Cache. More...
TimingCacheValue	query (TimingCacheKey const &key) const noexcept
	Query value in a cache entry. More...
bool	update (TimingCacheKey const &key, TimingCacheValue const &value) noexcept
	Update values in a cache entry. More...

Protected Attributes
apiv::VTimingCache *	mImpl

Additional Inherited Members
Protected Member Functions inherited from nvinfer1::INoCopy
	INoCopy ()=default
virtual	~INoCopy ()=default
	INoCopy (INoCopy const &other)=delete
INoCopy &	operator= (INoCopy const &other)=delete
	INoCopy (INoCopy &&other)=delete
INoCopy &	operator= (INoCopy &&other)=delete

Detailed Description

Class to handle tactic timing info collected from builder.

The timing cache is created or initialized by IBuilderConfig. It can be shared across builder instances to reduce the builder wallclock time.

Warning

It is a known issue that the same timing cache may not guarantee stable engine build reproducibility in all cases.

See also

IBuilderConfig

Constructor & Destructor Documentation

~ITimingCache()

virtual nvinfer1::ITimingCache::~ITimingCache ( )

virtualdefaultnoexcept

Member Function Documentation

combine()

bool nvinfer1::ITimingCache::combine ( ITimingCache const &  inputCache, bool  ignoreMismatch  )

inlinenoexcept

Combine input timing cache into local instance.

This function allows combining entries in the input timing cache to local cache object.

Parameters

inputCache	The input timing cache.
ignoreMismatch	Whether or not to allow cache verification header mismatch.

Returns

True if combined successfully, false otherwise.

Append entries in input cache to local cache. Conflicting entries will be skipped The input cache must be generated by a TensorRT build of exact same version, otherwise combine will be skipped and return false. ignoreMismatch must be set to true if combining a timing cache created from a different device.

Warning

Combining caches generated from devices with different device properties may lead to functional/performance bugs!

query()

TimingCacheValue nvinfer1::ITimingCache::query ( TimingCacheKey const &  key ) const

inlinenoexcept

Query value in a cache entry.

The function queries the value in a specific cache entry.

Parameters

key	The query key.

Returns

Cache value if the key exists, otherwise an invalid value.

Query the value of the given cache key. If the key exists, write the value out, otherwise return an invalid value.

queryKeys()

int64_t nvinfer1::ITimingCache::queryKeys ( TimingCacheKey *  keyBuffer, int64_t  capacity  ) const

inlinenoexcept

Query cache keys from Timing Cache.

This function queries the entry count and writes the keys out.

Parameters

keyBuffer	The buffer to store keys.
capacity	The capacity of the buffer.

Returns

The count of entries in the cache and fill keys if keyBuffer is non-null. If an error occurrs, -1 will be returned.

Query the count of entries in the cache and write out cache keys if keyBuffer is provided. Any key entries exceeding the capacity of the keyBuffer will not be copied.

reset()

bool nvinfer1::ITimingCache::reset ( )

inlinenoexcept

Empty the timing cache.

Returns

True if reset successfully, false otherwise.

serialize()

nvinfer1::IHostMemory * nvinfer1::ITimingCache::serialize ( ) const

inlinenoexcept

Serialize a timing cache to IHostMemory object.

This function allows serialization of current timing cache.

Returns

A pointer to a IHostMemory object that contains a serialized timing cache.

See also

IHostMemory

update()

bool nvinfer1::ITimingCache::update ( TimingCacheKey const &  key, TimingCacheValue const &  value  )

inlinenoexcept

Update values in a cache entry.

The function updates the value in a specific cache entry.

Parameters

key	The key to the entry to be updated.
value	New cache value.

Returns

True if update succeeds, otherwise false.

Update the value of the given cache key. If the key does not exist, return false. If the key exists and the new tactic timing is NaN, delete the cache entry and return true. If tactic timing is not NaN and the new value is valid, override the cache value and return true. False is returned when the new value is invalid. If this layer cannot use the new tactic, build errors will be reported when building the next engine.

Member Data Documentation

mImpl

apiv::VTimingCache* nvinfer1::ITimingCache::mImpl

protected

---

The documentation for this class was generated from the following file:

• NvInfer.h