Detailed Description

Defines 2D-based template tracking.

Data Structures
struct	dwTemplateArray
	Pointer list to the CUDA table with type = DW_CUDA_TABLE_SIMPLE. More...

struct	dwTemplateTrackerParameters
	Configuration parameters for a dwTemplateTrackerIA. More...

Macros
#define	DW_MAX_TEMPLATE2D_SIZE 128
	defines the maximum template size More...

Typedefs
typedef struct dwTemplateTrackerObject const *	dwConstTemplateTrackerHandle_t
	Handle representing a const TemplateTracker tracker. More...

typedef struct dwTemplateTrackerObject *	dwTemplateTrackerHandle_t
	Handle representing a TemplateTracker tracker. More...

Enumerations
enum	dwTemplateTrackerAlgorithm { DW_TEMPLATE_TRACKER_ALGORITHM_IA = 0, DW_TEMPLATE_TRACKER_ALGORITHM_IC }
	Different versions of the template tracker Both can be found in paper: "Lucas-Kanade 20 Years On: A Unifying Framework" by SIMON BAKER AND IAIN MATTHEWS. More...

Functions
DW_API_PUBLIC dwStatus	dwTemplateArray_copyAsync (dwTemplateArray dstTemplateArray, const dwTemplateArray srcTemplateArray, cudaStream_t stream)
	Deep copy all contents from `srcTemplateArray` to `dstTemplateArray` More...

DW_API_PUBLIC dwStatus	dwTemplateArray_create (dwTemplateArray *templateArray, const uint32_t maxTemplateCount, const dwMemoryType memoryType, dwContextHandle_t context)
	Creates and initializes a template array. More...

DW_API_PUBLIC dwStatus	dwTemplateArray_destroy (dwTemplateArray templateArray)
	Destroys the template array and frees any memory created by dwTemplateArray_create(). More...

DW_API_PUBLIC dwStatus	dwTemplateArray_reset (dwTemplateArray *templateArray)
	Resets the template array. More...

DW_API_PUBLIC dwStatus	dwTemplateTracker_getCUDAStream (cudaStream_t *cudaStream, dwTemplateTrackerHandle_t obj)
	Gets the CUDA stream used. More...

DW_API_PUBLIC dwStatus	dwTemplateTracker_initDefaultParams (dwTemplateTrackerParameters *params)
	Initializes TemplateTracker parameters with default values. More...

DW_API_PUBLIC dwStatus	dwTemplateTracker_initialize (dwTemplateTrackerHandle_t obj, const dwTemplateTrackerParameters params, cudaStream_t stream, dwContextHandle_t context)
	Initialize the TemplateTracker module. More...

DW_API_PUBLIC dwStatus	dwTemplateTracker_release (dwTemplateTrackerHandle_t obj)
	Releases the TemplateTracker module. More...

DW_API_PUBLIC dwStatus	dwTemplateTracker_reset (dwTemplateTrackerHandle_t obj)
	Resets the TemplateTracker. More...

DW_API_PUBLIC dwStatus	dwTemplateTracker_setCUDAStream (cudaStream_t cudaStream, dwTemplateTrackerHandle_t obj)
	Sets the CUDA stream used. More...

DW_API_PUBLIC dwStatus	dwTemplateTracker_trackImage (dwTemplateArray templateArray, const dwImageCUDA currentImage, const dwImageCUDA *previousImage, dwTemplateTrackerHandle_t obj)
	Track the templates in currentImage . More...

DW_API_PUBLIC dwStatus	dwTemplateTracker_trackPyramid (dwTemplateArray templateArray, const dwPyramidImage currentPyramid, const dwPyramidImage *previousPyramid, dwTemplateTrackerHandle_t obj)
	Track the templates in currentPyramid . More...

Data Structure Documentation

◆ dwTemplateArray

struct dwTemplateArray

Data Fields
uint32_t *	ages	Age of each template. 1D array of size maxTemplates.
dwRectf *	bboxes	bounding box of each template. 1D array of size maxTemplates.
size_t	bytes	Bytes of raw data.
uint8_t *	data	Pointer to the raw data address.
uint32_t *	ids	Id of each template. 1D array of size maxTemplates
uint32_t	maxTemplates	Max number of templates in template array.
dwMemoryType	memoryType	Whether the template array is located on CPU or GPU.
uint32_t *	newToOldMap	New to old index map, 1D array of size maxTemplates. See more details in `dwTemplateTracker_trackImage`. and `dwTemplateTracker_trackPyramid`
float32_t *	scaleFactors	scaleFactor from frame N to N-1 of each template. 1D array of size maxTemplates.
dwFeature2DStatus *	statuses	Status of each template. 1D array of size maxTemplates.
uint32_t *	templateCount	Total number of templates. Single value.

◆ dwTemplateTrackerParameters

struct dwTemplateTrackerParameters

Data Fields
dwTemplateTrackerAlgorithm	algorithm	Tracking Algorithm defined by `dwTemplateTrackerAlgorithm`.
uint32_t	imageHeight	Height of the images that the tracker runs on.
uint32_t	imageWidth	Width of the images that the tracker runs on.
uint32_t	maxPyramidLevel	Max pyramid level to track.
float32_t	maxScaleChange	If scalingFactor between frame N to N-1 is outside range [1/maxScaleChange, maxScaleChange] tracking will be killed.
uint32_t	maxTemplateCount	Upper bound on number of templates handled.
uint32_t	numIterationsCoarse	for DW_TEMPLATE_TRACKER_ALGO_IA only Iteration number to apply the coarse KLT for robustness. Coarse tracking converges with smaller iteration number, but is not as accurate as fine tracking. If setting as 0, the algorithm will skip the coarse pass, use fine only iteration
uint32_t	numIterationsFine	Iteration number to apply the KLT tracker.
dwProcessorType	processorType	Processor type which determines on which processor the algorithm should be executed on. Supported options are: DW_PROCESSOR_TYPE_GPU, DW_PROCESSOR_TYPE_PVA_0, DW_PROCESSOR_TYPE_PVA_1. Note DW_PROCESSOR_TYPE_GPU supports both DW_TEMPLATE_TRACKER_ALGORITHM_IA and DW_TEMPLATE_TRACKER_ALGORITHM_IC DW_PROCESSOR_TYPE_PVA_0/1 supports only DW_TEMPLATE_TRACKER_ALGORITHM_IC
float32_t	thresholdKill	Killing threshold in [0, 1]. If ZNCC between 2 frames is less than this value, tracking will be killed. Smaller value means less possible to be killed. If thresholdKill = 0 or -1, the tracking will not be marked as failure until it's out of time or move out of boundary. If thresholdKill = 1, the tracking will always be killed.
float32_t	thresholdStop	for DW_TEMPLATE_TRACKER_ALGO_IC only Stop threshold in [-1, 1]. If ZNCC between 2 frames is larger than this value, tracking will have early stop even before it reaches the maximal number of allowed iterations. Larger value means less possible to be considered as converged track. If nccThresholdStop = 1, the tracking will always run the maximal number of allowed iterations. If nccThresholdStop = -1, the tracking will run one iteration.
float32_t	thresholdUpdate	Updating threshold in [0, 1]. If ZNCC between 2 frames is less than this value, tracking template will be updated. Otherwise only updates its location but keep the original size and image data unchanged. Larger value means updating the template more frequently. If thresholdUpdate = 1, template will be updated every frame If thresholdUpdate = 0, template won't be updated unless the size of stored template and new one differs too much.
float32_t	validHeight	Maximum valid template height, any templates with bbox.height > validHeight will be killed after tracking. validHeight = -1.f means there's no height limitation
float32_t	validWidth	Maximum valid template width, any templates with bbox.width > validWidth will be killed after tracking. validWidth = -1.f means there's no width limitation

Macro Definition Documentation

◆ DW_MAX_TEMPLATE2D_SIZE

#define DW_MAX_TEMPLATE2D_SIZE 128

defines the maximum template size

Template
It's a subimage defined by ROI (regions of interest) that is going to be tracked in the new frame.
Tracker workflow
*
1. Add new templates to the list. This module does not provide detector. App decides what new templates to add.
2. Do tracking.
Template updating strategy
The template is not updated every frame. When KLT iteration is finished and the tracker gets the new predicted template in the new frame, it will calculate the ZNCC (zero mean normalized cross-correlation) between the warped template and the predicted one, only when ZNCC is less than a predefined updateThreshold, the stored template and its location/size will be updated, elsewise the tracker will use the old template and its size as an estimation and moves its location to the new one. There is another killing threshold (smaller than updateThreshold), if ZNCC is even less than killing threshold, it will be considered that the tracked result differs so much to the stored template that the tracking is failed and the current template entry will be killed.

Template tracking is usually applied on video sequence, so the scale between frame N-1 and N should be smooth. maxScaleChange is a parameter to kill significant size changes. Once the ratio of old and new template size is greater than it, the current template entry will be killed.


Pseudo code:

if (ZNCC < killThreshold)
    tracking is lost, kill the current template
    return

if (N-1 to N scaler is not within [1/maxScaleChang, maxScaleChang])
    Too significant size change, kill the current template
    return

if (ZNCC < updateThreshold)
    template image <– new image
    template location <– new location
    template size <– new size
else
    if ((templateCenter - newCenter) > 1px)
        template location <– new location
    else
        template image <– new image
        template location <– new location
        template size <– new size

Definition at line 111 of file TemplateTracker.h.

Typedef Documentation

◆ dwConstTemplateTrackerHandle_t

typedef struct dwTemplateTrackerObject const* dwConstTemplateTrackerHandle_t

Handle representing a const TemplateTracker tracker.

Definition at line 117 of file TemplateTracker.h.

◆ dwTemplateTrackerHandle_t

typedef struct dwTemplateTrackerObject* dwTemplateTrackerHandle_t

Handle representing a TemplateTracker tracker.

Definition at line 114 of file TemplateTracker.h.

Enumeration Type Documentation

◆ dwTemplateTrackerAlgorithm

enum dwTemplateTrackerAlgorithm

Different versions of the template tracker Both can be found in paper: "Lucas-Kanade 20 Years On: A Unifying Framework" by SIMON BAKER AND IAIN MATTHEWS.

Enumerator
DW_TEMPLATE_TRACKER_ALGORITHM_IA	inverse additive KLT
DW_TEMPLATE_TRACKER_ALGORITHM_IC	inverse compositional KLT.

Definition at line 122 of file TemplateTracker.h.

Function Documentation

◆ dwTemplateArray_copyAsync()

DW_API_PUBLIC dwStatus dwTemplateArray_copyAsync	(	dwTemplateArray *	dstTemplateArray,
		const dwTemplateArray *	srcTemplateArray,
		cudaStream_t	stream
	)

Deep copy all contents from srcTemplateArray to dstTemplateArray

Parameters

[out]	dstTemplateArray	`dwTemplateArray` to copy to
[in]	srcTemplateArray	`dwTemplateArray` to copy from
[in]	stream	Working cuda stream

Returns: DW_INVALID_ARGUMENT if dstTemplateArray or srcTemplateArray is NULL.
if dstTemplateArray.bytes != srcTemplateArray.bytes.
A specific CUDA error in case of an underlying cuda failure.
DW_SUCCESS otherwise.

Note: If the copy function is run on a non-zero stream, it's asynchronous calling, need do stream sync or event sync to make sure the copy is done.

◆ dwTemplateArray_create()

DW_API_PUBLIC dwStatus dwTemplateArray_create	(	dwTemplateArray *	templateArray,
		const uint32_t	maxTemplateCount,
		const dwMemoryType	memoryType,
		dwContextHandle_t	context
	)

Creates and initializes a template array.

Parameters

[out]	templateArray	pointer to the dwTemplateArray is returned here.
[in]	maxTemplateCount	maximum number of templates that the template array can have.
[in]	memoryType	DW_MEMORY_TYPE_CUDA for CUDA array, DW_MEMORY_TYPE_CPU for CPU array, DW_MEMORY_TYPE_PINNED for pinned memory
[in]	context	handle to the context under which it is created.

Returns: DW_INVALID_ARGUMENT if template arry or context are NULL.
if maxTemplateCount is 0.
if memoryType is not listed in dwMemoryType.
DW_SUCCESS otherwise.

◆ dwTemplateArray_destroy()

DW_API_PUBLIC dwStatus dwTemplateArray_destroy ( dwTemplateArray templateArray )

Destroys the template array and frees any memory created by dwTemplateArray_create().

Parameters

[in] templateArray template array to be destroyed.

Returns: DW_INVALID_ARGUMENT if templateArray contains invalid pointers.
if templateArray.memoryType is not listed in dwMemoryType.
DW_SUCCESS otherwise.

◆ dwTemplateArray_reset()

DW_API_PUBLIC dwStatus dwTemplateArray_reset ( dwTemplateArray * templateArray )

Resets the template array.

Sets the template count back to zero.

Parameters

[in] templateArray template array to be reset.

Returns: DW_INVALID_ARGUMENT if templateArray.templateCount is NULL.
if templateArray.memoryType is not listed in dwMemoryType.
DW_SUCCESS otherwise.

◆ dwTemplateTracker_getCUDAStream()

DW_API_PUBLIC dwStatus dwTemplateTracker_getCUDAStream	(	cudaStream_t *	cudaStream,
		dwTemplateTrackerHandle_t	obj
	)

Gets the CUDA stream used.

Parameters

[out]	cudaStream	The CUDA stream used.
[in]	obj	A pointer to the TemplateTracker handle that is updated.

Returns: DW_INVALID_ARGUMENT if TemplateTracker handle or cudaStream are NULL.
DW_SUCCESS otherwise.

◆ dwTemplateTracker_initDefaultParams()

DW_API_PUBLIC dwStatus dwTemplateTracker_initDefaultParams ( dwTemplateTrackerParameters * params )

Initializes TemplateTracker parameters with default values.

Parameters

[out] params TemplateTracker parameters.

Returns: DW_INVALID_ARGUMENT if params is NULL.
DW_SUCCESS otherwise.

◆ dwTemplateTracker_initialize()

DW_API_PUBLIC dwStatus dwTemplateTracker_initialize	(	dwTemplateTrackerHandle_t *	obj,
		const dwTemplateTrackerParameters *	params,
		cudaStream_t	stream,
		dwContextHandle_t	context
	)

Initialize the TemplateTracker module.

Parameters

[out]	obj	A pointer to TemplateTracker handle that is initialized from parameters.
[in]	params	TemplateTracker parameters.
[in]	stream	Specifies the cuda stream to use
[in]	context	Specifies the handle to the context.

Returns: DW_INVALID_ARGUMENT if TemplateTracker handle, params or context are NULL, or the params has invalid values.
DW_SUCCESS otherwise.

◆ dwTemplateTracker_release()

DW_API_PUBLIC dwStatus dwTemplateTracker_release ( dwTemplateTrackerHandle_t obj )

Releases the TemplateTracker module.

Parameters

[in] obj The object handle to release.

Returns: DW_INVALID_ARGUMENT if TemplateTracker handle is NULL.
DW_SUCCESS otherwise.

Note: This method renders the handle unusable.

◆ dwTemplateTracker_reset()

DW_API_PUBLIC dwStatus dwTemplateTracker_reset ( dwTemplateTrackerHandle_t obj )

Resets the TemplateTracker.

Parameters

[in] obj Handle to reset.

Returns: DW_INVALID_ARGUMENT if TemplateTracker handle is NULL.
DW_SUCCESS otherwise.

◆ dwTemplateTracker_setCUDAStream()

DW_API_PUBLIC dwStatus dwTemplateTracker_setCUDAStream	(	cudaStream_t	cudaStream,
		dwTemplateTrackerHandle_t	obj
	)

Sets the CUDA stream used.

Parameters

[in]	cudaStream	The CUDA stream used.
[in]	obj	A pointer to the TemplateTracker handle that is updated.

Returns: DW_INVALID_ARGUMENT if TemplateTracker handle is NULL.
DW_SUCCESS otherwise.

◆ dwTemplateTracker_trackImage()

DW_API_PUBLIC dwStatus dwTemplateTracker_trackImage	(	dwTemplateArray *	templateArray,
		const dwImageCUDA *	currentImage,
		const dwImageCUDA *	previousImage,
		dwTemplateTrackerHandle_t	obj
	)

Track the templates in currentImage
.

The computation takes place asynchronously on the device (GPU).

Parameters

[in,out]	templateArray	template to be tracked, in-place tracking
[in]	currentImage	Current image data to track to.
[in]	previousImage	Previous image data to track from.
[in]	obj	Specifies the TemplateTracker handle.

Returns: DW_INVALID_ARGUMENT if any parameter is NULL.
DW_SUCCESS otherwise.

Note: Output templateArray array contains a list of compacted templates, dwTemplateArray::newToOldMap which maps new->old index: newToOldMap[newIdx] == oldIdx. newToOldMap[i] = j means the i-th template in prediction is the j-th item in input array. i.e. predictedTemplate[i] = oldTemplateToTrack[newToOldMap[i]], below is an example: data: [1 2 3 4 5 6 7 8 9] tracked/untracked: [O O X X O X O X O] compacted output (overwrite 3 and 4 by 9 and 7): data: [1 2 9 7 5] newToOldMap: [0 1 8 6 4]

◆ dwTemplateTracker_trackPyramid()

DW_API_PUBLIC dwStatus dwTemplateTracker_trackPyramid	(	dwTemplateArray *	templateArray,
		const dwPyramidImage *	currentPyramid,
		const dwPyramidImage *	previousPyramid,
		dwTemplateTrackerHandle_t	obj
	)

Track the templates in currentPyramid
.

The computation takes place asynchronously on the device (GPU).

Parameters

[in,out]	templateArray	template to be tracked, in-place tracking
[in]	currentPyramid	Current pyramid data to track to.
[in]	previousPyramid	Previous pyramid data to track from.
[in]	obj	Specifies the TemplateTracker handle.

Returns: DW_INVALID_ARGUMENT if any parameter is NULL.
DW_SUCCESS otherwise.

Note: Output templateArray array contains a list of compacted templates, dwTemplateArray::newToOldMap which maps new->old index: newToOldMap[newIdx] == oldIdx. newToOldMap[i] = j means the i-th template in prediction is the j-th item in input array. i.e. predictedTemplate[i] = oldTemplateToTrack[newToOldMap[i]], below is an example: data: [1 2 3 4 5 6 7 8 9] tracked/untracked: [O O X X O X O X O] compacted output (overwrite 3 and 4 by 9 and 7): data: [1 2 9 7 5] newToOldMap: [0 1 8 6 4]

Detailed Description

Data Structures

Macros

Typedefs

Enumerations

Functions

Data Structure Documentation

◆ dwTemplateArray

◆ dwTemplateTrackerParameters

Macro Definition Documentation

◆ DW_MAX_TEMPLATE2D_SIZE

Typedef Documentation

◆ dwConstTemplateTrackerHandle_t

◆ dwTemplateTrackerHandle_t

Enumeration Type Documentation

◆ dwTemplateTrackerAlgorithm

Function Documentation

◆ dwTemplateArray_copyAsync()

◆ dwTemplateArray_create()

◆ dwTemplateArray_destroy()

◆ dwTemplateArray_reset()

◆ dwTemplateTracker_getCUDAStream()

◆ dwTemplateTracker_initDefaultParams()

◆ dwTemplateTracker_initialize()

◆ dwTemplateTracker_release()

◆ dwTemplateTracker_reset()

◆ dwTemplateTracker_setCUDAStream()

◆ dwTemplateTracker_trackImage()

◆ dwTemplateTracker_trackPyramid()