DCF Tracker

Tracks object bounding boxes on a sequence of frames using the Discriminative Correlation Filter (DCF) algorithm. More...

Data Structures
struct	VPIDCFTrackerCreationParams
	Creation parameters of DCF Tracker. More...
struct	VPIDCFTrackerParams
	Structure that defines the parameters for vpiCreateDCFTracker. More...
struct	VPIDCFTrackedBoundingBox
	Stores information about an object tracked by DCF Tracker. More...

Enumerations
enum	VPIDCFTrackerCreationFlag
	Flags to customize DCF Tracker algorithm. More...

Functions
VPIStatus	vpiInitDCFTrackerCreationParams (VPIDCFTrackerCreationParams *params)
	Initialize VPIDCFTrackerCreationParams with default values. More...
VPIStatus	vpiCreateDCFTracker (uint64_t backends, int32_t maxNumSequences, int32_t maxNumObjects, const VPIDCFTrackerCreationParams *params, VPIPayload *payload)
	Creates payload for DCF Tracker. More...
VPIStatus	vpiInitDCFTrackerParams (VPIDCFTrackerParams *params)
	Initialize VPIDCFTrackerParams with default values. More...
VPIStatus	vpiSubmitDCFTrackerLocalizeBatch (VPIStream stream, uint64_t backend, VPIPayload payload, const int32_t *enabledSequences, int32_t numSequences, VPIImage featureMaskingWindow, VPIImage inPatches, VPIArray inObjects, VPIArray outObjects, VPIImage outCorrelationResponses, VPIArray outMaxCorrelationResponses, const VPIDCFTrackerParams *params)
	Localizes each tracked object in the input image patches using the Discriminative Correlation Filter method. More...
VPIStatus	vpiSubmitDCFTrackerUpdateBatch (VPIStream stream, uint64_t backend, VPIPayload payload, const int32_t *enabledSequences, int32_t numSequences, VPIImage featureMaskingWindow, VPIImage modelMaskingWindow, VPIImage inPatches, VPIArray trackedObjects, const VPIDCFTrackerParams *params)
	Update internal object tracking information based on its state and its corresponding input image patch. More...
VPIStatus	vpiDCFTrackerGetChannelWeights (VPIPayload payload, VPIArray *channelWeights, int32_t *numFeatureChannels)
	Returns the array with channel weight information for each tracked object. More...

Detailed Description

Tracks object bounding boxes on a sequence of frames using the Discriminative Correlation Filter (DCF) algorithm.

The processing flow allows for external customization of the tracking algorithm, including tracking termination criteria and external refinement of estimated of the tracked object's bounding box.

The high-level pseudo-code of the processing flow is as follows:

 VPIDCFTrackerCreationParams creatParams;
 vpiInitDCFTrackerCreationParams(&creatParams);
 // customize creatParams as needed
 
 VPIPayload dcfPayload;
 vpiCreateDCFTracker(..., NUM_SEQUENCES, MAX_OBJECTS, &creatParams, &dcfPayload);
 
 VPIPayload cropPayload;
 vpiCreateCropScaler(..., , NUM_SEQUENCES, MAX_OBJECTS, &cropPayload);
 
 // Optionally user can retrieve the internal array that stores the channel weights and
 // the maximum correlation response for each tracked object. These can be used, together
 // with the correlation map, to decide whether tracking was lost of not.
 VPIArray channelWeights;
 int32_t numFeatureChannels
 vpiDCFTrackerGetChannelWeights(dcfPayload, &channelWeights, &numFeatureChannels);
 
 // Array storing maximum correlation responses per object (optional)
 VPIArray outMaxCorrelationResponses = // ...
 
 // Stores the current (first) input frame per input sequence
 VPIImage frames[NUM_SEQUENCES] = // ...
 // Set element to 1 if objects in given sequence must be tracked, otherwise
 // 0 to ignore them.
 int32_t enabledSequences[NUM_SEQUENCES] = // ...
 
 // Array of objects detected in input frames. It must be populated with
 // objects detected in all input frame sequence
 // The state of the objects must be \ref VPI_TRACKING_STATE_NEW.
 VPIArray inObjects = // ...
 
 // Array of objects with estimated bounding boxes.
 // Initially it's an empty array with same capacity and format as inObjects.
 VPIArray outObjects = // ...
 
 // Image that stores the image patches of all objects.
 // Dimensions must be:
 //    width:  createParams.featurePatchSize*createParams.hogCellSize
 //    height: width * (inObjects' capacity)
 VPIImage objPatches = // ...
 
 VPIDCFTrackerParams params;
 vpiInitDCFTrackerParams(&params);
 // customize params as needed
 
 // Extract from the first frame the image patches of each object given its current bounding box.
 vpiSubmitCropScalerBatch(stream, cropPayload, frames, NUM_SEQUENCES,
                          inObjects, objPatches);
 
 // Update internal state with the new objects detected in first frame.
 vpiSubmitDCFTrackerUpdateBatch(stream, dcfPayload, enabledSequences, NUM_SEQUENCES,
                                NULL, objPatches, inObjects, &params);
 while(HasMoreFrames())
 {
     // Load next set of frames from the input sequences.
     LoadNextFramesFromSequence(frames, NUM_SEQUENCES);
 
     for(int i=0; i<NUM_SEQUENCES; ++i) enabledSequences[i] = frames[i] != NULL;
 
     // Extract from the current frame the image patches of each object given
     // its current bounding box.
     vpiSubmitCropScalerBatch(stream, cropPayload, frames, NUM_SEQUENCES,
                              NULL, inObjects, objPatches);
 
     // Localize the tracked objects in the new image patches.
     vpiSubmitDCFTrackerLocalizeBatch(stream, dcfPayload, enabledSequences, NUM_SEQUENCES,
                                      NULL, objPatches, inObjects, outObjects, outCorrelationResponses,
                                      outMaxCorrelationResponses, &params);
     // object's bounding box will reflect its position in input frame(s)
 
     // By using 'outMaxCorrelationResponses', 'outCorrelationResponses' and 'channelWeights',
     // the following processing can be done at this point:
     // 1. Object's bounding boxes can be refined by some external method.
     // 2. Object's state can be updated, e.g. whether tracking was lost or not. If tracking
     //    was lost, object's state must be set to \ref VPI_TRACKING_STATE_LOST.
     // 3. New objects can be added to outObjects, their state must be \ref VPI_TRACKING_STATE_NEW.
     //    Objects in the array whose state is \ref VPI_TRACKING_STATE_LOST can be reused for new objects.
 
     // Extract from the current frame the image patches of each object given its refined bounding box.
     vpiSubmitCropScalerBatch(stream, cropPayload, frames, NUM_SEQUENCES, inObjects, objPatches);
 
     // Update tracking information
     vpiSubmitDCFTrackerUpdateBatch(stream, dcfPayload, enabledSequences, NUM_SEQUENCES,
                                    NULL, objPatches, outObjects, &params);
 
     // Do a ping-pong with the in and out objects.
     // Output object array will be the input in next iteration, and current input will be the output.
     std::swap(inObjects, outObjects);
}

---

Data Structure Documentation

VPIDCFTrackerCreationParams

struct VPIDCFTrackerCreationParams

Creation parameters of DCF Tracker.

Definition at line 191 of file DCFTracker.h.

Collaboration diagram for VPIDCFTrackerCreationParams:

Data Fields
uint32_t	flags	Flags to customize DCF Tracker algorithm. Must be one or more of the VPIDCFTrackerCreationFlag enums, or-ed together. Must select either colornames or HOG or both features. Must be both colonames and hog for PVA backend.
int32_t	featurePatchSize	Size of an object feature patch. The patch image is always square, thus the value passed correspond to one dimension. For VPI_BACKEND_CUDA, the valid sizes are: 12, 18, 24, 30, 32, 36. For VPI_BACKEND_PVA, the valid sizes are: 24.
int32_t	hogCellSize	Cell size for features from Histogram of Oriented Gradients. Must be between 2 and 32, inclusive. Must be a power of two.
float	featureFocusVertOffsetFactor	Offset for the center of the Hanning window relative to the patch height. Must be between -0.5 to 0.5, inclusive
float	gaussianSigma	Standard deviation for gaussian for desired response. Must be > 0.0.

VPIDCFTrackerParams

struct VPIDCFTrackerParams

Structure that defines the parameters for vpiCreateDCFTracker.

Definition at line 277 of file DCFTracker.h.

Collaboration diagram for VPIDCFTrackerParams:

Data Fields
float	dcfRegFactor	Regularization factor used in DCF filter creation.

VPIDCFTrackedBoundingBox

struct VPIDCFTrackedBoundingBox

Stores information about an object tracked by DCF Tracker.

Definition at line 515 of file Types.h.

Collaboration diagram for VPIDCFTrackedBoundingBox:

Data Fields
VPIAxisAlignedBoundingBoxF32	bbox	Bounding box around the object being tracked.
VPITrackingState	state	Tracking status of this bounding box.
int32_t	seqIndex	Index of the input sequence where the tracked object is in. If seqIndex < 0, the sequence is assumed to be disabled, i.e., object won't be processed.
float	filterLR	Learning rate for DCF filter in exponential moving average. Must be between 0.0 and 1.0, inclusive.
float	filterChannelWeightsLR	Learning rate for weights of different feature channels in DCF. Must be between 0.0 and 1.0, inclusive.
void *	userData	Pointer to some unspecified user data. This is used to associated some externa data to the tracked object. DCFTracker won't interpret it in any way, it'll just copy it over.

Enumeration Type Documentation

VPIDCFTrackerCreationFlag

enum VPIDCFTrackerCreationFlag

#include <vpi/algo/DCFTracker.h>

Flags to customize DCF Tracker algorithm.

Enumerator
VPI_DCFTRACKER_USE_COLORNAMES	Use ColorNames feature.
VPI_DCFTRACKER_USE_HOG	Use Histogram of Oriented Gradients feature.
VPI_DCFTRACKER_USE_HIGH_PRECISION_FEATURES	Use 16-bit high precision features. If not set, use 8-bit precision features.
VPI_DCFTRACKER_DETERMINISTIC	Use deterministic mode. Determinism can only be guaranteed per architecture.

Definition at line 178 of file DCFTracker.h.

Function Documentation

vpiInitDCFTrackerCreationParams()

VPIStatus vpiInitDCFTrackerCreationParams

(

VPIDCFTrackerCreationParams *

params

)

#include <vpi/algo/DCFTracker.h>

Initialize VPIDCFTrackerCreationParams with default values.

Default values:

• flags: VPI_DCFTRACKER_USE_COLORNAMES | VPI_DCFTRACKER_USE_HOG
• featurePatchSize: 24
• hogCellSize: 4
• featureFocusVertOffsetFactor: 0.0
• gaussianSigma: 0.75

Parameters

[out]

params

Structure to be filled with default values.

Return values

VPI_ERROR_INVALID_ARGUMENT	params is NULL.
VPI_SUCCESS	Operation executed successfully.

vpiCreateDCFTracker()

VPIStatus vpiCreateDCFTracker	(	uint64_t	backends,
		int32_t	maxNumSequences,
		int32_t	maxNumObjects,
		const VPIDCFTrackerCreationParams *	params,
		VPIPayload *	payload
	)

#include <vpi/algo/DCFTracker.h>

Creates payload for DCF Tracker.

Parameters

[in]	backends	VPI backends that are eligible to execute the algorithm. Valid values: VPI_BACKEND_PVA VPI_BACKEND_CUDA Backend must be enabled in current context.
[in]	maxNumSequences	Maximum number of sequences that contains objects. Must be >= 1. For VPI_BACKEND_PVA, must be <= 33 For VPI_BACKEND_CUDA, must be <= 1024
[in]	maxNumObjects	Maximum number of objects to track. Must be >= 1. For VPI_BACKEND_PVA, must be <= 512
[in]	params	Configuration parameters. Pass NULL to use defaults specified by vpiInitDCFTrackerCreationParams.
[out]	payload	Pointer to the payload variable that receives the created handle.

Return values

VPI_ERROR_INVALID_ARGUMENT	payload is NULL.
VPI_ERROR_INVALID_ARGUMENT	backends refers to an invalid backend.
VPI_ERROR_INVALID_ARGUMENT	maxTrackedObjects is outside valid range.
VPI_ERROR_INVALID_ARGUMENT	One or more configuration parameters are invalid.
VPI_ERROR_INVALID_OPERATION	PVA hardware is not available.
VPI_ERROR_INVALID_OPERATION	Backend isn't enabled in current context.
VPI_ERROR_NOT_IMPLEMENTED	DCF Tracker algorithm is not supported by given backend.
VPI_ERROR_INVALID_CONTEXT	Current context is destroyed.
VPI_ERROR_OUT_OF_MEMORY	Cannot allocate required resources.
VPI_SUCCESS	Operation executed successfully.

vpiInitDCFTrackerParams()

VPIStatus vpiInitDCFTrackerParams

(

VPIDCFTrackerParams *

params

)

#include <vpi/algo/DCFTracker.h>

Initialize VPIDCFTrackerParams with default values.

Default values:

• dcfRegFactor: 5.f/100

Parameters

[out]

params

Structure to be filled with default values.

Returns

an error code on failure else VPI_SUCCESS.

vpiSubmitDCFTrackerLocalizeBatch()

VPIStatus vpiSubmitDCFTrackerLocalizeBatch	(	VPIStream	stream,
		uint64_t	backend,
		VPIPayload	payload,
		const int32_t *	enabledSequences,
		int32_t	numSequences,
		VPIImage	featureMaskingWindow,
		VPIImage	inPatches,
		VPIArray	inObjects,
		VPIArray	outObjects,
		VPIImage	outCorrelationResponses,
		VPIArray	outMaxCorrelationResponses,
		const VPIDCFTrackerParams *	params
	)

#include <vpi/algo/DCFTracker.h>

Localizes each tracked object in the input image patches using the Discriminative Correlation Filter method.

The bounding box of each tracked object will be updated with the object's estimated position in its corresponding input patch.

Parameters

[in]	stream	The stream where the operation will be queued in. Must not be NULL. Stream must have enabled the backends that will execute the algorithm.
[in]	backend	Backend that will execute the algorithm. Must be the backend specified during payload creation or 0 as a shorthand to use this backend.
[in]	payload	Payload created by vpiCreateDCFTracker. Must not be NULL.
[in]	enabledSequences	Defines the sequences whose objects are to be processed, irrespective of their state. If an element in the array is 0, the objects that refer the sequence with given index won't be processed, otherwise the algorithm will use the object's state to tell if objects is to be processed or not. If NULL, all sequences will be enabled.
[in]	numSequences	Number of elements in enabledSequences. If an object refer to a sequence whose index is >= numSequences, this sequence will be assumed to be enabled. Or else if the sequence index refered is < 0, this sequence will be assumed to be disabled. Must be >= 0 and <= max sequences specified in payload creation.
[in]	featureMaskingWindow	Image used as window applied to each object feature channel, commonly used to reduce border effects. If NULL, a standard Hanning window will be used in all objects being tracked. Image format must be VPI_IMAGE_FORMAT_F32. Image size must be equal to feature size as specified in featurePatchSize configuration.
[in]	inPatches	Image containing the image patches that tentatively contain the tracked objects. Patches are usually generated by the CropScaler algorithm. Must not be NULL. Image width must be equal to featurePatchSize * hogCellSize (see VPIDCFTrackerCreationParams). If HOG isn't being used, consider hogCellSize==1. Image height must be a multiple of featurePatchSize * hogCellSize, and at be at least featurePatchSize * hogCellSize * (number of objects in inObjects). If HOG isn't being used, consider hogCellSize==1. Valid Image formats: Formats CUDA PVA VPI_IMAGE_FORMAT_RGBA8 * VPI_IMAGE_FORMAT_RGB8p *
[in]	inObjects	Input array with objects being tracked. Must not be NULL. Must have type VPI_ARRAY_TYPE_DCF_TRACKED_BOUNDING_BOX. Array must have enabled the backends that will execute the algorithm.
[out]	outObjects	Output objects' array with their bounding box updated with the estimated position on input sequence. Only objects whose state is VPI_TRACKING_STATE_TRACKED are updated. Remaining objects are just copied over. Must not be NULL. Must have type VPI_ARRAY_TYPE_DCF_TRACKED_BOUNDING_BOX. Its capacity must be equal or greater than the number of elements in inObjects. Array must have enabled the backends that will execute the algorithm.
[out]	outCorrelationResponses	Optional output with the correlation response map for each object being tracked. If NULL, correlation response won't be generated. The correlation map for an object 'o' will start at row o * featurePatchSize and have height 'featurePatchSize'. Must be an image with format VPI_IMAGE_FORMAT_F32. Image width must be equal to patch width (see VPIDCFTrackerCreationParams::featurePatchSize) Image height must be equal to patch height times the number of objects in inObjects.
[out]	outMaxCorrelationResponses	Option output with the maximum correlation response for each tracked object. The maximum correlation response is only valid for objects whose state is VPI_TRACKING_STATE_TRACKED and VPI_TRACKING_STATE_SHADOW_TRACKED. Must be an array with type VPI_ARRAY_TYPE_F32. Array capacity must be >= the number of objects in inObjets array.
[in]	params	Control parameters of the DCF tracker algorithm. If NULL, use defaults as specified by vpiInitDCFTrackerParams.

Return values

VPI_ERROR_INVALID_ARGUMENT	stream is NULL.
VPI_ERROR_INVALID_ARGUMENT	inPatches, inObjects or outObjects are NULL.
VPI_ERROR_INVALID_ARGUMENT	payload not created by vpiCreateDCFTracker.
VPI_ERROR_INVALID_ARGUMENT	Unsupported array type in inObjects or outObjects.
VPI_ERROR_INVALID_ARGUMENT	outObjects capacity too small.
VPI_ERROR_INVALID_ARGUMENT	Parameter(s) in params outside valid range.
VPI_ERROR_INVALID_PAYLOAD_TYPE	payload is invalid.
VPI_ERROR_INVALID_OPERATION	The needed backends aren't enabled in stream, inPatches, inObjects or outObjects.
VPI_SUCCESS	Operation executed successfully.

vpiSubmitDCFTrackerUpdateBatch()

VPIStatus vpiSubmitDCFTrackerUpdateBatch	(	VPIStream	stream,
		uint64_t	backend,
		VPIPayload	payload,
		const int32_t *	enabledSequences,
		int32_t	numSequences,
		VPIImage	featureMaskingWindow,
		VPIImage	modelMaskingWindow,
		VPIImage	inPatches,
		VPIArray	trackedObjects,
		const VPIDCFTrackerParams *	params
	)

#include <vpi/algo/DCFTracker.h>

Update internal object tracking information based on its state and its corresponding input image patch.

The operation performed on each object depends on the object's state:

• VPI_TRACKING_STATE_NEW : object will be set up for tracking starting from the given input image patch. Upon function execution in the given stream, its state will transition to VPI_TRACKING_STATE_TRACKED if object can be tracked, or VPI_TRACKING_STATE_LOST otherwise.
• VPI_TRACKING_STATE_TRACKED : updates internal information about the object using the input image patch that must correspond to its bounding box in the current sequence frame, which could have been modified between localize and update calls.

Parameters

[in]	stream	The stream where the operation will be queued in. Must not be NULL. Stream must have enabled the backends that will execute the algorithm.
[in]	backend	Backend that will execute the algorithm. Must be the backend specified during payload creation or 0 as a shorthand to use this backend.
[in]	payload	Payload created by vpiCreateDCFTracker. Must not be NULL.
[in]	enabledSequences	Defines the sequences whose objects are to be processed, irrespective of their state. If an element in the array is 0, the objects that refer the sequence with given index won't be processed, otherwise the algorithm will use the object's state to tell if objects is to be processed or not. If NULL, all sequences will be enabled.
[in]	numSequences	Number of elements in enabledSequences. If an object refer to a sequence whose index is >= numSequences, this sequence will be assumed to be enabled. Or else if the sequence index refered is < 0, this sequence will be assumed to be disabled. Must be >= 0 and <= max sequences specified in payload creation.
[in]	featureMaskingWindow	Image used as window applied to each object feature channel, commonly used to reduce border effects. If NULL, a standard Hanning window will be used for all patches being tracked. Image format must be VPI_IMAGE_FORMAT_F32. Image size must be equal to patch size as specified in featurePatchSize configuration.
[in]	modelMaskingWindow	Image used as window applied to the internal object model. If NULL, the feature masking window will be used. Image format must be VPI_IMAGE_FORMAT_F32. Image size must be equal to patch size as specified in featurePatchSize configuration.
[in]	inPatches	Image containing the patches of each object in trackedObjects that corresponds to its bounding box. Patches are usually generated by the CropScaler algorithm. Must not be NULL. Image width must be equal to featurePatchSize * hogCellSize (see VPIDCFTrackerCreationParams). If HOG isn't being used, consider hogCellSize==1. Image height must be a multiple of featurePatchSize * hogCellSize, and at be at least featurePatchSize * hogCellSize * (number of objects in trackedObjects). If HOG isn't being used, consider hogCellSize==1. Valid Image formats: Formats CUDA PVA VPI_IMAGE_FORMAT_RGBA8 * VPI_IMAGE_FORMAT_RGB8p *
[in,out]	trackedObjects	Array with tracked objects. Must not be NULL. Must have type VPI_ARRAY_TYPE_DCF_TRACKED_BOUNDING_BOX. Array must have enabled the backends that will execute the algorithm.
[in]	params	Control parameters of the DCF tracker algorithm. If NULL, use defaults as specified by vpiInitDCFTrackerParams.

Return values

VPI_ERROR_INVALID_ARGUMENT	stream is NULL.
VPI_ERROR_INVALID_ARGUMENT	inPatches or trackedObjects are NULL.
VPI_ERROR_INVALID_ARGUMENT	payload not created by vpiCreateDCFTracker.
VPI_ERROR_INVALID_ARGUMENT	inPatches dimensions outside valid range.
VPI_ERROR_INVALID_ARGUMENT	Unsupported array type in trackedObjects.
VPI_ERROR_INVALID_ARGUMENT	Parameter(s) in params outside valid range.
VPI_ERROR_INVALID_PAYLOAD_TYPE	payload is invalid.
VPI_ERROR_INVALID_OPERATION	The needed backends aren't enabled in stream, inPatches or trackedObjects.
VPI_SUCCESS	Operation executed successfully.

vpiDCFTrackerGetChannelWeights()

VPIStatus vpiDCFTrackerGetChannelWeights	(	VPIPayload	payload,
		VPIArray *	channelWeights,
		int32_t *	numFeatureChannels
	)

#include <vpi/algo/DCFTracker.h>

Returns the array with channel weight information for each tracked object.

Parameters

[in]	payload	Payload created by vpiCreateDCFTracker. Must not be NULL.
[out]	channelWeights	Array of channel weights for each tracked object. The returned VPIArray has type VPI_ARRAY_TYPE_F32 and size equal to numFeatureChannels times the number of tracked objects. The weight 'c' of object with index 'o' in array buffer 'd' is given by: \[ w_d(c,o) = (float)d + c * numFeatureChannels \] The weight information is only valid for objects whose state is VPI_TRACKING_STATE_TRACKED. Must not be NULL.

Note

This array must not be destroyed via vpiArrayDestroy.

Parameters

[out]

numFeatureChannels

Number of feature channels per object.

Return values

VPI_ERROR_INVALID_ARGUMENT	channelWeights or numFeatureChannels is NULL.
VPI_SUCCESS	Operation executed successfully.