The 2D Tracker module can detect and track feature points or templates between frames recorded by one camera. From the perspective of hardware, it can be divided into two categories: GPU 2D Tracker and PVA 2D Tracker. For GPU 2D Tracker, all functionalities are implemented as CUDA kernels and runs asynchronously on the GPU. As for PVA 2D Tracker, the tracking algorithm runs asynchronously on the PVA instead of GPU.
To track 2D feature points between two frames, dwFeatureTracker_trackFeaturesAsync() takes as input two pyramids and list of 2D feature points as well as predicted locations for those points. The prediction can be the current position or can be computed by some motion model for the feature points. The output is the location and status for each input point, e.g. if the feature could be tracked successfully or not.
Feature2D tracker doesn't detect feature points, it must have an external feature list that contains the features to be tracked as input, usually from Feature2D detector, see more info from Features and Feature 2D Detector Interface
There're 3 types of tracking algorithm: DW_FEATURE2D_TRACKER_ALGORITHM_STD, DW_FEATURE2D_TRACKER_ALGORITHM_EX and DW_FEATURE2D_TRACKER_ALGORITHM_EX_FAST. DW_FEATURE2D_TRACKER_ALGORITHM_STD uses IA (inverse additive) KLT, always tracks feature from previous frame to current frame, it is translation only tracking. It's the fastest mode, but the accuracy is not as good as the other 2 ones. DW_FEATURE2D_TRACKER_ALGORITHM_EX uses IC (inverse compositive) KLT, it supports the 3-DOF translation+scale tracking, it also maintains the feature templates inside the tracker, so it tracks frame N-k to the current frame N. The interanl template from frame N-k will only be updated when it differs too much to current frame N. This mode gives better accuracy but is the slowest. DW_FEATURE2D_TRACKER_ALGORITHM_EX_FAST uses IC KLT and supports translation+scale tracking just like DW_FEATURE2D_TRACKER_ALGORITHM_EX mode, but it doesn't maintain the internal feature templates. It always tracks features from previous frame to current frame as DW_FEATURE2D_TRACKER_ALGORITHM_STD. This mode provides better accuracy than DW_FEATURE2D_TRACKER_ALGORITHM_STD, and is faster than DW_FEATURE2D_TRACKER_ALGORITHM_EX, it is the recommended mode in most cases.
Template Tracker module tracks templates between frames recorded by one camera. Template contains both position and size information. A 2D bounding box is considered a template.
The module supports both GPU and PVA as processing units and has different restrictions accordingly.
On GPU, the module supports templates up to 128x128. On PVA, the module supports templates up to 64x64 and up to 64 templates.
For those features with larger size, it will use the center maximum subregion supported for prediction. The module only does tracking work, user need to do detection by the help of other DriveWorks modules.
Besides tracking the module comes with functionality to manage lists of 2D templates. All functionalities are implemented as CUDA kernels and runs asynchronously on the GPU.
The user defines an upper bound on the number of templates during initialization. This defines the size of allocations inside the scaling tracker module. The runtime of the scaling tracker can still vary per the actual number of templates. The tracker doesn’t do detections, it only removes the features failing to be tracked. If there are new features, user must update template list themselves.
To track 2D templates between two frames
(Assume the template list is updated) dwTemplateTracker_trackAsync() takes as input the target frame to be tracked and list of 2D template points. The output is the location, the size, the scale change factor and status for each input template, e.g. if the feature could be tracked successfully or not. dwTemplateTracker_updateTemplateAsync() takes as input the tracked templates and the new template image to be tracked from. To track templates from Frame N-1 to Frame N, Frame N-1 is the template frame while Frame N is the target frame. dwTemplateTracker_trackAsync() and dwTemplateTracker_trackAsync() must be called by pair to ensure template updating.
Template list manages an ordered list of 2d templates. Each template has a 2D location, its 2D size, and status flag indicating if the feature has been successfully tracked or not. It also provides the scale factor to indicate the change of size and the template location/size information.
Besides storing the templates and serving as I/O to the tracker, the template list also comes with basic housekeeping functionality. Although the scaling tracker supports feature size larger than 128x128 (or 64x64 in case of PVA) by selecting the center part, it may lose precision slightly for large feature size. To avoid the too large templates, dwTemplateList_applySizeFilter() will mark all features with size larger than the given value as invalid.
To add new templates to the list, dwTemplateList_addEmptyFeatures() must be called so that the new added features can be assigned with correct initial properties automatically. Input is the number of new features to be added.
To remove templates from the list that have a status indicating that they were not successfully tracked the combination of dwTemplateList_selectValid() and dwTemplateList_compact() can be used. The output is a compacted template list with only valid features in the input list.
2D Box Tracker module tracks rigid objects for a limited duration. The tracker contains a rich set of configuration parameters that address different tracking scenarios. To successfully track objects, you can modify parameters with contextual information and apply additional constraints for their specific application setup.