DriveWorks SDK Reference
3.0.4260 Release
For Test and Development only

Tracking
Note
SW Release Applicability: This module is available in both NVIDIA DriveWorks and NVIDIA DRIVE Software releases.

About This Module

The 2D Tracker module can detect and track feature points or templates between frames recorded by one camera. From the perspective of hardware, it can be divided into two categories: GPU 2D Tracker and PVA 2D Tracker. For GPU 2D Tracker, all functionalities are implemented as CUDA kernels and runs asynchronously on the GPU. As for PVA 2D Tracker, the tracking algorithm runs asynchronously on the PVA instead of GPU.

Feature2D Tracker

To track 2D feature points between two frames, dwFeatureTracker_trackFeaturesAsync() takes as input two pyramids and list of 2D feature points as well as predicted locations for those points. The prediction can be the current position or can be computed by some motion model for the feature points. The output is the location and status for each input point, e.g. if the feature could be tracked successfully or not.

Template Tracker

Template Tracker module tracks templates between frames recorded by one camera. Template contains both position and size information. A 2D bounding box is considered a template.

The module supports both GPU and PVA as processing units and has different restrictions accordingly.

On GPU, the module supports templates up to 128x128. On PVA, the module supports templates up to 64x64 and up to 64 templates.

For those features with larger size, it will use the center maximum subregion supported for prediction. The module only does tracking work, user need to do detection by the help of other DriveWorks modules.

Besides tracking the module comes with functionality to manage lists of 2D templates. All functionalities are implemented as CUDA kernels and runs asynchronously on the GPU.

The user defines an upper bound on the number of templates during initialization. This defines the size of allocations inside the scaling tracker module. The runtime of the scaling tracker can still vary per the actual number of templates. The tracker doesn’t do detections, it only removes the features failing to be tracked. If there are new features, user must update template list themselves.

To track 2D templates between two frames

(Assume the template list is updated) dwTemplateTracker_trackAsync() takes as input the target frame to be tracked and list of 2D template points. The output is the location, the size, the scale change factor and status for each input template, e.g. if the feature could be tracked successfully or not. dwTemplateTracker_updateTemplateAsync() takes as input the tracked templates and the new template image to be tracked from. To track templates from Frame N-1 to Frame N, Frame N-1 is the template frame while Frame N is the target frame. dwTemplateTracker_trackAsync() and dwTemplateTracker_trackAsync() must be called by pair to ensure template updating.

Template Lists

Template list manages an ordered list of 2d templates. Each template has a 2D location, its 2D size, and status flag indicating if the feature has been successfully tracked or not. It also provides the scale factor to indicate the change of size and the template location/size information.

Besides storing the templates and serving as I/O to the tracker, the template list also comes with basic housekeeping functionality. Although the scaling tracker supports feature size larger than 128x128 (or 64x64 in case of PVA) by selecting the center part, it may lose precision slightly for large feature size. To avoid the too large templates, dwTemplateList_applySizeFilter() will mark all features with size larger than the given value as invalid.

To add new templates to the list, dwTemplateList_addEmptyFeatures() must be called so that the new added features can be assigned with correct initial properties automatically. Input is the number of new features to be added.

To remove templates from the list that have a status indicating that they were not successfully tracked the combination of dwTemplateList_selectValid() and dwTemplateList_compact() can be used. The output is a compacted template list with only valid features in the input list.

2D Box Tracker

2D Box Tracker module tracks rigid objects for a limited duration. The tracker contains a rich set of configuration parameters that address different tracking scenarios. To successfully track objects, you can modify parameters with contextual information and apply additional constraints for their specific application setup.

Relevant Tutorials

APIs