Single Camera Template Tracking

Note

SW Release Applicability: This tutorial is applicable to modules in both NVIDIA DriveWorks and NVIDIA DRIVE Software releases.

The tutorial shows the general structure of a program that uses the 2D template tracker to track templates in a single camera.

Initialize template array on both CPU and GPU

contextHandle is assumed to be previously initialized dwContextHandle_t.

dwTemplateArray_create(&templateCPU, maxFeatureCount,
                        DW_FEATURE2D_MEMORY_TYPE_CPU, contextHandle);
dwTemplateArray_create(&templateGPU, maxFeatureCount,
                       DW_FEATURE2D_MEMORY_TYPE_CUDA, contextHandle);

Initialize template tracker parameters with default values

dwTemplateTrackerParameters params = {};
dwTemplateTracker_initDefaultParams(&params);

Modify parameters according to tracking requirements

imageProps is the dwImageProperties structure of image to be tracked

dwImage_getPixelType(&params.pxlType, imageProps.format);
params.validWidth = DW_MAX_TEMPLATE2D_SIZE;
params.validHeight = DW_MAX_TEMPLATE2D_SIZE;

Select on which processor the tracker should be performed on. Possible options are DW_PROCESSOR_TYPE_GPU, DW_PROCESSOR_TYPE_PVA_0, DW_PROCESSOR_TYPE_PVA_1. Note that PVA is only supported on NVIDIA DRIVE^™ platforms. Moreover, if the processor is set to PVA, the image dimensions need to be provided at initialization.

params.processorType = DW_PROCESSOR_TYPE_PVA_0;
params.imageWidth = 1280;
params.imageHeight = 800;

Initialize template tracker with given parameters and bind output

dwTemplateTracker_initialize(&tracker, &params, 0, contextHandle);

Create 2 image pyramids as detection/tracking input: one is current, one is previous, the 2 pyramids work as a double buffer (or ping-pong buffer). They swap each other after each detection/tracking cycle. So new frame will always be read into pyramidCurrent and the previous frame will be stored to pyramidPrevious.

dwPyramid_create only creates pyramid image and allocates memory, pyramid will be filled in dwImageFilter_computePyramid. Top level (level 0) in pyramid is always the same size as original input image.

inputImageProps.width, inputImageProps.height are the same in tracker/detector initialization

imagePxlType should be got from dwImage_getPixelType(inputImageProps.format).

contextHandle is assumed to be previously initialized dwContextHandle_t.

dwPyramid_create(&pyramidPrevious, levelCount,
    inputImageProps.width, inputImageProps.height, pxlType, context);

dwPyramid_create(&pyramidCurrent, levelCount,
    inputImageProps.width, inputImageProps.height, pxlType,context);

Add bounding boxes to be tracked to the tracker. BBox can be extracted from the DNN detector, or defined by the user.

// get bbox from DNN detector or user defining
uint32_t nNewBoxes = ...;
dwBox2Df* newBoxes = ...;

*featureData.featureCount = nNewBoxes;
for (uint32_t i = 0; i < nNewBoxes; i++) {
    featureData.locations[i] = {newBoxes[i].x + newBoxes[i].width/2, newBoxes[i].y + newBoxes[i].height/2};
    featureData.sizes[i]     = {newBoxes[i].width, newBoxes[i].height}
    featureData.statuses[i]  = DW_FEATURE_STATUS_DETECTED;
}

// copy data to GPU
dwTemplateArray_copy(&templateGPU, &templateCPU, 0);

Start tracking the loop

while(true)
{
    // swap current/previous pyramid
    // so new frame will always be built in pyramidCurrent
    // and old pyramid will be stored in pyramidPrevious
    std::swap(pyramidCurrent, pyramidPrevious);

    // CODE: Read image frame code
    dwImageHandle_t image = ...;

    // compute pyramid based on the new read frame image
    dwImageFilter_computePyramid(&pyramidCurrent, image, 0, contextHandle);

    dwTemplateTracker_trackPyramid(&templateGPU, &pyramidCurrent,
                                   &pyramidPrevious, tracker);

    // copy detected results to CPU
    dwTemplateArray_copy(&templateCPU, &templateGPU, 0);

    // break the loop when all features are killed
    if (*featureData.featureCount == 0)
        break;

    // do some stuff in CPU
    featureData.locations[...];
    featureData.sizes[...];
}

Finally, free previously allocated memory.

dwPyramid_destroy(pyramidPrevious);
dwPyramid_destroy(pyramidCurrent);

dwTemplateArray_destroy(templateCPU);
dwTemplateArray_destroy(templateGPU);
dwTemplateTracker_release(&tracker);

For the full implementation, refer to Template Tracker Sample.