imageprocessing/tracking/docs/usecase1.md

Go to the documentation of this file.

# Copyright (c) 2019-2020, NVIDIA CORPORATION.  All rights reserved.

@page imageprocessing_tracking_usecase1 Single Camera Template Tracking

@note SW Release Applicability: This tutorial is applicable to modules in both **NVIDIA DriveWorks** and **NVIDIA DRIVE Software** releases.

The tutorial shows the general structure of a program that uses the 2D template tracker to track templates in a single camera.

#### Initialize template array on both CPU and GPU

`contextHandle` is assumed to be previously initialized `::dwContextHandle_t`.
```{.cpp}
    dwTemplateArray_create(&templateCPU, maxFeatureCount,
                            DW_FEATURE2D_MEMORY_TYPE_CPU, contextHandle);
    dwTemplateArray_create(&templateGPU, maxFeatureCount,
                           DW_FEATURE2D_MEMORY_TYPE_CUDA, contextHandle);

```

#### Initialize template tracker parameters with default values

```{.cpp}
    dwTemplateTrackerParameters params = {};
    dwTemplateTracker_initDefaultParams(&params);
```

#### Modify parameters according to tracking requirements

`imageProps` is the `dwImageProperties` structure of image to be tracked

```{.cpp}
    dwImage_getPixelType(&params.pxlType, imageProps.format);
    params.validWidth = DW_MAX_TEMPLATE2D_SIZE;
    params.validHeight = DW_MAX_TEMPLATE2D_SIZE;

```

Select on which processor the tracker should be performed on. Possible options are
`DW_PROCESSOR_TYPE_GPU`, `DW_PROCESSOR_TYPE_PVA_0`, `DW_PROCESSOR_TYPE_PVA_1`.
Note that PVA is only supported on NVIDIA DRIVE<sup>&trade;</sup> platforms.
Moreover, if the processor is set to PVA, the image dimensions need to be provided at initialization.

```{.cpp}
    params.processorType = DW_PROCESSOR_TYPE_PVA_0;
    params.imageWidth = 1280;
    params.imageHeight = 800;
```

#### Initialize template tracker with given parameters and bind output

```{.cpp}
    dwTemplateTracker_initialize(&tracker, &params, 0, contextHandle);
```

Create 2 image pyramids as detection/tracking input: one is current, one is
previous, the 2 pyramids work as a double buffer (or ping-pong buffer). They
swap each other after each detection/tracking cycle. So new frame will always
be read into pyramidCurrent and the previous frame will be stored to
pyramidPrevious.

`dwPyramid_create` only creates pyramid image and allocates memory, pyramid will be filled in `dwImageFilter_computePyramid`. Top level (level 0) in pyramid is always the same size as original input image.

`inputImageProps.width`, `inputImageProps.height` are the same in tracker/detector initialization

`imagePxlType` should be got from `dwImage_getPixelType(inputImageProps.format)`.

`contextHandle` is assumed to be previously initialized dwContextHandle_t.

```{.cpp}
    dwPyramid_create(&pyramidPrevious, levelCount,
        inputImageProps.width, inputImageProps.height, pxlType, context);

    dwPyramid_create(&pyramidCurrent, levelCount,
        inputImageProps.width, inputImageProps.height, pxlType,context);
```

#### Add bounding boxes to be tracked to the tracker. BBox can be extracted from the DNN detector, or defined by the user.

```{.cpp}
    // get bbox from DNN detector or user defining
    uint32_t nNewBoxes = ...;
    dwBox2Df* newBoxes = ...;

    *featureData.featureCount = nNewBoxes;
    for (uint32_t i = 0; i < nNewBoxes; i++) {
        featureData.locations[i] = {newBoxes[i].x + newBoxes[i].width/2, newBoxes[i].y + newBoxes[i].height/2};
        featureData.sizes[i]     = {newBoxes[i].width, newBoxes[i].height}
        featureData.statuses[i]  = DW_FEATURE_STATUS_DETECTED;
    }

    // copy data to GPU
    dwTemplateArray_copy(&templateGPU, &templateCPU, 0);
```

#### Start tracking the loop

```{.cpp}
    while(true)
    {
        // swap current/previous pyramid
        // so new frame will always be built in pyramidCurrent
        // and old pyramid will be stored in pyramidPrevious
        std::swap(pyramidCurrent, pyramidPrevious);

        // CODE: Read image frame code
        dwImageHandle_t image = ...;

        // compute pyramid based on the new read frame image
        dwImageFilter_computePyramid(&pyramidCurrent, image, 0, contextHandle);

        dwTemplateTracker_trackPyramid(&templateGPU, &pyramidCurrent,
                                       &pyramidPrevious, tracker);

        // copy detected results to CPU
        dwTemplateArray_copy(&templateCPU, &templateGPU, 0);

        // break the loop when all features are killed
        if (*featureData.featureCount == 0)
            break;

        // do some stuff in CPU
        featureData.locations[...];
        featureData.sizes[...];
    }
```

#### Finally, free previously allocated memory.

```{.cpp}
    dwPyramid_destroy(pyramidPrevious);
    dwPyramid_destroy(pyramidCurrent);

    dwTemplateArray_destroy(templateCPU);
    dwTemplateArray_destroy(templateGPU);
    dwTemplateTracker_release(&tracker);
```

For the full implementation, refer to @ref dwx_template_tracker_sample.