imageprocessing/tracking/docs/usecase2.md

Go to the documentation of this file.

# Copyright (c) 2019-2020 NVIDIA CORPORATION.  All rights reserved.

@page imageprocessing_tracking_usecase2 2D Box Tracking

@note SW Release Applicability: This tutorial is applicable to modules in both **NVIDIA DriveWorks** and **NVIDIA DRIVE Software** releases.

#### Initialization

NVIDIA<sup>&reg;</sup> DriveWorks includes a sample pipeline demonstrating tracking of multiple cars across multiple video frames. The following summarizes the sample’s initialization steps:
 
```{.cpp}
    std::vector<dwBox2D> initBoxes;
    dwBoxTracker2DHandle_t boxTracker;

    // CODE: initialize the Box Tracker parameter
    // CODE: initialize the Box Tracker
    // CODE: allocate initBoxes
    // CODE: fill initBoxes with detected bounding boxes

    // Feed initBoxes to Box Tracker
    dwBoxTracker2D_add(initBoxes.data(), initBoxes.size(), boxTracker);
```

#### Initializing the Box Tracking Parameters

The DriveWorks dwBoxTracker2D_initParams() function initializes the Box Tracker parameter with default values. You can revise the tracker parameters for their specific use case. For a definition of the `dwBoxTracker2DParams()` fields, see DriveWorks API Reference. 

#### Initializing the Box Tracker

After you have customized the tracking parameters, you must call the `dwBoxTracker2D_initialize()` function to initialize the Box Tracker internal states and memory. 

#### Initializing Initial 2D Object Locations

The Box Tracker module requires initial 2D object locations represented by 2D rectangles. You can feed detection results to the Box Tracker. 

#### Applying Clustering

DriveWorks provides two methods for applying clustering to reduce redundant/adjacent “raw” detection results:

- With `dwBoxTracker2D_add()`, which is internal to the Box Tracker module
- With `dwBoxTracker2D_addPreClustered()`, which applies clustering outside Box Tracker. This function assumes you first cluster the bounding boxes and add it to Box Tracker afterwards. 
- Manually specify the object locations to start with. For example: 

```{.cpp}
    int32_t x, y;
    int32_t width, height;
    dwBox2D userBox;

    // CODE: specify top left of the bounding box (x, y)
    // CODE: specify width and height of the bounding box (width, height)

    userBox.x = x;
    userBox.y = y;

    userBox.width  = width;
    userBox.height = height;

    dwBoxTracker2D_add(userBox, ...);
```

#### Process

Given the initial bounding boxes, the Box Tracker module computes and associates 2D image features in adjacent image frames. The module uses such feature displacement to estimate the bounding box motion in the next frame. In the figure below, blue, and black crosses represent the successfully tracked 2D image features from frame N to frame N+1. Red cross failed to find association in frame N+1 is thus not used to update the red rectangle in frame N+1. For more information, see 2D Tracker Module in this chapter.

![](tracking_2d_box.png)

To predict the bounding box locations in the current frame using the tracked 2D image features, call the `dwBoxTracker2D_track()` function. Tracking features in multiple frames could suffer from failure association such that the tracked feature number decreases. To avoid such a degradation, the DriveWorks sample Box Tracker application detects 2D image feature for each frame, after previous frame features are tracked. In this way, there are new features available for the next frame to use. Once the 2D feature detection is complete, Box Tracker updates each tracked bounding box with the detected 2D image features. The DriveWorks `dwBoxTracker2D_updateFeatures()` function offers the update mechanism. 

The Box Tracker sample shows how to use `dwBoxTracker2D_track()` and `dwBoxTracker2D_updateFeatures()`.

To obtain the tracked bounding boxes, call `dwBoxTracker2D_get()`. That function returns information such as:

- List of 2D rectangles and tracking states, i.e. confidence
- Box ID
- Total number of tracked frames
- 2d feature locations used to predict the bounding boxes

#### Additional Information

2D Object tracking is used in @ref dwx_object_detector_tracker_sample.