DriveWorks SDK Reference
3.5.78 Release
For Test and Development only

Feature 2D Tracker Interface

Detailed Description

Defines 2D-based feature detection and tracking.

Note
SW Release Applicability: These APIs are available in both NVIDIA DriveWorks and NVIDIA DRIVE Software releases.

Data Structures

struct  dwFeature2DTrackerConfig
 Holds configuration parameters for a feature tracker. More...
 
struct  dwFeature2DTrackerConfigNew
 Holds configuration parameters for a feature tracker. More...
 

Macros

#define DW_FEATURE2D_TRACKER_MAX_WINDOW_SIZE   16
 Maximum tracking window size. More...
 

Typedefs

typedef struct dwFeature2DTrackerObject const * dwConstFeature2DTrackerHandle_t
 Handle representing a const feature tracker. More...
 
typedef struct dwFeature2DTrackerObject * dwFeature2DTrackerHandle_t
 Handle representing a feature tracker. More...
 

Enumerations

enum  dwFeature2DTrackerAlgorithm {
  DW_FEATURE2D_TRACKER_ALGORITHM_STD = 0,
  DW_FEATURE2D_TRACKER_ALGORITHM_EX,
  DW_FEATURE2D_TRACKER_ALGORITHM_EX_FAST,
  DW_DEPRECATED_ENUM =("deprecated, please use dwFeature2DTrackerConfigNew::enableSparseOutput=1"),
  DW_FEATURE2D_TRACKER_ALGORITHM_COUNT
}
 Defines different KLT tracking algorithms. More...
 

Functions

DW_API_PUBLIC dwStatus dwFeature2DTracker_compact (dwFeatureHistoryArray *featureHistoryArray, dwFeature2DTrackerHandle_t obj)
 Remove invalid features. More...
 
DW_API_PUBLIC dwStatus dwFeature2DTracker_getCUDAStream (cudaStream_t *stream, dwFeature2DTrackerHandle_t obj)
 Gets the CUDA stream used by the feature tracker. More...
 
DW_API_PUBLIC dwStatus dwFeature2DTracker_initDefaultParams (dwFeature2DTrackerConfig *params)
 Initializes dwFeature2DTracker parameters with default values. More...
 
DW_API_PUBLIC dwStatus dwFeature2DTracker_initDefaultParamsForCamera (dwFeature2DTrackerConfig *params, const dwTransformation3f *cameraToRig, dwConstCameraModelHandle_t cameraHandle)
 Initializes dwFeature2DTracker parameters with values best suited for the given camera using camera extrinsic (dwTransformation3f* cameraToRig) and camera intrinsic (dwConstCameraModelHandle_t cameraHandle) More...
 
DW_API_PUBLIC dwStatus dwFeature2DTracker_initDefaultParamsForCameraNew (dwFeature2DTrackerConfigNew *params, const dwTransformation3f *cameraToRig, dwConstCameraModelHandle_t cameraHandle)
 Initializes dwFeature2DTracker parameters with values best suited for the given camera using camera extrinsic (dwTransformation3f* cameraToRig) and camera intrinsic (dwConstCameraModelHandle_t cameraHandle) More...
 
DW_API_PUBLIC dwStatus dwFeature2DTracker_initDefaultParamsNew (dwFeature2DTrackerConfigNew *params)
 Initializes dwFeature2DTracker parameters with default values. More...
 
DW_API_PUBLIC dwStatus dwFeature2DTracker_initialize (dwFeature2DTrackerHandle_t *obj, const dwFeature2DTrackerConfig *config, cudaStream_t cudaStream, dwContextHandle_t context)
 Creates and initializes a feature tracker. More...
 
DW_API_PUBLIC dwStatus dwFeature2DTracker_initializeNew (dwFeature2DTrackerHandle_t *obj, const dwFeature2DTrackerConfigNew *config, cudaStream_t cudaStream, dwContextHandle_t context)
 Creates and initializes a feature tracker. More...
 
DW_API_PUBLIC dwStatus dwFeature2DTracker_release (dwFeature2DTrackerHandle_t obj)
 Releases the feature tracker. More...
 
DW_API_PUBLIC dwStatus dwFeature2DTracker_reset (dwFeature2DTrackerHandle_t obj)
 Resets a feature tracker. More...
 
DW_API_PUBLIC dwStatus dwFeature2DTracker_setCUDAStream (cudaStream_t stream, dwFeature2DTrackerHandle_t obj)
 Sets the CUDA stream for CUDA related operations. More...
 
DW_API_PUBLIC dwStatus dwFeature2DTracker_trackFeatures (dwFeatureHistoryArray *featureHistoryArray, dwFeatureArray *predictedFeatures, float32_t *d_normalizedCrossCorrelation, const dwFeatureArray *featuresToTrack, const dwVector2f *d_predictedPositions, const dwPyramidImage *previousPyramid, const dwPyramidImage *currentPyramid, dwFeature2DTrackerHandle_t obj)
 Tracks features and store the tracked results to predictedFeatures between the previous and current images. More...
 

Data Structure Documentation

◆ dwFeature2DTrackerConfig

struct dwFeature2DTrackerConfig
Data Fields
dwFeature2DTrackerAlgorithm algorithm Tracking Algorithm defined by dwFeature2DTrackerAlgorithm.
dwFeature2DDetectorType detectorType Type of detector that connects to the tracker, should the be the same value as dwFeature2DDetectorConfigNew::type during detector initialization.
uint32_t historyCapacity Upper bound of history in feature history array.
uint32_t imageHeight Height of the images that the tracker runs on.
uint32_t imageWidth Width of the images that the tracker runs on.
float32_t largeMotionKillRatio Features will be killed if the motion is larger than the template size times the large motion killing threshold during tracking.

Valid only when algorithm = DW_FEATURE2D_TRACKER_ALGORITHM_EX

uint32_t maxFeatureCount Upper bound on number of features handled.
float32_t maxScaleChange The maximum allowed scale change for the tracked points across consecutive frames.

Valid only when algorithm = DW_FEATURE2D_TRACKER_ALGORITHM_EX

float32_t nccKillThreshold Features will be killed if the Cross Correlation Score is less than this threshold during tracking.

The value should be between -1.0 and 1.0. Valid only when algorithm = DW_FEATURE2D_TRACKER_ALGORITHM_EX

float32_t nccUpdateThreshold Feature template will be updated if the Cross Correlation Score is less than this threshold during tracking.

The value should be between -1.0 and 1.0. The value should be no less than nccKillThreshold. Valid only when algorithm = DW_FEATURE2D_TRACKER_ALGORITHM_EX

uint32_t numIterScaling Upper bound on number of 3-DOF translation+scaling KLT iterations per level.

Valid only when algorithm = DW_FEATURE2D_TRACKER_ALGORITHM_EX

uint32_t numIterTranslationOnly Upper bound on number of 2-DOF translation-only KLT iterations per level.

If set to zero the default value will be used. More iterations helps improve tracking results but cost more time

uint32_t numLevelTranslationOnly Number of levels in pyramid that will use translation-only KLT tracking, level [maxPyramidLevel - 1, maxPyramidLevel - numTranslationOnlyLevel] will use translation-only track level [maxPyramidLevel - numTranslationOnlyLevel - 1, 0] will use scaling track if numTranslationOnlyLevel = 0, scaling track will be applied to all levels in pyramid.

Valid only when algorithm = DW_FEATURE2D_TRACKER_ALGORITHM_EX

dwProcessorType processorType Processor type which determines on which processor the algorithm should be executed on.

Supported options are: DW_PROCESSOR_TYPE_GPU, DW_PROCESSOR_TYPE_PVA_0, DW_PROCESSOR_TYPE_PVA_1.

Note
DW_PROCESSOR_TYPE_GPU supports DW_FEATURE2D_TRACKER_ALGORITHM_STD, DW_FEATURE2D_TRACKER_ALGORITHM_EX and DW_FEATURE2D_TRACKER_ALGORITHM_SFM
DW_PROCESSOR_TYPE_PVA_0/1 supports DW_FEATURE2D_TRACKER_ALGORITHM_STD
uint32_t pyramidLevelCount Levels of pyramid to track Valid only when algorithm = DW_FEATURE2D_TRACKER_ALGORITHM_EX.
uint32_t windowSizeLK Window size used in the KLT tracker.

Supported sizes are ** DW_FEATURE2D_TRACKER_ALGORITHM_STD: 6, 8, 10, 12, 14. ** DW_FEATURE2D_TRACKER_ALGORITHM_EX: 10, 12.

If set to zero the default value will be used. Larger window size provides better tracking results but costs more time.

◆ dwFeature2DTrackerConfigNew

struct dwFeature2DTrackerConfigNew
Data Fields
dwFeature2DTrackerAlgorithm algorithm Tracking Algorithm defined by dwFeature2DTrackerAlgorithm.
dwFeature2DDetectorType detectorType Type of detector that connects to the tracker, should the be the same value as dwFeature2DDetectorConfigNew::type during detector initialization.
float32_t displacementThreshold If difference of translation prediction between 2 adjacent KLT iteration is less than this value, it is thought the result is converged and will abort tracking iteration.

Default value = 0.1f, larger value causes faster convergence, but will lose the precision.

bool enableAdaptiveWindowSizeLK Enable adaptive window size If enabled, the tracker will use windowSizeLK to track only at the first and the last pyramid levels, and use smaller window size at the rest pyramid levels.

Valid only when algorithm = DW_FEATURE2D_TRACKER_ALGORITHM_EX

uint32_t enableSparseOutput Enable sparse output if set to 1.

If enabled, the output feature array will contain invalid features, which can be removed by explicitly calling dwFeature2DTracker_compact

uint32_t historyCapacity Upper bound of history in feature history array.
uint32_t imageHeight Height of the images that the tracker runs on.
uint32_t imageWidth Width of the images that the tracker runs on.
float32_t largeMotionKillRatio Features will be killed if the motion is larger than the template size times the large motion killing threshold during tracking.

Valid only when algorithm = DW_FEATURE2D_TRACKER_ALGORITHM_EX

uint32_t maxFeatureCount Upper bound on number of features handled.
float32_t maxScaleChange The maximum allowed scale change for the tracked points across consecutive frames.

Valid only when algorithm = DW_FEATURE2D_TRACKER_ALGORITHM_EX

float32_t nccKillThreshold Features will be killed if the Cross Correlation Score is less than this threshold during tracking.

The value should be between -1.0 and 1.0. Valid only when algorithm = DW_FEATURE2D_TRACKER_ALGORITHM_EX

float32_t nccUpdateThreshold Feature template will be updated if the Cross Correlation Score is less than this threshold during tracking.

The value should be between -1.0 and 1.0. The value should be no less than nccKillThreshold. Valid only when algorithm = DW_FEATURE2D_TRACKER_ALGORITHM_EX

uint32_t numIterScaling Upper bound on number of 3-DOF translation+scaling KLT iterations per level.

Valid only when algorithm = DW_FEATURE2D_TRACKER_ALGORITHM_EX

uint32_t numIterTranslationOnly Upper bound on number of 2-DOF translation-only KLT iterations per level.

If set to zero the default value will be used. More iterations helps improve tracking results but cost more time

uint32_t numLevelTranslationOnly Number of levels in pyramid that will use translation-only KLT tracking, level [maxPyramidLevel - 1, maxPyramidLevel - numTranslationOnlyLevel] will use translation-only track level [maxPyramidLevel - numTranslationOnlyLevel - 1, 0] will use scaling track if numTranslationOnlyLevel = 0, scaling track will be applied to all levels in pyramid.

Valid only when algorithm = DW_FEATURE2D_TRACKER_ALGORITHM_EX

dwProcessorType processorType Processor type which determines on which processor the algorithm should be executed on.

Supported options are: DW_PROCESSOR_TYPE_GPU, DW_PROCESSOR_TYPE_PVA_0, DW_PROCESSOR_TYPE_PVA_1.

Note
DW_PROCESSOR_TYPE_GPU supports DW_FEATURE2D_TRACKER_ALGORITHM_STD, DW_FEATURE2D_TRACKER_ALGORITHM_EX and DW_FEATURE2D_TRACKER_ALGORITHM_SFM
DW_PROCESSOR_TYPE_PVA_0/1 supports DW_FEATURE2D_TRACKER_ALGORITHM_STD
uint32_t pyramidLevelCount Levels of pyramid to track Valid only when algorithm = DW_FEATURE2D_TRACKER_ALGORITHM_EX.
bool useHalf Set it to true to use half float as intermediate results during tracking It saves register usage and will be faster than full 32-bit float, but will lose a bit of precision.

Valid only when algorithm = DW_FEATURE2D_TRACKER_ALGORITHM_EX

uint32_t windowSizeLK Window size used in the KLT tracker.

Supported sizes are ** DW_FEATURE2D_TRACKER_ALGORITHM_STD: 6, 8, 10, 12, 14. ** DW_FEATURE2D_TRACKER_ALGORITHM_EX: 10, 12.

If set to zero the default value will be used. Larger window size provides better tracking results but costs more time.

Macro Definition Documentation

◆ DW_FEATURE2D_TRACKER_MAX_WINDOW_SIZE

#define DW_FEATURE2D_TRACKER_MAX_WINDOW_SIZE   16

Maximum tracking window size.

Definition at line 65 of file FeatureTracker.h.

Typedef Documentation

◆ dwConstFeature2DTrackerHandle_t

typedef struct dwFeature2DTrackerObject const* dwConstFeature2DTrackerHandle_t

Handle representing a const feature tracker.

Definition at line 71 of file FeatureTracker.h.

◆ dwFeature2DTrackerHandle_t

typedef struct dwFeature2DTrackerObject* dwFeature2DTrackerHandle_t

Handle representing a feature tracker.

Definition at line 68 of file FeatureTracker.h.

Enumeration Type Documentation

◆ dwFeature2DTrackerAlgorithm

Defines different KLT tracking algorithms.

Enumerator
DW_FEATURE2D_TRACKER_ALGORITHM_STD 

3-DOF (dx, dy, dscale) standard KLT tracking.

DW_FEATURE2D_TRACKER_ALGORITHM_EX 

3-DOF (dx, dy, dscale) extended KLT tracking.

DW_FEATURE2D_TRACKER_ALGORITHM_EX_FAST 

3-DOF (dx, dy, dscale) fast extended KLT tracking.

DW_DEPRECATED_ENUM 

Tracker for SFM.

DW_FEATURE2D_TRACKER_ALGORITHM_COUNT 

Definition at line 76 of file FeatureTracker.h.

Function Documentation

◆ dwFeature2DTracker_compact()

DW_API_PUBLIC dwStatus dwFeature2DTracker_compact ( dwFeatureHistoryArray featureHistoryArray,
dwFeature2DTrackerHandle_t  obj 
)

Remove invalid features.

Parameters
[in,out]featureHistoryArraySparse feature history array, the features whose status = DW_FEATURE2D_STATUS_INVALID will be removed after calling
[in]objSpecifies the feature tracker handle.
Returns
DW_INVALID_ARGUMENT if featureHistoryArray or obj is NULL.
A specific CUDA error in case of an underlying cuda failure.
DW_SUCCESS otherwise.
Note
this API does real work only when dwFeature2DTrackerConfigNew::algorithm = DW_FEATURE2D_TRACKER_ALGORITHM_SFM, It will also update the dwFeatureHistoryArray::newToOldMap.

◆ dwFeature2DTracker_getCUDAStream()

DW_API_PUBLIC dwStatus dwFeature2DTracker_getCUDAStream ( cudaStream_t *  stream,
dwFeature2DTrackerHandle_t  obj 
)

Gets the CUDA stream used by the feature tracker.

Parameters
[out]streamThe CUDA stream currently used.
[in]objA handle to the feature tracker module.
Returns
DW_INVALID_ARGUMENT if feature tracker handle or stream are NULL.
DW_SUCCESS otherwise.

◆ dwFeature2DTracker_initDefaultParams()

DW_API_PUBLIC dwStatus dwFeature2DTracker_initDefaultParams ( dwFeature2DTrackerConfig params)

Initializes dwFeature2DTracker parameters with default values.

Parameters
[out]paramsdwFeature2DTracker parameters
Returns
DW_INVALID_ARGUMENT if params is NULL.
DW_SUCCESS otherwise.

◆ dwFeature2DTracker_initDefaultParamsForCamera()

DW_API_PUBLIC dwStatus dwFeature2DTracker_initDefaultParamsForCamera ( dwFeature2DTrackerConfig params,
const dwTransformation3f cameraToRig,
dwConstCameraModelHandle_t  cameraHandle 
)

Initializes dwFeature2DTracker parameters with values best suited for the given camera using camera extrinsic (dwTransformation3f* cameraToRig) and camera intrinsic (dwConstCameraModelHandle_t cameraHandle)

Parameters
[out]paramsdwFeature2DTracker parameters
[in]cameraToRigtransformed camera extrinsic parameters with respect to rig
[in]cameraHandleholds camera intrinsic information
Returns
DW_INVALID_ARGUMENT if any parameter is NULL.
DW_SUCCESS otherwise.

◆ dwFeature2DTracker_initDefaultParamsForCameraNew()

DW_API_PUBLIC dwStatus dwFeature2DTracker_initDefaultParamsForCameraNew ( dwFeature2DTrackerConfigNew params,
const dwTransformation3f cameraToRig,
dwConstCameraModelHandle_t  cameraHandle 
)

Initializes dwFeature2DTracker parameters with values best suited for the given camera using camera extrinsic (dwTransformation3f* cameraToRig) and camera intrinsic (dwConstCameraModelHandle_t cameraHandle)

Parameters
[out]paramsdwFeature2DTracker parameters
[in]cameraToRigtransformed camera extrinsic parameters with respect to rig
[in]cameraHandleholds camera intrinsic information
Returns
DW_INVALID_ARGUMENT if any parameter is NULL.
DW_SUCCESS otherwise.

◆ dwFeature2DTracker_initDefaultParamsNew()

DW_API_PUBLIC dwStatus dwFeature2DTracker_initDefaultParamsNew ( dwFeature2DTrackerConfigNew params)

Initializes dwFeature2DTracker parameters with default values.

Parameters
[out]paramsdwFeature2DTracker parameters
Returns
DW_INVALID_ARGUMENT if params is NULL.
DW_SUCCESS otherwise.

◆ dwFeature2DTracker_initialize()

DW_API_PUBLIC dwStatus dwFeature2DTracker_initialize ( dwFeature2DTrackerHandle_t obj,
const dwFeature2DTrackerConfig config,
cudaStream_t  cudaStream,
dwContextHandle_t  context 
)

Creates and initializes a feature tracker.

Parameters
[out]objA pointer to the feature tracker handle is returned here.
[in]configthe configuration parameters for tracker.
[in]cudaStreamthe CUDA stream to use for tracker operations.
[in]contextthe handle to the context under which it is created.
Returns
DW_INVALID_ARGUMENT if feature tracker handle, featureHistoryArray or context are NULL.
if config is invalid.
if featureHistoryArray is not allocated on GPU if featureHistoryArray contains invalid pointers DW_SUCCESS otherwise.

◆ dwFeature2DTracker_initializeNew()

DW_API_PUBLIC dwStatus dwFeature2DTracker_initializeNew ( dwFeature2DTrackerHandle_t obj,
const dwFeature2DTrackerConfigNew config,
cudaStream_t  cudaStream,
dwContextHandle_t  context 
)

Creates and initializes a feature tracker.

Parameters
[out]objA pointer to the feature tracker handle is returned here.
[in]configthe configuration parameters for tracker.
[in]cudaStreamthe CUDA stream to use for tracker operations.
[in]contextthe handle to the context under which it is created.
Returns
DW_INVALID_ARGUMENT if feature tracker handle, featureHistoryArray or context are NULL.
if config is invalid.
if featureHistoryArray is not allocated on GPU if featureHistoryArray contains invalid pointers DW_SUCCESS otherwise.

◆ dwFeature2DTracker_release()

DW_API_PUBLIC dwStatus dwFeature2DTracker_release ( dwFeature2DTrackerHandle_t  obj)

Releases the feature tracker.

This method releases all resources associated with a feature tracker.

Note
This method renders the handle unusable.
Parameters
[in]objThe object handle to be released.
Returns
DW_INVALID_ARGUMENT if feature tracker handle is NULL.
DW_SUCCESS otherwise.

◆ dwFeature2DTracker_reset()

DW_API_PUBLIC dwStatus dwFeature2DTracker_reset ( dwFeature2DTrackerHandle_t  obj)

Resets a feature tracker.

Parameters
[in]objSpecifies the feature tracker handle to be reset.
Returns
DW_INVALID_ARGUMENT if feature tracker handle is NULL.
DW_SUCCESS otherwise.

◆ dwFeature2DTracker_setCUDAStream()

DW_API_PUBLIC dwStatus dwFeature2DTracker_setCUDAStream ( cudaStream_t  stream,
dwFeature2DTrackerHandle_t  obj 
)

Sets the CUDA stream for CUDA related operations.

Note
The ownership of the stream remains by the callee.
Parameters
[in]streamThe CUDA stream to be used. Default is the one passed during dwFeature2DTracker_initializeNew.
[in]objA handle to the feature tracker module to set CUDA stream for.
Returns
DW_INVALID_ARGUMENT if feature tracker handle is NULL.
DW_SUCCESS otherwise.

◆ dwFeature2DTracker_trackFeatures()

DW_API_PUBLIC dwStatus dwFeature2DTracker_trackFeatures ( dwFeatureHistoryArray featureHistoryArray,
dwFeatureArray predictedFeatures,
float32_t d_normalizedCrossCorrelation,
const dwFeatureArray featuresToTrack,
const dwVector2f d_predictedPositions,
const dwPyramidImage previousPyramid,
const dwPyramidImage currentPyramid,
dwFeature2DTrackerHandle_t  obj 
)

Tracks features and store the tracked results to predictedFeatures between the previous and current images.

Features to track are defined by dwFeatureHistoryArray bound to the tracker during initialization

Parameters
[out]featureHistoryArrayoutput feature history array, must be on GPU memory.
[out]predictedFeatureslist of predicted features, it's also the top slice of dwFeatureHistoryArray
[out]d_normalizedCrossCorrelationDevice pointer to nccScore of tracked features, will be ignored if it's NULL.
[in]featuresToTracklist of features to be tracked, usually the output of feature detector.
[in]d_predictedPositionsA GPU pointer to a list of expected positions of the features to be tracked. The indexes of this list must match the indexes of the internal feature list. If this is NULL(== 0), then use the locations in featuresToTrack
[in]previousPyramidpyramid constructed from the last image.
[in]currentPyramidpyramid constructed from the current image.
[in]objSpecifies the feature tracker handle.
Returns
DW_INVALID_ARGUMENT if any parameter except d_normalizedCrossCorrelation and d_predictedPositions is NULL, or previous and current pyramids have a different number of levels.
if previousFeatures or detectedFeatures are not on GPU.
A specific CUDA error in case of an underlying cuda failure.
DW_SUCCESS otherwise.
Note
Output predictedFeatures array contains a list of compacted features, invalid trackings are removed automatically, the orders between input featuresToTrack and output predictedFeatures can be queried by dwFeatureHistoryArray::newToOldMap which maps new->old index: newToOldMap[newIdx] == oldIdx. newToOldMap[i] = j means the i-th feature in output predictedFeatures is the j-th item in input featuresToTrack array. i.e. predictedFeatures[i] = featuresToTrack[newToOldMap[i]], below is an example: data: [1 2 3 4 5 6 7 8 9] tracked/untracked: [O O X X O X O X O] compacted output (overwrite 3 and 4 by 9 and 7): data: [1 2 9 7 5] newToOldMap: [0 1 8 6 4]