Detailed Description

Defines 2D-based feature detection and tracking.

Note: SW Release Applicability: These APIs are available in both NVIDIA DriveWorks and NVIDIA DRIVE Software releases.

Data Structures
struct	dwFeature2DTrackerConfig
	Holds configuration parameters for a feature tracker. More...

Typedefs
typedef struct dwFeature2DTrackerObject const *	dwConstFeature2DTrackerHandle_t
	Handle representing a const feature tracker. More...

typedef struct dwFeature2DTrackerObject *	dwFeature2DTrackerHandle_t
	Handle representing a feature tracker. More...

Enumerations
enum	dwFeature2DTrackerAlgorithm { DW_FEATURE2D_TRACKER_ALGORITHM_STD = 0, DW_FEATURE2D_TRACKER_ALGORITHM_EX, DW_FEATURE2D_TRACKER_ALGORITHM_SFM, DW_FEATURE2D_TRACKER_ALGORITHM_COUNT }
	Defines different KLT tracking algorithms. More...

Functions
DW_API_PUBLIC dwStatus	dwFeature2DTracker_compact (dwFeatureHistoryArray *featureHistoryArray, dwFeature2DTrackerHandle_t obj)
	Remove invalid features. More...

DW_API_PUBLIC dwStatus	dwFeature2DTracker_getCUDAStream (cudaStream_t *stream, dwFeature2DTrackerHandle_t obj)
	Gets the CUDA stream used by the feature tracker. More...

DW_API_PUBLIC dwStatus	dwFeature2DTracker_initDefaultParams (dwFeature2DTrackerConfig *params)
	Initializes dwFeature2DTracker parameters with default values. More...

DW_API_PUBLIC dwStatus	dwFeature2DTracker_initDefaultParamsForCamera (dwFeature2DTrackerConfig params, const dwTransformation3f cameraToRig, dwConstCameraModelHandle_t cameraHandle)
	Initializes dwFeature2DTracker parameters with values best suited for the given camera using camera extrinsic (dwTransformation3f* cameraToRig) and camera intrinsic (dwConstCameraModelHandle_t cameraHandle) More...

DW_API_PUBLIC dwStatus	dwFeature2DTracker_initialize (dwFeature2DTrackerHandle_t obj, const dwFeature2DTrackerConfig config, cudaStream_t cudaStream, dwContextHandle_t context)
	Creates and initializes a feature tracker. More...

DW_API_PUBLIC dwStatus	dwFeature2DTracker_release (dwFeature2DTrackerHandle_t obj)
	Releases the feature tracker. More...

DW_API_PUBLIC dwStatus	dwFeature2DTracker_reset (dwFeature2DTrackerHandle_t obj)
	Resets a feature tracker. More...

DW_API_PUBLIC dwStatus	dwFeature2DTracker_setCUDAStream (cudaStream_t stream, dwFeature2DTrackerHandle_t obj)
	Sets the CUDA stream for CUDA related operations. More...

DW_API_PUBLIC dwStatus	dwFeature2DTracker_trackFeatures (dwFeatureHistoryArray featureHistoryArray, dwFeatureArray predictedFeatures, float32_t d_normalizedCrossCorrelation, const dwFeatureArray featuresToTrack, const dwVector2f d_predictedPositions, const dwPyramidImage previousPyramid, const dwPyramidImage *currentPyramid, dwFeature2DTrackerHandle_t obj)
	Tracks features and store the tracked results to `predictedFeatures` between the previous and current images. More...

Data Structure Documentation

◆ dwFeature2DTrackerConfig

struct dwFeature2DTrackerConfig

Data Fields
dwFeature2DTrackerAlgorithm	algorithm	Tracking Algorithm defined by `dwFeature2DTrackerAlgorithm`.
dwFeature2DDetectorType	detectorType	Type of detector that connects to the tracker, should the be the same value as dwFeature2DDetectorConfig::type during detector initialization.
uint32_t	historyCapacity	Upper bound of history in feature history array.
uint32_t	imageHeight	Height of the images that the tracker runs on.
uint32_t	imageWidth	Width of the images that the tracker runs on.
float32_t	largeMotionKillRatio	Features will be killed if the motion is larger than the template size times the large motion killing threshold during tracking. Valid only when algorithm = DW_FEATURE2D_TRACKER_ALGORITHM_EX
uint32_t	maxFeatureCount	Upper bound on number of features handled.
float32_t	maxScaleChange	The maximum allowed scale change for the tracked points across consecutive frames. Valid only when algorithm = DW_FEATURE2D_TRACKER_ALGORITHM_EX
float32_t	nccKillThreshold	Features will be killed if the Cross Correlation Score is less than this threshold during tracking. The value should be between -1.0 and 1.0. Valid only when algorithm = DW_FEATURE2D_TRACKER_ALGORITHM_EX
float32_t	nccUpdateThreshold	Feature template will be updated if the Cross Correlation Score is less than this threshold during tracking. The value should be between -1.0 and 1.0. The value should be no less than nccKillThreshold. Valid only when algorithm = DW_FEATURE2D_TRACKER_ALGORITHM_EX
uint32_t	numIterScaling	Upper bound on number of 3-DOF translation+scaling KLT iterations per level. Valid only when algorithm = DW_FEATURE2D_TRACKER_ALGORITHM_EX
uint32_t	numIterTranslationOnly	Upper bound on number of 2-DOF translation-only KLT iterations per level. If set to zero the default value will be used. More iterations helps improve tracking results but cost more time
uint32_t	numLevelTranslationOnly	Number of levels in pyramid that will use translation-only KLT tracking, level [maxPyramidLevel - 1, maxPyramidLevel - numTranslationOnlyLevel] will use translation-only track level [maxPyramidLevel - numTranslationOnlyLevel - 1, 0] will use scaling track if numTranslationOnlyLevel = 0, scaling track will be applied to all levels in pyramid. Valid only when algorithm = DW_FEATURE2D_TRACKER_ALGORITHM_EX
dwProcessorType	processorType	Processor type which determines on which processor the algorithm should be executed on. Supported options are: DW_PROCESSOR_TYPE_GPU, DW_PROCESSOR_TYPE_PVA_0, DW_PROCESSOR_TYPE_PVA_1. Note DW_PROCESSOR_TYPE_GPU supports DW_FEATURE2D_TRACKER_ALGORITHM_STD, DW_FEATURE2D_TRACKER_ALGORITHM_EX and DW_FEATURE2D_TRACKER_ALGORITHM_SFM DW_PROCESSOR_TYPE_PVA_0/1 supports DW_FEATURE2D_TRACKER_ALGORITHM_STD
uint32_t	pyramidLevelCount	Levels of pyramid to track Valid only when algorithm = DW_FEATURE2D_TRACKER_ALGORITHM_EX.
uint32_t	windowSizeLK	Window size used in the KLT tracker. Supported sizes are DW_FEATURE2D_TRACKER_ALGORITHM_STD: 6, 8, 10, 12, 14. DW_FEATURE2D_TRACKER_ALGORITHM_EX: 10, 12. If set to zero the default value will be used. Larger window size provides better tracking results but costs more time.

Typedef Documentation

◆ dwConstFeature2DTrackerHandle_t

typedef struct dwFeature2DTrackerObject const* dwConstFeature2DTrackerHandle_t

Handle representing a const feature tracker.

Definition at line 68 of file FeatureTracker.h.

◆ dwFeature2DTrackerHandle_t

typedef struct dwFeature2DTrackerObject* dwFeature2DTrackerHandle_t

Handle representing a feature tracker.

Definition at line 65 of file FeatureTracker.h.

Enumeration Type Documentation

◆ dwFeature2DTrackerAlgorithm

enum dwFeature2DTrackerAlgorithm

Defines different KLT tracking algorithms.

Enumerator
DW_FEATURE2D_TRACKER_ALGORITHM_STD	2-DOF (dx, dy) standard Translation-only KLT tracking.
DW_FEATURE2D_TRACKER_ALGORITHM_EX	3-DOF (dx, dy, dscale) extended Translation+Scaling KLT tracking.
DW_FEATURE2D_TRACKER_ALGORITHM_SFM	Tracker for SFM.
DW_FEATURE2D_TRACKER_ALGORITHM_COUNT

Definition at line 73 of file FeatureTracker.h.

Function Documentation

◆ dwFeature2DTracker_compact()

DW_API_PUBLIC dwStatus dwFeature2DTracker_compact	(	dwFeatureHistoryArray *	featureHistoryArray,
		dwFeature2DTrackerHandle_t	obj
	)

Remove invalid features.

Parameters

[in,out]	featureHistoryArray	Sparse feature history array, the features whose status = DW_FEATURE2D_STATUS_INVALID will be removed after calling
[in]	obj	Specifies the feature tracker handle.

Returns: DW_INVALID_ARGUMENT if featureHistoryArray or obj is NULL.
A specific CUDA error in case of an underlying cuda failure.
DW_SUCCESS otherwise.

Note: this API does real work only when dwFeature2DTrackerConfig::algorithm = DW_FEATURE2D_TRACKER_ALGORITHM_SFM, It will also update the dwFeatureHistoryArray::newToOldMap.

◆ dwFeature2DTracker_getCUDAStream()

DW_API_PUBLIC dwStatus dwFeature2DTracker_getCUDAStream	(	cudaStream_t *	stream,
		dwFeature2DTrackerHandle_t	obj
	)

Gets the CUDA stream used by the feature tracker.

Parameters

[out]	stream	The CUDA stream currently used.
[in]	obj	A handle to the feature tracker module.

Returns: DW_INVALID_ARGUMENT if feature tracker handle or stream are NULL.
DW_SUCCESS otherwise.

◆ dwFeature2DTracker_initDefaultParams()

DW_API_PUBLIC dwStatus dwFeature2DTracker_initDefaultParams ( dwFeature2DTrackerConfig * params )

Initializes dwFeature2DTracker parameters with default values.

Parameters

[out] params dwFeature2DTracker parameters

Returns: DW_INVALID_ARGUMENT if params is NULL.
DW_SUCCESS otherwise.

◆ dwFeature2DTracker_initDefaultParamsForCamera()

DW_API_PUBLIC dwStatus dwFeature2DTracker_initDefaultParamsForCamera	(	dwFeature2DTrackerConfig *	params,
		const dwTransformation3f *	cameraToRig,
		dwConstCameraModelHandle_t	cameraHandle
	)

Initializes dwFeature2DTracker parameters with values best suited for the given camera using camera extrinsic (dwTransformation3f* cameraToRig) and camera intrinsic (dwConstCameraModelHandle_t cameraHandle)

Parameters

[out]	params	dwFeature2DTracker parameters
[in]	cameraToRig	transformed camera extrinsic parameters with respect to rig
[in]	cameraHandle	holds camera intrinsic information

Returns: DW_INVALID_ARGUMENT if any parameter is NULL.
DW_SUCCESS otherwise.

◆ dwFeature2DTracker_initialize()

DW_API_PUBLIC dwStatus dwFeature2DTracker_initialize	(	dwFeature2DTrackerHandle_t *	obj,
		const dwFeature2DTrackerConfig *	config,
		cudaStream_t	cudaStream,
		dwContextHandle_t	context
	)

Creates and initializes a feature tracker.

Parameters

[out]	obj	A pointer to the feature tracker handle is returned here.
[in]	config	the configuration parameters for tracker.
[in]	cudaStream	the CUDA stream to use for tracker operations.
[in]	context	the handle to the context under which it is created.

Returns: DW_INVALID_ARGUMENT if feature tracker handle, featureHistoryArray or context are NULL.
if config is invalid.
if featureHistoryArray is not allocated on GPU if featureHistoryArray contains invalid pointers DW_SUCCESS otherwise.

◆ dwFeature2DTracker_release()

DW_API_PUBLIC dwStatus dwFeature2DTracker_release ( dwFeature2DTrackerHandle_t obj )

Releases the feature tracker.

This method releases all resources associated with a feature tracker.

Note: This method renders the handle unusable.

Parameters

[in] obj The object handle to be released.

Returns: DW_INVALID_ARGUMENT if feature tracker handle is NULL.
DW_SUCCESS otherwise.

◆ dwFeature2DTracker_reset()

DW_API_PUBLIC dwStatus dwFeature2DTracker_reset ( dwFeature2DTrackerHandle_t obj )

Resets a feature tracker.

Parameters

[in] obj Specifies the feature tracker handle to be reset.

Returns: DW_INVALID_ARGUMENT if feature tracker handle is NULL.
DW_SUCCESS otherwise.

◆ dwFeature2DTracker_setCUDAStream()

DW_API_PUBLIC dwStatus dwFeature2DTracker_setCUDAStream	(	cudaStream_t	stream,
		dwFeature2DTrackerHandle_t	obj
	)

Sets the CUDA stream for CUDA related operations.

Note: The ownership of the stream remains by the callee.

Parameters

[in]	stream	The CUDA stream to be used. Default is the one passed during dwFeature2DTracker_initialize.
[in]	obj	A handle to the feature tracker module to set CUDA stream for.

Returns: DW_INVALID_ARGUMENT if feature tracker handle is NULL.
DW_SUCCESS otherwise.

◆ dwFeature2DTracker_trackFeatures()

DW_API_PUBLIC dwStatus dwFeature2DTracker_trackFeatures	(	dwFeatureHistoryArray *	featureHistoryArray,
		dwFeatureArray *	predictedFeatures,
		float32_t *	d_normalizedCrossCorrelation,
		const dwFeatureArray *	featuresToTrack,
		const dwVector2f *	d_predictedPositions,
		const dwPyramidImage *	previousPyramid,
		const dwPyramidImage *	currentPyramid,
		dwFeature2DTrackerHandle_t	obj
	)

Tracks features and store the tracked results to predictedFeatures between the previous and current images.

Features to track are defined by dwFeatureHistoryArray bound to the tracker during initialization

Parameters

[out]	featureHistoryArray	output feature history array, must be on GPU memory.
[out]	predictedFeatures	list of predicted features, it's also the top slice of `dwFeatureHistoryArray`
[out]	d_normalizedCrossCorrelation	Device pointer to nccScore of tracked features, will be ignored if it's NULL.
[in]	featuresToTrack	list of features to be tracked, usually the output of feature detector.
[in]	d_predictedPositions	A GPU pointer to a list of expected positions of the features to be tracked. The indexes of this list must match the indexes of the internal feature list. If this is NULL(== 0), then use the locations in `featuresToTrack`
[in]	previousPyramid	pyramid constructed from the last image.
[in]	currentPyramid	pyramid constructed from the current image.
[in]	obj	Specifies the feature tracker handle.

Returns: DW_INVALID_ARGUMENT if any parameter except d_normalizedCrossCorrelation and d_predictedPositions is NULL, or previous and current pyramids have a different number of levels.
if previousFeatures or detectedFeatures are not on GPU.
A specific CUDA error in case of an underlying cuda failure.
DW_SUCCESS otherwise.

Note: Output predictedFeatures array contains a list of compacted features, invalid trackings are removed automatically, the orders between input featuresToTrack and output predictedFeatures can be queried by dwFeatureHistoryArray::newToOldMap which maps new->old index: newToOldMap[newIdx] == oldIdx. newToOldMap[i] = j means the i-th feature in output predictedFeatures is the j-th item in input featuresToTrack array. i.e. predictedFeatures[i] = featuresToTrack[newToOldMap[i]], below is an example: data: [1 2 3 4 5 6 7 8 9] tracked/untracked: [O O X X O X O X O] compacted output (overwrite 3 and 4 by 9 and 7): data: [1 2 9 7 5] newToOldMap: [0 1 8 6 4]

Detailed Description

Data Structures

Typedefs

Enumerations

Functions

Data Structure Documentation

◆ dwFeature2DTrackerConfig

Typedef Documentation

◆ dwConstFeature2DTrackerHandle_t

◆ dwFeature2DTrackerHandle_t

Enumeration Type Documentation

◆ dwFeature2DTrackerAlgorithm

Function Documentation

◆ dwFeature2DTracker_compact()

◆ dwFeature2DTracker_getCUDAStream()

◆ dwFeature2DTracker_initDefaultParams()

◆ dwFeature2DTracker_initDefaultParamsForCamera()

◆ dwFeature2DTracker_initialize()

◆ dwFeature2DTracker_release()

◆ dwFeature2DTracker_reset()

◆ dwFeature2DTracker_setCUDAStream()

◆ dwFeature2DTracker_trackFeatures()