1 # Copyright (c) 2018-2019, NVIDIA CORPORATION. All rights reserved.
3 @page dwx_porting_guide_2_0 Porting Guide from SDK 1.5 to SDK 2.0
6 This document will help you port your applications from DriveWorks SDK 1.5 to DriveWorks SDK 2.0.
8 ------------------------------------------------------------------
10 @section pg_calibration Calibration
12 The module `dwCalibratedCamera` has been separated from `dwRig` renamed to `dwCameraModel`. It resides in the folder `dw/calibration/cameramodel`. It provides functions to handle calibrated intrinsic camera models.
14 - Header file has been renamed from `Camera.h` to `CameraModel.h`.
15 - The object handles have been renamed to `dwCameraModelHandle_t` and `dwConstCameraModelHandle_t`, respectively.
16 - The prefixes for all functions declared in `CameraModel.h` have been updated from `dwCalibratedCamera` to `dwCameraModel`.
17 - `dwCameraModel_applyImageTransform()` has been updated and now it takes the additional parameter `bool updateMasks`.
18 - `dwCameraModel_initialize` has been moved here from `dwRig`.
20 For more information see @ref rig_mainsection.
22 ------------------------------------------------------------------
26 This release adds support for generating and using DLA (Deep-Learning-Accelerator) models. This change requires selection
27 of the processor type at the time of initialization; therefore, the following functions' signatures have been updated accordingly:
29 - `dwDNN_initializeTensorRTFromFile()` has been renamed to `dwDNN_initializeTensorRTFromFileNew()` and requires an additional argument of type `dwProcessorType` indicating on which processor the model shall run on. Note that the model must be generated on the corresponding processor.
31 - `dwDNN_initializeTensorRTFromMemory()` has been renamed to `dwDNN_initializeTensorRTFromMemoryNew()` and requires an additional argument of type `dwProcessorType` indicating on which processor the model shall run on. Note that the model must be generated on the corresponding processor.
33 @note The old APIs and the `New` suffix for the new APIs will be removed in the next major release.
35 For more information see @ref dnn_mainsection.
37 -----------------------------------------------------------------------
39 @section pg_egomotion Egomotion
41 - `dwEgoMotionParameters` has been renamed to `dwEgomotionParameters`.
42 - `dwEgoMotionSensorCharacteristics` has been renamed to `dwEgomotionSensorCharacteristics`.
43 - The `dwEgomotionParameters.enableLocationFilter` has been renamed to `dwEgomotionParameters.disableLocationFilter`.
44 - `dw/radarmotion` has been moved to `dw/egomotion/radar`.
46 For more information see @ref egomotion_mainsection.
48 ------------------------------------------------------------------
50 @section pg_icp Iterative Closest Point
52 ICP has been completelly moved to point cloud processing module.
54 For more information see @ref pointcloudprocessing_usecase5.
56 ------------------------------------------------------------------
58 @section pg_imageprocessing Image Processing
60 #### Port `dwPyramidHandle_t` to `dwPyramidImage`
61 | Old name | New name |
62 | ---------- | ---------- |
63 | dwPyramid_initialize() | dwPyramid_create() |
64 | dwPyramid_release() | dwPyramid_destroy() |
65 | dwPyramid_build() | dwImageFilter_computePyramid() |
66 | dwPyramid_getLevelCount() | dwPyramidImage::levelCount |
68 `dwPyramidImage::levelImages[level]` provides an `::dwImageHandle_t` for the (level)-th image, `dwImage_getProperties()` acquires the image size, and `dwImage_getCUDA()` acquires `dwImageCUDA` data.
71 Old code for creating, building and destroying a pyramid:
74 dwPyramidConfig config{};
76 config.height = height;
77 config.levelCount = numLevel;
78 config.dataType = DW_TYPE_UINT8;
79 dwPyramid_initialize(&pyramid, &config, 0, context);
80 dwPyramid_build(&singlePlaneImage, pyramid);
81 dwPyramid_release(pyramid);
84 New code for creating, building and destroying a pyramid:
87 dwPyramid_create(&pyramid, numLevel, width, height, DW_TYPE_UINT8, context);
88 dwImageFilter_computePyramid(&pyramid, &singlePlaneImage, 0, context);
89 dwPyramid_destroy(pyramid);
92 #### Port `dwFeatureListHandle_t` and `dwFeatureListPointers` to `dwFeatureHistoryArray`.
93 - `dwFeatureStatus` has been renamed to `::dwFeature2DStatus`.
94 - `DW_FEATURE_STATUS_XXX` has been renamed to `DW_FEATURE2D_STATUS_XXX`.
96 The new `dwFeatureHistoryArray` struct provides pointers such as: `*statuses`, `*ages`, `*ids`, `*locationHistory` and `*featureCount` like the old `dwFeatureListPointers` struct did. It is also created and destroyed like the old handle `dwFeatureListHandle_t`. `dwFeatureHistoryArray` can be created on page-able/pinned CPU or CUDA memory.
98 | Old name | New name |
99 | ---------- | ---------- |
100 | dwFeatureListPointers::statuses | dwFeatureHistoryArray::statuses |
101 | dwFeatureListPointers::ages | dwFeatureHistoryArray::ages |
102 | --- | dwFeatureHistoryArray::ids |
103 | dwFeatureListPointers::locationHistory | dwFeatureHistoryArray::locationHistory |
104 | dwFeatureListPointers::featureCount | dwFeatureHistoryArray::featureCount |
105 | dwFeatureList_initialize() | dwFeatureHistoryArray_create() |
106 | dwFeatureList_release() | dwFeatureHistoryArray_destroy() |
107 | dwFeatureList_reset() | dwFeatureHistoryArray_reset() |
108 | dwFeatureList_getDataBasePointer() | dwFeatureHistoryArray::data, dwFeatureHistoryArray::bytes |
109 | dwFeatureList_getCurrentTimeIdx() | dwFeatureHistoryArray::currentTimeIdx |
110 | dwFeatureList_setCurrentTimeIdx() | dwFeatureHistoryArray::currentTimeIdx |
111 | dwFeatureList_getFeatureListSize() | dwFeatureHistoryArray::maxFeatures, dwFeatureHistoryArray::maxHistory |
113 For new `dw_imageprocessing` APIs, there's no need to call `selectValid()`, `compact()` and `proximityFilter()` explicitly. They are automatically called within the DriveWorks module.
116 In the old `dw_feature` APIs, the application needed to create a `dwFeatureListHandle_t` object and call `dwFeatureList_getDataPointers()` to get the interpretable `dwFeatureListPointers`.
119 dwFeatureList_initialize(&featureList, maxFeatureCount, historyCapacity, width, height, 0, context);
122 dwFeatureListPointers featureDataGPU;
123 dwFeatureList_getDataBasePointer(&d_database8, &databytes, featureList);
124 dwFeatureList_getDataPointers(&featureDataGPU, d_database8, featureList));
126 // access the pointers of dwFeatureListPointers on GPU
127 featureDataGPU.statuses[...]
128 featureDataGPU.locationHistory[...]
131 If the application needed a CPU copy of the `featureList` handle, it first had to allocate CPU memory and call `dwFeatureList_getDataPointers()` to interpret it into `dwFeatureListPointers`.
134 uint8_t* h_database8 = new uint8_t[databytes];
135 dwFeatureListPointers featureDataCPU;
136 dwFeatureList_getDataPointers(&featureDataCPU, h_database8, featureList));
138 // access the pointers of dwFeatureListPointers on CPU
139 featureDataCPU.statuses[...]
140 featureDataCPU.locationHistory[...]
143 It also needed the raw data pointer to copy data between the CPU and GPU.
146 cudaMemcpy(h_database8, d_database8, databytes, cudaMemcpyDeviceToHost);
149 cudaMemcpy(d_database8, h_database8, databytes, cudaMemcpyHostToDevice);
152 In the new `dw_imageprocessing` APIs, the application only needs to create the `::dwFeatureHistoryArray` object on the CPU and GPU. It provides both raw pointers and interpretable pointers.
153 It also provides the `dwFeatureHistoryArray_copyAsync()` API to easily copy data between two feature history arrays.
156 dwFeatureHistoryArray_create(&featureHistoryCPU, maxFeatureCount, historyCapacity, DW_MEMORY_TYPE_CPU, context);
157 dwFeatureHistoryArray_create(&featureHistoryGPU, maxFeatureCount, historyCapacity, DW_MEMORY_TYPE_CUDA, context);
159 // access the pointers on GPU
160 featureHistoryGPU.statuses[...]
161 featureHistoryGPU.locationHistory[...]
163 // access the pointers on CPU
164 featureHistoryCPU.statuses[...]
165 featureHistoryCPU.locationHistory[...]
168 dwFeatureHistoryArray_copyAsync(&featureHistoryCPU, &featureHistoryGPU, 0);
171 dwFeatureHistoryArray_copyAsync(&featureHistoryGPU, &featureHistoryCPU, 0);
174 The application can also create `dwFeatureHistoryArray` on pinned memory, it only needs to change `::DW_MEMORY_TYPE_CPU` to `::DW_MEMORY_TYPE_PINNED` during creation.
176 Like `dwFeatureListHandle_t`, there are also `reset()` and `destroy()` APIs. `::dwFeatureHistoryArray` must be destroyed before the application exits.
179 dwFeatureHistoryArray_destroy(featureHistoryCPU);
180 dwFeatureHistoryArray_destroy(featureHistoryGPU);
183 #### Get particular `timeIdx` slice from `dwFeatureHistoryArray`
184 Similar to `dwFeatureListPointers::locationHistory`, `dwFeatureHistoryArray::locationHistory` stores `historyCapacity` time period of `maxFeatureCount` features. To get a particular `feature[i]` for `timeIdx`, it used to be:
187 uint32_t currentTimeIdx;
188 dwFeatureList_getCurrentTimeIdx(¤tTimeIdx, featureList);
190 uint32_t histIdx = (currentTimeIdx + timeIdx) % historyCapacity;
191 dwVector2f* pLocation = featureDataCPU.locationHistory + histIdx * maxFeatureCount;
193 // get the particular feature[i]
197 The new `dw_imageprocessing` modules provides a `dwFeatureArray` struct that contains features for a single time slice, so there's no need to compute `histIdx` any more.
200 dwFeatureArray features;
201 dwFeatureHistoryArray_get(&features, timeIdx, &featureHistoryCPU);
203 // get the particular feature[i]
204 features.locations[i]...
207 It can also get the most recent (top-most) time slice, or the slice that is 1 frame earlier than the top-most.
210 dwFeatureArray currentFeatures, previousFeatures;
211 dwFeatureHistoryArray_getCurrent(¤tFeatures, &featureHistoryCPU);
212 dwFeatureHistoryArray_getPrevious(&previousFeatures, &featureHistoryCPU);
214 // get the particular feature[i]
215 currentFeatures.locations[i]...
216 previousFeatures.locations[i]...
219 `dwFeatureArray` acquired by `dwFeatureHistoryArray_getXXX` APIs don't need to be destroyed as they are only shallow copies. Resources are automatically freed when the corresponding `dwFeatureHistoryArray` is destroyed. However, `dwFeatureArray` also provides creation APIs, where the usage is very similar to `dwFeatureHistoryArray`:
222 dwFeatureArray_create(&featureCPU, maxFeatureCount, DW_MEMORY_TYPE_CPU, context);
223 dwFeatureArray_create(&featureGPU, maxFeatureCount, DW_MEMORY_TYPE_CUDA, context);
226 `dwFeatureArray` created by `dwFeatureArray_create()` APIs must be destroyed by `dwFeatureArray_destroy()` before the application exits.
227 Similarly, there is the `dwFeatureArray_copyAsync()` API to copy data between two feature array objects.
229 #### Port `dwFeatureTrackerHandle_t` to `dwFeature2DDetectorHandle_t` and `dwFeature2DTrackerHandle_t`
230 ##### Harris corner detector handle `dwFeature2DDetectorHandle_t`
231 New imageprocessing module splits the Harris corner detector from `dwFeatureTrackerHandle_t`. The new Harris detector handle is `::dwFeature2DDetectorHandle_t`.
233 | Old name | New name |
234 | ---------- | ---------- |
235 | dwFeatureTracker_initialize() | dwFeature2DDetector_initialize() |
236 | dwFeatureTracker_initDefaultParams() | dwFeature2DDetector_initDefaultParams() |
237 | dwFeatureTracker_reset() | dwFeature2DDetector_reset() |
238 | dwFeatureTracker_release() | dwFeature2DDetector_release() |
239 | dwFeatureTracker_setMask() | dwFeature2DDetector_setMask() |
240 | dwFeatureTracker_detectNewFeatures() | dwFeature2DDetector_detectFromPyramid() |
241 | --- | dwFeature2DDetector_detectFromImage() |
243 The new `::dwFeature2DDetectorConfig` parameter not only covers all settings from the old `dwFeatureTrackerConfig` parameter, it also provides more flexible settings.
245 | Old dwFeatureTrackerConfig | New dwFeature2DDetectorConfig |
246 | ---------- | ---------- |
248 | dwFeatureTrackerConfig::imageWidth | dwFeature2DDetectorConfig::imageWidth |
249 | dwFeatureTrackerConfig::imageHeight | dwFeature2DDetectorConfig::imageHeight |
250 | dwFeatureTrackerConfig::maxFeatureCount | dwFeature2DDetectorConfig::maxFeatureCount |
251 | --- | dwFeature2DDetectorConfig::harrisK |
252 | dwFeatureTrackerConfig::cellSize | dwFeature2DDetectorConfig::cellSize |
253 | dwFeatureTrackerConfig::detectorScoreThreshold | dwFeature2DDetectorConfig::scoreThreshold |
254 | dwFeatureTrackerConfig::detectorDetailThreshold | dwFeature2DDetectorConfig::detailThreshold |
255 | dwFeatureTrackerConfig::numEvenDistributionPerCell | dwFeature2DDetectorConfig::numEvenDistributionPerCell |
256 | --- | dwFeature2DDetectorConfig::detectionLevel |
257 | --- | dwFeature2DDetectorConfig::harrisRadius |
258 | --- | dwFeature2DDetectorConfig::NMSRadius |
259 | --- | dwFeature2DDetectorConfig::maskType |
260 | --- | dwFeature2DDetectorConfig::isMaskAdjustmentEnabled |
262 Example of initializing a detector with the old code:
265 // Detector and Tracker share the same dwFeatureTrackerHandle_t
266 dwFeatureTrackerConfig trackerConfig{};
267 dwFeatureTracker_initDefaultParams(&trackerConfig);
268 trackerConfig.imageWidth = imageWidth;
269 trackerConfig.imageHeight = imageHeight;
270 trackerConfig.maxFeatureCount = maxFeatureCount;
271 trackerConfig.cellSize = cellSize;
272 trackerConfig.detectorScoreThreshold = scoreThreshold;
273 trackerConfig.detectorDetailThreshold = detailThreshold;
274 trackerConfig.numEvenDistributionPerCell = numEvenDistributionPerCell;
276 dwFeatureTracker_initialize(&tracker, &trackerConfig, 0, context);
279 Example of initializing a detector with the new code:
282 dwFeature2DDetectorConfig detectorConfig = {};
283 dwFeature2DDetector_initDefaultParams(&detectorConfig);
284 detectorConfig.imageWidth = imageWidth;
285 detectorConfig.imageHeight = imageHeight;
286 detectorConfig.maxFeatureCount = maxFeatureCount;
287 detectorConfig.cellSize = cellSize;
288 detectorConfig.scoreThreshold = scoreThreshold;
289 detectorConfig.detailThreshold = detailThreshold;
290 detectorConfig.numEvenDistributionPerCell = numEvenDistributionPerCell;
292 dwFeature2DDetector_initialize(&detector, &detectorConfig, 0, context);
295 ##### LK tracker handle `dwFeature2DTrackerHandle_t`
296 The new imageprocessing module splits LK tracker from `dwFeatureTrackerHandle_t`. The new LK tracker handle is `::dwFeature2DTrackerHandle_t`.
298 | Old name | New name |
299 | ---------- | ---------- |
300 | dwFeatureTracker_initialize() | dwFeature2DTracker_initialize() |
301 | dwFeatureTracker_initDefaultParams() | dwFeature2DTracker_initDefaultParams() |
302 | dwFeatureTracker_reset() | dwFeature2DTracker_reset() |
303 | dwFeatureTracker_release() | dwFeature2DTracker_release() |
304 | dwFeatureTracker_trackFeatures() | dwFeature2DTracker_trackFeatures() |
305 | --- | dwFeature2DTracker_getNewToOldMap() |
307 The new `::dwFeature2DTrackerConfig` parameter not only covers all settings from the the old `dwFeatureTrackerConfig` parameter, it also provides more flexible settings.
309 | Old dwFeatureTrackerConfig | New dwFeature2DTrackerConfig |
310 | ---------- | ---------- |
311 | --- | dwFeature2DTrackerConfig::algorithm |
312 | --- | dwFeature2DTrackerConfig::detectorType |
313 | --- | dwFeature2DTrackerConfig::isOutputCompact |
314 | dwFeatureTrackerConfig::imageWidth | dwFeature2DTrackerConfig::imageWidth |
315 | dwFeatureTrackerConfig::imageHeight | dwFeature2DTrackerConfig::imageHeight |
316 | dwFeatureTrackerConfig::maxFeatureCount | dwFeature2DTrackerConfig::maxFeatureCount |
317 | --- | dwFeature2DTrackerConfig::historyCapacity |
318 | dwFeatureTrackerConfig::windowSizeLK | dwFeature2DTrackerConfig::windowSizeLK |
319 | dwFeatureTrackerConfig::iterationsLK | dwFeature2DTrackerConfig::numIterTranslationOnly |
320 | --- | dwFeature2DTrackerConfig::numIterScaling |
321 | --- | dwFeature2DTrackerConfig::numLevelTranslationOnly |
322 | --- | dwFeature2DTrackerConfig::maxScaleChange |
323 | --- | dwFeature2DTrackerConfig::nccKillThreshold |
324 | --- | dwFeature2DTrackerConfig::nccUpdateThreshold |
326 Example of initializing a tracker with the old code:
329 // Detector and Tracker share the same dwFeatureTrackerHandle_t
330 dwFeatureTrackerConfig trackerConfig{};
331 dwFeatureTracker_initDefaultParams(&trackerConfig);
332 trackerConfig.imageWidth = imageWidth;
333 trackerConfig.imageHeight = imageHeight;
334 trackerConfig.maxFeatureCount = maxFeatureCount;
335 trackerConfig.windowSizeLK = windowSize;
336 trackerConfig.iterationsLK = numIterTranslation;
338 dwFeatureTracker_initialize(&tracker, &trackerConfig, 0, context);
341 Example of initializing a tracker with the new code:
344 dwFeature2DTrackerConfig trackerConfig = {};
345 dwFeature2DTracker_initDefaultParams(&trackerConfig);
346 trackerConfig.detectorType = detectorConfig.type;
347 trackerConfig.maxFeatureCount = maxFeatureCount;
348 trackerConfig.historyCapacity = historyCapacity;
349 trackerConfig.imageWidth = imageWidth;
350 trackerConfig.imageHeight = imageHeight;
351 trackerConfig.windowSizeLK = windowSize;
352 trackerConfig.numIterTranslationOnly = numIterTranslation;
354 dwFeature2DTracker_initialize(&tracker, &trackerConfig, 0, context);
357 ##### Use the new `imageprocessing` module to detect and track Harris corners
359 Here's a complete example showing the pipeline with the old `dw_feature` APIs and new `dw_imageprocessing` APIs.
362 // Old dw_features code
364 // Create feature list
365 dwFeatureListHandle_t featureList;
366 dwFeatureList_initialize(&featureList, maxFeatureCount, historyCapacity, width, height, 0, context);
369 dwPyramidHandle_t pyramidCurrent, pyramidPrevious;
370 dwPyramidConfig config{};
371 config.width = width;
372 config.height = height;
373 config.levelCount = numLevel;
374 config.dataType = DW_TYPE_UINT8;
375 dwPyramid_initialize(&pyramidCurrent, &config, cudaStream, context);
376 dwPyramid_initialize(&pyramidPrevious, &config, cudaStream, context);
378 // create tracker and detector
379 dwFeatureTrackerConfig trackerConfig{};
380 dwFeatureTracker_initDefaultParams(&trackerConfig);
381 trackerConfig.imageWidth = width;
382 trackerConfig.imageHeight = height;
383 trackerConfig.maxFeatureCount = maxFeatureCount;
384 dwFeatureTracker_initialize(&tracker, &trackerConfig, cudaStream, context);
386 // Old dw_feature APIs requires additional CUDA buffers for compact.
387 uint32_t *d_validCount, *d_validIndexes, *d_invalidCount, *d_invalidIndexes;
388 cudaMalloc(&d_validCount, sizeof(uint32_t));
389 cudaMalloc(&d_validIndexes, sizeof(uint32_t) * maxFeatureCount);
390 cudaMalloc(&d_invalidCount, sizeof(uint32_t));
391 cudaMalloc(&d_invalidIndexes, sizeof(uint32_t) * maxFeatureCount);
393 // Assuming sensor provides single channel image as input
394 while (image = camera->getFrame())
396 std::swap(pyramidCurrent, pyramidPrevious);
397 dwPyramid_build(&image, pyramidCurrent);
400 dwFeatureTracker_trackFeatures(featureList, pyramidPrevious, pyramidCurrent, nullptr, tracker);
402 // Apply proximity filters to make features uniformly distributed
403 dwFeatureList_proximityFilter(featureList);
405 // Determine which features to throw away
406 dwFeatureList_selectValid(d_validCount, d_validIndexes,
407 d_invalidCount, d_invalidIndexes, featureList);
410 dwFeatureList_compact(featureList, d_validCount, d_validIndexes,
411 d_invalidCount, d_invalidIndexes);
414 dwFeatureTracker_detectNewFeatures(featureList, pyramidCurrent, tracker);
418 dwFeatureTracker_release(tracker);
419 dwPyramid_release(pyramidCurrent);
420 dwPyramid_release(pyramidPrevious);
421 dwFeatureList_release(featureList);
423 cudaFree(d_validCount);
424 cudaFree(d_validIndexes);
425 cudaFree(d_invalidCount);
426 cudaFree(d_invalidIndexes);
431 // New dw_imageprocessing code
433 // New dw_imageprocessing APIs requires a GPU dwFeatureArray as the output of Harris corner detector.
434 // It also requires a GPU dwFeatureHistoryArray as output of LK tracker.
435 dwFeatureArray featureDetectedGPU;
436 dwFeatureArray_create(&featureDetectedGPU, maxFeatureCount, DW_MEMORY_TYPE_CUDA, context);
438 dwFeatureHistoryArray featureHistoryGPU;
439 dwFeatureHistoryArray_create(&featureHistoryGPU, maxFeatureCount, historyCapacity, DW_MEMORY_TYPE_CUDA, context);
442 dwPyramidImage pyramidCurrent, pyramidPrevious;
443 dwPyramid_create(&pyramidCurrent, numLevel, width, height, DW_TYPE_UINT8, context);
444 dwPyramid_create(&pyramidPrevious, numLevel, width, height, DW_TYPE_UINT8, context);
447 dwFeature2DDetectorConfig detectorConfig = {};
448 dwFeature2DDetector_initDefaultParams(&detectorConfig);
449 detectorConfig.imageWidth = imageWidth;
450 detectorConfig.imageHeight = imageHeight;
451 detectorConfig.maxFeatureCount = maxFeatureCount;
452 dwFeature2DDetector_initialize(&detector, &detectorConfig, cudaStream, context);
455 dwFeature2DTrackerConfig trackerConfig = {};
456 dwFeature2DTracker_initDefaultParams(&trackerConfig);
457 trackerConfig.detectorType = detectorConfig.type;
458 trackerConfig.maxFeatureCount = maxFeatureCount;
459 trackerConfig.historyCapacity = historyCapacity;
460 trackerConfig.imageWidth = imageWidth;
461 trackerConfig.imageHeight = imageHeight;
462 dwFeature2DTracker_initialize(&tracker, &trackerConfig, cudaStream, context);
464 // Assuming sensor provides single channel image as input, all handles and buffers are initialized.
465 while (image = camera->getFrame())
467 std::swap(pyramidCurrent, pyramidPrevious);
468 dwImageFilter_computePyramid(&pyramidCurrent, &imageY, cudaStream, context);
471 dwFeatureArray featurePredicted;
472 dwFeature2DTracker_trackFeatures(&featureHistoryGPU, &featurePredicted, nullptr,
473 &featureDetectedGPU, nullptr,
474 &pyramidPrevious, &pyramidCurrent, tracker);
477 dwFeature2DDetector_detectFromPyramid(&featureDetectedGPU, &pyramidCurrent,
478 &featurePredicted, nullptr, detector);
482 dwFeature2DDetector_release(detector);
483 dwFeature2DTracker_release(tracker);
485 dwFeatureHistoryArray_destory(featureHistoryGPU);
486 dwFeatureArray_destroy(featureDetectedGPU);
487 dwPyramid_destroy(pyramidCurrent);
488 dwPyramid_destroy(pyramidPrevious);
492 ##### Advanced detector and tracker
493 The new `::dwFeature2DDetectorHandle_t` and `::dwFeature2DTrackerHandle_t` provides advanced detecting/tracking options. To enable advanced mode, the application needs to set the detector type as `::DW_FEATURE2D_DETECTOR_TYPE_EX` and the tracker algorithm as `::DW_FEATURE2D_TRACKER_ALGORITHM_EX`:
495 dwFeature2DDetectorConfig detectorConfig = {};
496 dwFeature2DDetector_initDefaultParams(&detectorConfig);
497 detectorConfig.type = DW_FEATURE2D_DETECTOR_TYPE_EX;
498 // ... Other settings
501 dwFeature2DTrackerConfig trackerConfig = {};
502 dwFeature2DTracker_initDefaultParams(&trackerConfig);
503 trackerConfig.algorithm = DW_FEATURE2D_TRACKER_ALGORITHM_EX;
504 trackerConfig.detectorType = detectorConfig.type;
505 // ... Other settings
507 Please refer to documentation for more details regarding available configuration.
509 Advanced mode also requires an additional device buffer `nccScore`. It can be left as `nullptr`, but a valid `nccScore` buffer is recommended for better tracking/detecting quality.
511 float32_t* nccScore = nullptr;
512 cudaMalloc(&nccScore, sizeof(float32_t) * maxFeatureCount);
514 // Initialization code
517 // Start detecting/tracking loop
518 while (image = camera->getFrame())
520 std::swap(pyramidCurrent, pyramidPrevious);
521 dwImageFilter_computePyramid(&pyramidCurrent, &imageY, cudaStream, context);
524 dwFeatureArray featurePredicted;
525 dwFeature2DTracker_trackFeatures(&featureHistoryGPU, &featurePredicted, nccScore,
526 &featureDetectedGPU, nullptr,
527 &pyramidPrevious, &pyramidCurrent, tracker);
530 dwFeature2DDetector_detectFromPyramid(&featureDetectedGPU, &pyramidCurrent,
531 &featurePredicted, nccScore, detector);
535 The `nccScore` buffer is recommended only when both the detector and tracker are in advanced mode. It's valid to connect the `::DW_FEATURE2D_DETECTOR_TYPE_STD` normal detector with the `::DW_FEATURE2D_TRACKER_ALGORITHM_EX` advanced tracker, or to connect the `::DW_FEATURE2D_DETECTOR_TYPE_EX` advanced detector with the `::DW_FEATURE2D_TRACKER_ALGORITHM_STD` advanced tracker. The application only needs to set `::dwFeature2DTrackerConfig::detectorType` with the correct value of its corresponding detector.
537 ##### Working with SFM APIs
538 Below are the recommended steps for using `imageprocessing` APIs in SFM:
539 1. Set `dwFeature2DTrackerConfig:.algorithm = ::DW_FEATURE2D_TRACKER_ALGORITHM_SFM` to allow sparse tracking results output.
540 2. Perform tracking and reconstruction. DO NOT detect new features immediately after tracking.
541 3. Call `dwFeature2DTracker_compact()` explicitly after SFM reconstruction is done. Fill the `d_newToOldMap` device array by `dwFeature2DTracker_getNewToOldMap()`.
542 4. Call `dwReconstructor_compact*` by `d_newToOldMap` received in step 3.
543 5. Detect new features when `compact` is done.
545 Pseudo code for a typical workflow:
548 dwFeature2DTracker_trackFeatures();
554 dwFeature2DTracker_compact();
555 dwFeature2DTracker_getNewToOldMap();
557 dwReconstructor_compactFeatureHistory();
558 dwReconstructor_compactWorldPoints();
560 // Detect new features
561 dwFeature2DDetector_detectFromPyramid();
565 For the full implementation refer to @ref dwx_struct_from_motion_sample.
567 #### Port `dwScalingFeatureListHandle_t` and `dwScalingFeatureListPointers` to `dwTemplateArray`
568 Similar to `::dwFeatureHistoryArray`, template tracker provides a `::dwTemplateArray` to replace the old `dwScalingFeatureListHandle_t` and `dwScalingFeatureListPointers`.
570 The new `::dwTemplateArray` struct provides pointers such as: `*bboxes`, `*statuses`, `*ids`, `*ages`, `*scaleFactors` and `*templateCount` like the old `dwScalingFeatureListPointers` did. It is also created and destroyed like old handle `dwScalingFeatureListHandle_t`. `::dwTemplateArray` can be created on page-able/pinned CPU or CUDA memory.
572 | Old name | New name |
573 | ---------- | ---------- |
574 | dwScalingFeatureListPointers::locations | dwTemplateArray::bboxes |
575 | dwScalingFeatureListPointers::sizes | dwTemplateArray::bboxes |
576 | dwScalingFeatureListPointers::statuses | dwTemplateArray::statuses |
577 | dwScalingFeatureListPointers::ids | dwTemplateArray::ids |
578 | dwScalingFeatureListPointers::ages | dwTemplateArray::ages |
579 | dwScalingFeatureListPointers::scaleFactors | dwTemplateArray::scaleFactors |
580 | dwScalingFeatureListPointers::bNewTemplate | --- |
581 | dwScalingFeatureListPointers::featureCount | dwTemplateArray::templateCount |
582 | dwScalingFeatureList_initialize() | dwTemplateArray_create() |
583 | dwScalingFeatureList_release() | dwTemplateArray_destroy() |
584 | dwScalingFeatureList_reset() | dwTemplateArray_reset() |
585 | dwScalingFeatureList_getDataBasePointer() | dwTemplateArray::data, dwTemplateArray::bytes |
587 For new `dw_imageprocessing` APIs, there's no need to call `addEmptyFeature`, `selectValid()`, `compact()` and `applySizeFilter()` explicitly. They are automatically called within the DriveWorks module.
590 In the old `dw_feature` APIs, the application needed to create a `dwScalingFeatureListHandle_t` object and call `dwScalingFeatureList_getDataPointers()` to get the interpretable `dwScalingFeatureListPointers`.
593 dwScalingFeatureList_initialize(&templateList, 0, maxTemplateCount, pxlType, context);
596 dwScalingFeatureListPointers templateDataGPU;
597 dwScalingFeatureList_getDataBasePointer(&d_database8, &databytes, templateList);
598 dwScalingFeatureList_getDataPointers(&templateDataGPU, d_database8, templateList));
600 // access the pointers of dwFeatureListPointers on GPU
601 templateDataGPU.locations[...]
602 templateDataGPU.sizes[...]
605 If the application needed a CPU copy of the `templateList` handle, it first had to allocate CPU memory and call `dwScalingFeatureList_getDataPointers()` to interpret it into `dwScalingFeatureListPointers`.
608 uint8_t* h_database8 = new uint8_t[databytes];
609 dwScalingFeatureListPointers templateDataCPU;
610 dwScalingFeatureList_getDataPointers(&templateDataCPU, h_database8, templateList));
612 // access the pointers of dwScalingFeatureListPointers on CPU
613 templateDataCPU.locations[...]
614 templateDataCPU.sizes[...]
617 It also needed the raw data pointer to copy data between the CPU and GPU.
620 cudaMemcpy(h_database8, d_database8, databytes, cudaMemcpyDeviceToHost);
623 cudaMemcpy(d_database8, h_database8, databytes, cudaMemcpyHostToDevice);
626 In the new `dw_imageprocessing` APIs, the application only needs to create the `::dwTemplateArray` object on the CPU and GPU. It provides both raw pointers and interpretable pointers.
627 It also provides the `dwTemplateArray_copyAsync()` API to easily copy data between two template arrays.
630 dwTemplateArray_create(&templateCPU, maxFeatureCount, DW_MEMORY_TYPE_CPU, context);
631 dwTemplateArray_create(&templateGPU, maxFeatureCount, DW_MEMORY_TYPE_CUDA, context);
633 // access the pointers on GPU
634 templateGPU.statuses[...]
635 templateGPU.bboxes[...]
637 // access the pointers on CPU
638 templateCPU.statuses[...]
639 templateCPU.bboxes[...]
642 dwTemplateArray_copyAsync(&templateCPU, &templateGPU, 0);
645 dwTemplateArray_copyAsync(&templateGPU, &templateCPU, 0);
648 The application can also create `::dwTemplateArray` on pinned memory, it only needs to change `::DW_MEMORY_TYPE_CPU` to `::DW_MEMORY_TYPE_PINNED` during creation.
650 Like `dwScalingFeatureListHandle_t`, there are also `reset()` and `destroy()` APIs. `::dwTemplateArray` must be destroyed before the application exits.
653 dwTemplateArray_destroy(templateCPU);
654 dwTemplateArray_destroy(templateGPU);
657 #### Port `dwScalingFeatureTrackerHandle_t` to `dwTemplateTrackerHandle_t`
658 | Old name | New name |
659 | ---------- | ---------- |
660 | dwScalingFeatureTracker_initialize() | dwTemplateTracker_initialize() |
661 | dwScalingFeatureTracker_initDefaultParams() | dwTemplateTracker_initDefaultParams() |
662 | dwScalingFeatureTracker_reset() | dwTemplateTracker_reset() |
663 | dwScalingFeatureTracker_release() | dwTemplateTracker_release() |
664 | dwScalingFeatureTracker_trackAsync() | dwTemplateTracker_trackImage() |
665 | --- | dwTemplateTracker_trackPyramid() |
666 | --- | dwFeature2DTracker_getNewToOldMap() |
668 For new `dw_imageprocessing` APIs, there's no need to call `dwScalingFeatureTracker_updateTemplateAsync()` explicitly. It is automatically called within the DriveWorks module.
670 The new `dwTemplateTrackerParameters` parameter covers all settings from the old `dwScalingFeatureTrackerConfig` and `dwScalingFeatureListParameters` parameters.
672 | Old dwScalingFeatureXXX settings | New dwTemplateTrackerParameters |
673 | ---------- | ---------- |
674 | --- | dwTemplateTrackerParameters::type |
675 | dwScalingFeatureListParameters::maxFeatureCount | dwTemplateTrackerParameters::maxTemplateCount |
676 | dwScalingFeatureListParameters::imageWidth | dwTemplateTrackerParameters::imageWidth |
677 | dwScalingFeatureListParameters::imageHeight | dwTemplateTrackerParameters::imageHeight |
678 | dwScalingFeatureListParameters::pyramidLevelCount | dwTemplateTrackerParameters::maxPyramidLevel |
679 | dwScalingFeatureTrackerConfig::numIterCoarse | dwTemplateTrackerParameters::numIterationsCoarse |
680 | dwScalingFeatureTrackerConfig::numIterFine | dwTemplateTrackerParameters::numIterations |
681 | dwScalingFeatureTrackerConfig::thresholdUpdate | dwTemplateTrackerParameters::thresholdUpdate |
682 | dwScalingFeatureTrackerConfig::thresholdKill | dwTemplateTrackerParameters::thresholdKill |
683 | dwScalingFeatureTrackerConfig::thresholdStop | dwTemplateTrackerParameters::thresholdStop |
684 | dwScalingFeatureTrackerConfig::maxScaleChange | dwTemplateTrackerParameters::maxScaleChange |
685 | --- | dwTemplateTrackerParameters::validWidth |
686 | --- | dwTemplateTrackerParameters::validHeight |
688 Here's a complete example showing the template tracker pipeline with the old `dw_feature` APIs and new `dw_imageprocessing` APIs.
693 // Old dw_features code
695 // Create template list
696 dwScalingFeatureListHandle_t templateList;
697 dwScalingFeatureList_initialize(&templateList, cudaStream, maxTemplateCount, DW_TYPE_UINT8, context);
700 dwPyramidHandle_t pyramidCurrent, pyramidPrevious;
701 dwPyramidConfig config{};
702 config.width = width;
703 config.height = height;
704 config.levelCount = numLevel;
705 config.dataType = DW_TYPE_UINT8;
706 dwPyramid_initialize(&pyramidCurrent, &config, cudaStream, context);
707 dwPyramid_initialize(&pyramidPrevious, &config, cudaStream, context);
709 // create tracker and detector
710 dwScalingFeatureTrackerConfig trackerConfig{};
711 dwScalingFeatureTracker_initDefaultParams(&trackerConfig);
712 dwScalingFeatureTracker_initialize(&tracker, &trackerConfig, cudaStream, context);
714 // Old dw_feature APIs requires additional CUDA buffers for compact.
715 uint32_t *d_validCount, *d_validIndexes, *d_invalidCount, *d_invalidIndexes;
716 cudaMalloc(&d_validCount, sizeof(uint32_t));
717 cudaMalloc(&d_validIndexes, sizeof(uint32_t) * maxFeatureCount);
718 cudaMalloc(&d_invalidCount, sizeof(uint32_t));
719 cudaMalloc(&d_invalidIndexes, sizeof(uint32_t) * maxFeatureCount);
721 // add bboxes to track, assuming there're nBoxToTrack, bounding box info is stored in hBBoxes array
722 uint8_t* d_database8;
724 dwScalingFeatureList_getDataBasePointer(&d_database8, &databytes, templateList);
726 uint8_t* h_database8 = new uint8_t[databytes];
727 dwScalingFeatureListPointers templateDataCPU;
728 dwScalingFeatureList_getDataPointers(&templateDataCPU, h_database8, templateList));
730 // assuming there're nBoxToTrack stored in hBBoxes array
731 for (uint32_t i = 0; i < nBoxToTrack; i++)
733 templateDataCPU.locations[i] = {hBBoxes[i].x, hBBoxes[i].y};
734 templateDataCPU.sizes[i] = {hBBoxes[i].width, hBBoxes[i].height};
735 templateDataCPU.statuses[i] = DW_FEATURE_STATUS_DETECTED;
736 templateDataCPU.ages[i] = 1;
737 templateDataCPU.bNewTemplate[i] = true;
739 *templateDataCPU.featureCount = nBoxToTrack;
740 cudaMemcpy(d_database8, h_database8, databytes, cudaMemcpyHostToDevice);
741 dwScalingFeatureList_addEmptyFeature(nBoxToTrack, dwScalingFeatureListHandle_t obj);
743 // Assuming sensor provides single channel image as input
744 while (image = camera->getFrame())
746 std::swap(pyramidCurrent, pyramidPrevious);
747 dwPyramid_build(&image, pyramidCurrent);
749 dwImageCUDA *imageCurrent, *imagePrevious;
750 dwPyramid_getLevelImageCUDA(&imageCurrent, 0, pyramidCurrent);
751 dwPyramid_getLevelImageCUDA(&imagePrevious, 0, pyramidPrevious);
754 dwScalingFeatureTracker_updateTemplateAsync(templateList, imagePrevious, tracker);
757 dwScalingFeatureTracker_trackAsync(templateList, imageCurrent, tracker);
759 // Apply proximity filters to make features uniformly distributed
760 dwScalingFeatureList_applySizeFilter(maxWidth, maxHeight, templateList);
762 // Determine which features to throw away
763 dwScalingFeatureList_selectValid(d_validCount, d_validIndexes,
764 d_invalidCount, d_invalidIndexes, templateList);
767 dwScalingFeatureList_compact(templateList, d_validCount, d_validIndexes,
768 d_invalidCount, d_invalidIndexes);
773 delete[] h_database8;
774 dwScalingFeatureList_release(featureList);
775 dwScalingFeatureTracker_release(tracker);
776 dwPyramid_release(pyramidCurrent);
777 dwPyramid_release(pyramidPrevious);
779 cudaFree(d_validCount);
780 cudaFree(d_validIndexes);
781 cudaFree(d_invalidCount);
782 cudaFree(d_invalidIndexes);
785 ##### new dw_imageprocessing
788 // New dw_imageprocessing code
790 // create template array
791 dwTemplateArray templateGPU, templateCPU;
792 dwTemplateArray_create(&templateGPU, maxTemplateCount, DW_MEMORY_TYPE_CUDA, context);
793 dwTemplateArray_create(&templateCPU, maxTemplateCount, DW_MEMORY_TYPE_CPU, context);
796 dwPyramidImage pyramidCurrent, pyramidPrevious;
797 dwPyramid_create(&pyramidCurrent, numLevel, width, height, DW_TYPE_UINT8, context);
798 dwPyramid_create(&pyramidPrevious, numLevel, width, height, DW_TYPE_UINT8, context);
800 // create template tracker
801 dwTemplateTrackerParameters trackerConfig = {};
802 dwTemplateTracker_initDefaultParams(&trackerConfig);
803 trackerConfig.maxTemplateCount = maxTemplateCount;
804 trackerConfig.imageWidth = width;
805 trackerConfig.imageHeight = height;
806 trackerConfig.validWidth = maxWidth;
807 trackerConfig.validHeight = maxHeight;
808 dwTemplateTracker_initialize(&tracker, &trackerConfig, cudaStream, context);
810 // assuming there're nBoxToTrack stored in hBBoxes array
811 for (uint32_t i = 0; i < nBoxToTrack; i++)
813 templateCPU.bboxes[i] = hBBoxes[i];
814 templateCPU.statuses[i] = DW_FEATURE2D_STATUS_DETECTED;
815 templateCPU.ages[i] = 1;
817 *templateCPU.templateCount = nBoxToTrack;
818 dwTemplateArray_copyAsync(&templateGPU, &templateCPU, 0);
820 // Assuming sensor provides single channel image as input, all handles and buffers are initialized.
821 while (image = camera->getFrame())
823 std::swap(pyramidCurrent, pyramidPrevious);
824 dwImageFilter_computePyramid(&pyramidCurrent, &imageY, cudaStream, context);
826 dwImageCUDA *imageCurrent, *imagePrevious;
827 dwImage_getCUDA(&imageCurrent, pyramidCurrent.levelImages[0]);
828 dwImage_getCUDA(&imagePrevous, pyramidPrevious.levelImages[0]);
831 dwTemplateTracker_trackImage(&templateGPU, imageCurrent, imagePrevious, tracker);
835 dwTemplateArray_destroy(templateGPU);
836 dwTemplateArray_destroy(templateCPU);
837 dwTemplateTracker_release(tracker);
839 dwPyramid_destroy(pyramidCurrent);
840 dwPyramid_destroy(pyramidPrevious);
843 ------------------------------------------------------------------
845 @section pg_lanegraph Lane Graph
847 The APIs previously contained in `dw/world/LaneGraphProducer.h` are no longer available,
848 while the data structures are now part of `LaneGraph.h`.
850 ------------------------------------------------------------------
852 @section pg_laneperception Lane Perception
854 The following data structures have been moved from `LaneDetector.h` to `LandmarkDetectorTypes.h`:
855 - ::dwLanePositionType.
859 ------------------------------------------------------------------
861 @section pg_lidaraccumulator Lidar Accumulator
863 The following API has been removed:
865 dwStatus dwLidarAccumulator_hasFullSweepAvailable(bool* available, dwLidarAccumulatorHandle_t obj);
868 `dwLidarAccumulator_addPacket()` will return `DW_SUCCESS` if the full sweep is ready.
870 For more information see @ref lidaraccumulator_mainsection.
872 ------------------------------------------------------------------
874 @section pg_maps Maps
876 The following APIs have been renamed:
878 | Old name | New name |
879 |----------|----------|
880 | dwMaps_transformLaneDividersToLocalLinesPerLocalLayout() | dwMaps_transformLaneDividersToLocalLines() |
881 | dwMaps_transformRoadFeaturesToLocalLinesPerLocalLayout() | dwMaps_transformRoadFeaturesToLocalLines() |
882 | dwMaps_computeBearing() | dwMaps_computeBearingFromGeoPoints() |
883 | dwMaps_computeBearingFromRotMatrix() | dwMaps_computeBearingFromRotation() |
884 | dwMaps_computeLocalToENUMatrix() | dwMaps_computeRotationFromBearing() |
885 | dwMapTracker_updateCurrentPose() | dwMapTracker_updateWithGlobalPose() |
887 The following API has been removed:
889 dwStatus dwMaps_transformLaneDividersToLocalLines(dwMapsLaneDividerLineBuffer* lineSegments,
890 const dwMapsLaneDividerBuffer* laneDividers,
891 const dwMapsGeoPoint* localOrigin,
892 const dwMatrix3d* localToENURotation33,
893 const dwMapsLocalBounds* bounds,
894 const dwVector3f* directionFilterVector,
895 float32_t directionFilterAngleRadian,
896 bool ignoreLaneDirection);
898 dwMaps_transformLaneDividersToLocalLines() should be used instead.
900 For more information see @ref maps_mainsection.
902 ------------------------------------------------------------------
904 @section pg_objectdetection Object Detection
906 The dwObjectDetector API has been simplified. The functions
908 - `dwObjectDetector_inferDeviceAsync()`
909 - `dwObjectDetector_interpretHost()`
910 - `dwObjectDetector_getClusteredObjects()`
911 - `dwObjectDetector_getRawDetections()`
912 - `dwObjectDetector_bindInput()`
913 - `dwObjectDetector_bindOutput()`
914 - `dwObjectDetector_processDeviceAsync()`
915 - `dwObjectDetector_processHost()`
917 have been removed in favor of `dwObjectDetector_detectObjects()`.
919 For more information see @ref object_mainsection.
921 ------------------------------------------------------------------
923 @section pg_objects Objects
925 The following data structures have been renamed:
927 | Old name | New name |
928 |----------|----------|
929 | dwObjectData | dwObjectGeneric |
930 | dwObjectDataCamera | dwObjectCamera |
931 | dwObjectDataRadar | dwObjectRadar |
932 | dwObjectDataLidar | dwObjectLidar |
933 | dwObjectDataFused | dwObjectFused |
935 dwObjectHandle_t has been removed and replaced with plain structs. In place of dwObjectHandleList, use dwObjectArray. Becase object handles are no longer part of the code base, the following APIs have been removed:
939 | dwStatus dwObject_createCamera(dwObjectHandle_t* handle, const dwObjectData* data, const dwObjectDataCamera* otherData)|
940 | dwStatus dwObject_createRadar(dwObjectHandle_t* handle, const dwObjectData* data, const dwObjectDataRadar* otherData)|
941 | dwStatus dwObject_createLidar(dwObjectHandle_t* handle, const dwObjectData* data, const dwObjectDataLidar* otherData)|
942 | dwStatus dwObject_createFused(dwObjectHandle_t* handle, const dwObjectData* data, const dwObjectDataFused* otherData)|
943 | dwStatus dwObject_destroy(dwObjectHandle_t handle)|
944 | dwStatus dwObject_reset(dwObjectHandle_t handle)|
945 | dwStatus dwObject_getData(dwObjectData* info, uint32_t index, dwConstObjectHandle_t handle)|
946 | dwStatus dwObject_getDataCamera(dwObjectDataCamera* info, uint32_t index, dwConstObjectHandle_t handle)|
947 | dwStatus dwObject_getDataRadar(dwObjectDataRadar* info, uint32_t index, dwConstObjectHandle_t handle)|
948 | dwStatus dwObject_getDataLidar(dwObjectDataLidar* info, uint32_t index, dwConstObjectHandle_t handle)|
949 | dwStatus dwObject_getDataFused(dwObjectDataFused* info, uint32_t index, dwConstObjectHandle_t handle)|
950 | dwStatus dwObject_setData(const dwObjectData* info, uint32_t index, dwObjectHandle_t handle)|
951 | dwStatus dwObject_setDataCamera(const dwObjectDataCamera* info, uint32_t index, dwObjectHandle_t handle)|
952 | dwStatus dwObject_setDataRadar(const dwObjectDataRadar* info, uint32_t index, dwObjectHandle_t handle)|
953 | dwStatus dwObject_setDataLidar(const dwObjectDataLidar* info, uint32_t index, dwObjectHandle_t handle)|
954 | dwStatus dwObject_setDataFused(const dwObjectDataFused* info, uint32_t index, dwObjectHandle_t handle)|
955 | dwStatus dwObject_addDataCamera(const dwObjectData* info, const dwObjectDataCamera* otherInfo, dwObjectHandle_t handle)|
956 | dwStatus dwObject_addDataRadar(const dwObjectData* info, const dwObjectDataRadar* otherInfo, dwObjectHandle_t handle)|
957 | dwStatus dwObject_addDataLidar(const dwObjectData* info, const dwObjectDataLidar* otherInfo, dwObjectHandle_t handle)|
958 | dwStatus dwObject_addDataFused(const dwObjectData* info, const dwObjectDataFused* otherInfo, dwObjectHandle_t handle)|
960 Because dwObjectHandle_t is no longer part of the code base, calling create to create an object is no longer necessary. For example, the old way of creating an object required calling:
963 dwObjectHandle_t objHandle = DW_NULL_HANDLE;
965 dwObjectDataCamera cameraData{};
966 dwObject_createCamera(&objHandle, &data, &cameraData);
969 The new way to do this is written as:
972 dwObjectCamera obj{};
975 Object history is no longer stored as part of the object, but is now a separate history array: `dwObjectHistoryArray`. The old way to get a historical object from an object handle was:
978 uint32_t historicalIndex = 10; // Arbitrary historical index, 0 being the most recent
979 dwObject_getData(&data, historicalIndex, objHandle);
981 The new way to do this is written as:
984 uint32_t historicalIndex = 10;
985 uint32_t objectIndex = 0; // This would be the index of the specific object in the history array
986 dwObjectArray objArray{};
988 dwObjectCamera* objects = static_cast<dwObjectCamera*>(objArray.objects);
989 objects[historicalIndex * objArray.maxObjectCount + objectIndex];
992 In order to add data to the history, you no longer call an API but use the indices into the objects array to insert into the history table.
994 Object arrays have also been introduced as `dwObjectArray`. These work very similarly to `dwObjectHistoryArray` except that it is a 1D contiguous array of objects.
996 ------------------------------------------------------------------
998 @section pg_pcp Point Cloud Processing
1000 `dwPointCloudMemoryType` has been replaced by the following data structure:
1003 `dwPointCloudRangeImage` structure has been removed. Range image creator now
1004 accepts `dwImageHandle_t` instead.
1006 For more information see @ref pointcloudprocessing_mainsection.
1008 ------------------------------------------------------------------
1010 @section pg_rigconfiguration Rig Configuration
1012 The module `dwRigConfiguration` has been renamed `dwRig`.
1014 - Header file has been renamed from `RigConfiguration.h` to `Rig.h`.
1015 - The object handles have been renamed to `dwRigHandle_t` and `dwConstRigHandle_t`, respectively.
1016 - The prefixes for all functions declared in `Rig.h` have been updated from `dwRigConfiguration` to `dwRig`.
1017 - `dwRig_initializeCalibratedCamera` is now `dwCameraModel_initialize` in `calibration/cameramodel/CameraModel.h`.
1019 ------------------------------------------------------------------
1022 - Feature status has been changed from `dwFeatureStatus` to `dwFeature2DStatus`.
1023 - Prototype for `dwReconstructor_compactWorldPoints` and `dwReconstructor_compactFeatureHistory` has changed. The input has changed from `const uint32_t* d_validIndexCount, const uint32_t* d_validIndexes, const uint32_t* d_invalidIndexCount, const uint32_t* d_invalidIndexes` to `const uint32_t* d_validIndexCount, const uint32_t* d_newToOldMap`, while `d_newToOldMap` is obtained with `dwFeature2DTracker_getNewToOldMap`.
1024 - Please refer to the "Working with SFM APIs" section in @ref pg_imageprocessing for how to work with new `dw_imageprocessing` APIs.
1026 For more information see @ref sfm_mainsection and @ref imageprocessing_features_mainsection.
1028 ------------------------------------------------------------------
1030 @section pg_vehicleio VehicleIO
1032 This release adds support for multiple simultanous VehicleIO backends, and the following changes are required:
1034 - `dwVehicleIO_addCANSensor()` has to be called before send/consume operations.
1035 - @ref vehicleio_mainsection can automatically detect to which backend to send commands to, therefore `dwVehicleIO_sendVehicleCommand()` can be just be replaced with `dwVehicleIO_sendCommand()`.
1036 - The same is not true when receiving messages and `dwVehicleIO_consume()` needs be replaced with `dwVehicleIO_consumeCANFrame()` with `sensorId` equal to the sensor added with `dwVehicleIO_addCANSensor()`.
1038 For more information see @ref vehicleio_mainsection.
1040 ------------------------------------------------------------------
1042 @section pg_waitconditions Wait Conditions
1044 The previous API header file `waitcondition/Classifier.h` has been split into `waitcondition/camera/TrafficLightSignClassifier.h` and `waitcondition/camera/TrafficLightSignClassifier_processPipeline.h`.
1046 - `dwObjectClassifierHandle_t` has been renamed to `dwTrafficLightSignClassifierHandle_t`.
1048 The following APIs have been renamed:
1050 | Old name | New name |
1051 |----------|----------|
1052 | dwObjectClassifier_initializeFromLightNet(dwObjectClassifierHandle_t* obj, dwLightNetHandle_t dnn, dwContextHandle_t ctx) | dwTrafficLightSignClassifier_initializeFromLightNet(dwTrafficLightSignClassifierHandle_t* obj, dwLightNetHandle_t dnn, dwContextHandle_t ctx)|
1053 | dwObjectClassifier_initializeFromSignNet(dwObjectClassifierHandle_t* obj, dwSignNetHandle_t dnn, dwContextHandle_t ctx) | dwTrafficLightSignClassifier_initializeFromSignNet(dwTrafficLightSignClassifierHandle_t* obj, dwSignNetHandle_t dnn, dwContextHandle_t ctx) |
1054 | dwObjectClassifier_reset(dwObjectClassifierHandle_t obj) | dwTrafficLightSignClassifier_reset(dwTrafficLightSignClassifierHandle_t obj) |
1055 | dwObjectClassifier_release(dwObjectClassifierHandle_t obj) | dwTrafficLightSignClassifier_release(dwTrafficLightSignClassifierHandle_t obj) |
1056 | dwObjectClassifier_setCUDAStream(cudaStream_t stream, dwObjectClassifierHandle_t obj) | dwTrafficLightSignClassifier_setCUDAStream(cudaStream_t stream, dwTrafficLightSignClassifierHandle_t obj) |
1057 | dwObjectClassifier_getCUDAStream(cudaStream_t* stream, dwObjectClassifierHandle_t obj) | dwTrafficLightSignClassifier_getCUDAStream(cudaStream_t* stream, dwTrafficLightSignClassifierHandle_t obj) |
1058 | dwObjectClassifier_interpretHost(uint32_t numObjects, dwObjectClass type, dwObjectClassifierHandle_t obj)| dwTrafficLightSignClassifier_interpretHost(uint32_t numObjects, dwObjectClass type, dwTrafficLightSignClassifierHandle_t obj) |
1059 | dwObjectClassifier_inferDeviceAsync(const dwImageCUDA* const* imageArray, dwObject* objectList, uint32_t numObjects, dwObjectClass type, dwObjectClassifierHandle_t obj) | dwTrafficLightSignClassifier_inferDeviceAsync(const dwImageCUDA* const* imageArray, dwObjectCamera* objectList, uint32_t numObjects, dwObjectClass type, dwTrafficLightSignClassifierHandle_t obj) |
1060 | dwObjectClassifier_getClassifiedObjects(dwObject* objectList, uint32_t numObjects, dwObjectClassifierHandle_t obj) | dwTrafficLightSignClassifier_getClassifiedObjects(dwObjectCamera* objectList, uint32_t numObjects, dwTrafficLightSignClassifierHandle_t obj) |
1062 For more information see @ref waitcondition_mainsection.