/dvs/git/dirty/gitlab-master_av/dw/sdk/doc/portingguide/2-2_to_3-0.md

Go to the documentation of this file.

# Copyright (c) 2019-2020 NVIDIA CORPORATION.  All rights reserved.

@page dwx_porting_guide_3_0 Porting Guide from SDK 2.2 to SDK 3.0
@tableofcontents

This document will help you port your applications from DriveWorks SDK 2.2 to DriveWorks SDK 3.0.

@section dwks_and_dws_release For NVIDIA DriveWorks and NVIDIA DRIVE Software Releases

@note The following subsections are applicable for both **NVIDIA DriveWorks** and **NVIDIA DRIVE Software** releases.

------------------------------------------------------------------

@subsection pg3_calibration Calibration

This release has removed the capability to select
`DW_CALIBRATION_CAMERA_SIGNAL_NONE` for camera self-calibration signal
specification, and has added the new option to select
`::DW_CALIBRATION_CAMERA_SIGNAL_DEFAULT` instead, which will automatically select
the calibration signals for a given camera based on it's properties. Specific
signals can still be selected for the non-default case as before.

The lidar self-calibration API `dwCalibrationEngine_addLidarSweep()` has been
replaced with the simpler `dwCalibrationEngine_addLidarPointCloud()` and now
expects a CAN bus index to be passed at initialization with
`dwCalibrationEngine_initializeLidar()`, so that `dwVehicleIOState` data from
this CAN bus provided with the existing `dwCalibrationEngine_addVehicleIOState()`
API is consumed internally. Also, the previously deprecated
`dwCalibrationEngine_addLidarPoses()` API has been removed, the more general
`dwCalibrationEngine_addLidarPose()` API should be used instead.

The IMU self-calibration has been adjusted to not require a dedicated egomotion handle
anymore. Instead, `dwCalibrationEngine_initializeIMU()` requires the index of a CAN bus
from the rig file and the calibration engine initialized by it must be passed `dwVehicleIOState`
data from this CAN bus via `dwCalibrationEngine_addVehicleIOState()`.

The radar self-calibration has been modified to remove the dependency on egomotion. The API 
`dwCalibrationEngine_initializeRadar()` doesn't need an input of egomotion handle any more. 
Instead, it accepts a CAN bus index as an additional input, so that `dwVehicleIOState` 
data from this CAN bus is provided with the existing `dwCalibrationEngine_addVehicleIOState()`.

------------------------------------------------------------------

@subsection pg3_core Core

This release has removed the following previously deprecated APIs:
- `dwContext_getNvMediaDevice`
- `dwContext_setNvMediaDevice`

and the `NvMediaDevice* nvmediaDevice` field of the `::dwContextParameters` struct.

------------------------------------------------------------------

@subsection pg3_dnn DNN

This release removes the following old APIs:
- `dwDNN_initializeTensorRTFromFile`
- `dwDNN_initializeTensorRTFromMemory`

And renames:
- `dwDNN_initializeTensorRTFromFileNew` into `dwDNN_initializeTensorRTFromFile()`
- `dwDNN_initializeTensorRTFromMemoryNew` into `dwDNN_initializeTensorRTFromMemory()`

The signature of `dwDNN_infer()` has been modified, and it accepts now a list of `::dwDNNTensorHandle_t` as input and output.
`dwDNN_inferRaw()` has been introduced to replace the previous `dwDNN_infer` function that works with raw device pointers.

------------------------------------------------------------------

@subsection pg3_dataconditioner DataConditioner Processing Pipeline

This release changes the signature of following APIs from:
- `dwStatus dwDataConditioner_setROI(const dwRect* roi, dwDataConditionerHandle_t obj)`
- `dwStatus dwDataConditioner_getROI(dwRect* roi, dwDataConditionerHandle_t obj)`

to following form that take an additional parameter `inputIdx`:
- `dwStatus dwDataConditioner_setROI(const dwRect* roi, uint32_t inputIdx, dwDataConditionerHandle_t obj)`
- `dwStatus dwDataConditioner_getROI(dwRect* roi, uint32_t inputIdx, dwDataConditionerHandle_t obj)`

`inputIdx` corresponds to the index of image to set/get ROI. Previously all input images share one global ROI, after the
change, each input image slot can have one unique ROI attached.

This release has renamed `dwDataConditioner_prepareData` to `dwDataConditioner_prepareDataRaw`. 

------------------------------------------------------------------

@subsection pg3_egomotion Egomotion

This release has exposed the capability to choose gains for the simple version of the
linear acceleration filter. It exposed `accelerationFilterTimeConst`, `processNoiseStdevSpeed`,
`processNoiseStdevAcceleration`, `measurementNoiseStdevSpeed`, and `measurementNoiseStdevAcceleration`.
These variables should all be set when `dwEgomotionLinearAccelerationFilterMode` is set to
`DW_EGOMOTION_ACC_FILTER_SIMPLE`.

This release has removed the model-based acceleration filter (DW_EGOMOTION_ACC_FILTER_MODELBASED).

This release has modified the following API functions:
- `dwEgomotion_initParamsFromRig()` (`gpsSensorName` has been removed)
- `dwEgomotion_initParamsFromRigByIndex()` (`gpsSensorIdx` has been removed)

This release has removed the following deprecated API functions:

- `dwEgomotion_getHistoryEntry` (please use `dwEgomotion_getHistoryElement()` instead)
- `dwEgomotion_applyRelativeUncertainty`
- `dwEgomotion_addGPSMeasurement` (please use the `dwGlobalEgomotion` module instead)

This release has removed the deprecated `estimationPeriod` parameter in `::dwEgomotionParameters`. Use
the `::dwEgomotionParameters.automaticUpdate` parameter going forward.

This release has removed the following GNSS-related parameters and flags:

- Configuration parameters:
  - `dwEgomotionSensorCharacteristics.gpsSamplingRateHz`
  - `dwEgomotionSensorCharacteristics.gpsHorizontalNoiseMeter`
  - `dwEgomotionSensorCharacteristics.gpsVerticalNoiseMeter`
  - `dwEgomotionParameters.gps2rig`
  - `dwEgomotionParameters.disableLocationFilter`

- Validity flags:
  - `dwEgomotionDataField.DW_EGOMOTION_LOCATION_LON`
  - `dwEgomotionDataField.DW_EGOMOTION_LOCATION_LAT`
  - `dwEgomotionDataField.DW_EGOMOTION_LOCATION_ALT`

- Types:
  - `dwEgomotionReferenceFrame`

- Estimates:
  - `dwEgomotionResult.location`
  - `dwEgomotionResult.rotationFrame`
  - `dwEgomotionUncertainty.location`

This release has added an interpolation and extrapolation limit; requests for interpolation over an
interval larger than 5 seconds or extrapolation of more than 2.5 seconds into the future will return
DW_NOT_AVAILABLE. Affected APIs:
- `dwEgomotion_estimate()`
- `dwEgomotion_computeRelativeTransformation()`


------------------------------------------------------------------

@subsection pg3_image_processing Image Processing

@subsubsection pg3_feature_det_track Feature Detector and Feature Tracker

`dwConnectedComponents_labelImage` has been removed. Instead the following API should be used:

- `dwConnectedComponents_bindInput()`
- `dwConnectedComponents_bindOutputLabels()`
- `dwConnectedComponents_setThreshold()`
- `dwConnectedComponents_process()`

This release removes `dw_features` module, header file `dw/features/Features.h` should be replaced to
`dw/imageprocessing/features/FeatureList.h`, `dw/imageprocessing/features/FeatureDetector.h`
and `dw/imageprocessing/tracking/FeatureTracker.h` respectively.

Refer to @ref dwx_porting_guide_2_0 for API changings.

------------------------------------------------------------------

@subsection pg3_ipc Inter Process Communication

@subsubsection pg3_socketclientserver SocketClientServer

This release has removed the following previously deprecated APIs:
- `dwSocketConnection_send`
- `dwSocketConnection_peek`
- `dwSocketConnection_peekNonBlock`
- `dwSocketConnection_recv`
- `dwSocketConnection_recvNonBlock`

and renamed:
- `dwSocketConnection_peekNew` to `dwSocketConnection_peek()`

------------------------------------------------------------------

@subsection pg3_lidaraccumulator Lidar Accumulator

`dwLidarAccumulator` has been removed. Use `dwPointCloudAccumulator` from `PointCloudAccumulator.h` instead.

------------------------------------------------------------------

@subsection pg3_pointcloudprocessing Point Cloud Processing

The following depracated items have been removed:
- `xyzi` member of `dwPointCloud` structure
- `dwPointCloudAccumulator_bindInput`
- `dwPointCloudStitcher_setMotionCompensation`

The old `dwPointCloudStitcher_bindInput` has been removed and `dwPointCloudStitcher_bindInputToSlot` has been renamed to `dwPointCloudStitcher_bindInput()`.

Refer to @ref dwx_porting_guide_2_0 for more details.

------------------------------------------------------------------

@subsection pg3_rigfile Rig File

This release updates the rig files to `version 6`, converting the rig throttle and brake maps to a torque map.
That is, instead of mapping the inputs to an output force it maps them to a torque value.
The old output force values can be mapped to torque values by multiplying them by the nominal wheel radius.
(Torque = Force x Nominal_Radius)

The fields [`throttleStateInput`, `throttleForceOutput`, `brakeStateInput`, `brakeForceOutput`]
are changed to [`throttlePedalInput`, `throttleSpeedInput`, `throttleTorqueOutput`,
`brakePedalInput`, `brakeTorqueOutput`] under the field `torqueLUT`.

The throttle map is also now a 2-d table, mapping a pedal position at a
given speed to a torque value. As such, the `throttleTorqueOutput` table is of size
(`throttlePedalInput` x `throttleSpeedInput`).

With `VERSION 6`, there is also an additional field `effectiveMass`, providing the effective mass due to vehicle rotational inertia
(wheel rotation, engine, and other parts of the CVT drive-train) in kg.

DriveWorks SDK documentation contains examples for the new `version 6` rig files.

------------------------------------------------------------------

@subsection pg3_sal Sensor Abstraction Layer

@subsubsection pg3_sensors Sensors

This release removes the following previously deprecated APIs:

- `dwSensor_getSeekTable`
- `dwSensorSerializer_serializeImage`
- `dwSensorSerializer_serializeImageAsync`

See wFrameCapture_appendFrame() and dwFrameCapture_appendFrameAsync() to support of the use cases previously covered by `dwSensorSerializer_serializeImage` and `dwSensorSerializer_serializeImageAsync`.

This release also removes support for Pointgrey cameras, therefore camera.pointgrey protocol is deprecated.

This release removes support for dwSoftISP and the isp module all together. Raw cameras and raw videos are to be processed via Tegra ISP, automatically in place when `output-format` contains `processed` and a processed image is requested from the acquired frame handle.

The previously separate concepts of HV (Human Vision) using Tegra ISP and MV (Machine Vision) using dwSoftISP are now surpassed for a unified vision paradigm. DNN (refer to the specific DNNs documentation) are now compatible with the unified paradigm, unless specified otherwise

The equivalent sample common classes CameraRawFramePipeline and RawSimpleCamera are also removed as based on dwSoftISP.

All samples have been stripped of dwSoftISP.

Protocol `camera.gmsl` based on NvMedia's IPP has been moved to `camera.gmsl.ipp` and replaced in functionality by NvSIPL library, see @ref camera_nvsipl_ipp_porting_guide for info

sample_camera_gmsl, sample_camera_gmsl_raw, sample_camera_multiple_gmsl and sample_camera_gmsl_custom are all combined into `sample_camera`. Refer to its README for more details

------------------------------------------------------------------

@subsection pg3_vehicle_and_motion Vehicle and Motion Actuation

@subsubsection pg3_rig Rig

This release has removed the API `dwRig_initializeFromStringWithPath` and integrated
its functionality into `dwRig_initializeFromString()` with a new parameter `relativeBasePath`.
If `relativeBasePath` is `NULL`, then the current working directory of the current process
is used to initialize it. Existing calls to `dwRig_initializeFromString()` can
be updated to pass `NULL` for `relativeBasePath`.

------------------------------------------------------------------

@subsubsection pg3_vehicleio VehicleIO

Following APIs have been removed:
- `dwVehicleIO_consume`
- `dwVehicleIO_sendVehicleCommand`
- `dwVehicleIOCommand::steeringSpeed` has been replaced by  `dwVehicleIOCommand::maxSteeringWheelSpeed` for specifying the maximum steering wheel speed.
- `dwVehicleIOCommand::steeringValid` has been replaced by `dwVehicleIOCommand::steeringWheelValid` for specifying the validity status of steering wheel-based control interface.
- `dwVehicleIOState::steeringCmd` has been replaced by `dwVehicleIOState::steeringWheelAngleCmd` for the steering wheel angle command.
- `dwVehicleIOState::steeringAngle` has been replaced by `dwVehicleIOState::frontSteeringAngle` for the front wheel steering angle. 

------------------------------------------------------------------

@section dws_release_only For NVIDIA DRIVE Software Releases

@note The following subsections are applicable for **NVIDIA DRIVE Software** releases only.

------------------------------------------------------------------

@subsection pg3_driveperception Perception

@subsubsection pg3_ahbnet AhbNet

AhbNet.h has been removed in favor of LightSourceNet.h.

------------------------------------------------------------------

@subsubsection pg3_clearsightnet ClearSightNet

ClearSightNet.h has been moved to `perception/safety/camera/`.

`dwClearSightNet_initDefaultParameters` now requires an additional argument: `::dwContextHandle_t`.

The context handle is required to determine the current GPU architecture so that the default precision is selected accordingly. If the architecture is `Pascal` and a DNN model for this architecture is available, the default precision is set to `FP32`. Otherwise, it is set to `FP16`.

------------------------------------------------------------------

@subsubsection pg3_drivenet DriveNet

DriveNet.h has been moved to `perception/object/camera/`.

`dwDriveNet_initDefaultParameters()` now requires an additional argument: `::dwContextHandle_t`.

The context handle is required to determine the current GPU architecture so that the default precision is selected accordingly. If the architecture is `Pascal` and a DNN model for this architecture is available, the default precision is set to `FP32`. Otherwise, it is set to `FP16`.

------------------------------------------------------------------

@subsubsection pg3_lanenet LaneNet

This module has been removed. Lane detection is now performed by `dwMapNet`.

------------------------------------------------------------------

@subsubsection pg3_lightnet LightNet

LightNet.h has been moved to `perception/object/camera/`.

`dwLightNet_initDefaultParameters` now requires an additional argument: `::dwContextHandle_t`.

The context handle is required to determine the current GPU architecture so that the default precision is selected accordingly. If the architecture is `Pascal` and a DNN model for this architecture is available, the default precision is set to `FP32`. Otherwise, it is set to `FP16`.

-----------------------------------------------------------------

@subsubsection pg3_mapnet MapNet

MapNet.h has been moved to `perception/landmarks/camera/`.

`dwMapNet_initDefaultParamsNew` has been renamed to `dwMapNet_initDefaultParams()`, and now requires additional arguments: `::dwMapNetType` and `::dwContextHandle_t`.

------------------------------------------------------------------

@subsubsection pg3_openroadnet OpenRoadNet

OpenRoadNet.h has been moved to `perception/freespace/camera/`.

`dwOpenRoadNet_initDefaultParameters` now requires an additional argument: `::dwContextHandle_t`.

The context handle is required to determine the current GPU architecture so that the default precision is selected accordingly. If the architecture is `Pascal` and a DNN model for this architecture is available, the default precision is set to `FP32`. Otherwise, it is set to `FP16`.

------------------------------------------------------------------

@subsubsection pg3_pathnet PathNet

PathNet.h has been moved to `perception/path/camera/`.

`dwPathNet_initDefaultParameters` now requires an additional argument: `::dwContextHandle_t`.

The context handle is required to determine the current GPU architecture so that the default precision is selected accordingly. If the architecture is `Pascal` and a DNN model for this architecture is available, the default precision is set to `FP32`. Otherwise, it is set to `FP16`.

------------------------------------------------------------------

@subsubsection pg3_signnet SignNet

SignNet.h has been moved to `perception/object/camera/`.

`dwSignNet_initDefaultParameters` now requires an additional argument: `::dwContextHandle_t`.

The context handle is required to determine the current GPU architecture so that the default precision is selected accordingly. If the architecture is `Pascal` and a DNN model for this architecture is available, the default precision is set to `FP32`. Otherwise, it is set to `FP16`.

------------------------------------------------------------------

@subsubsection pg3_waitnet WaitNet

WaitNet.h has been moved to `perception/object/camera/`.

`dwWaitNet_initDefaultParameters` now requires an additional argument: `::dwContextHandle_t`.

The context handle is required to determine the current GPU architecture so that the default precision is selected accordingly. If the architecture is `Pascal` and a DNN model for this architecture is available, the default precision is set to `FP32`. Otherwise, it is set to `FP16`.

-----------------------------------------------------------------

@subsubsection pg3_landmarkdetector LandmarkDetector

struct `dwLandmarkDetectorOutput` has been removed, please directly use `::dwLaneDetection` and `::dwLandmarkDetection` types to take LandmarkDetector output.
`dwLandmarkDetector_initializeDefaultParamsNew` has been renamed to `dwLandmarkDetector_initializeDefaultParams()`, and now takes frame width and height as additional input.
`dwLandmarkDetector_detectLandmarksNew` has been renamed to `dwLandmarkDetector_detectLandmarks()`, and now takes `::dwLaneDetection` and `::dwLandmarkDetection` as input instead of `dwLandmarkDetectorOutput`.
`dwLandmarkDetector_setDetectionROI`, `dwLandmarkDetector_setDetectionThreshold`, `dwLandmarkDetector_setTemporalSmoothFactor`, `dwLandmarkDetector_getTemporalSmoothFactor`, `dwLandmarkDetector_setModelParameterFitting` functions have all been removed, these are now set as part of `::dwLandmarkDetectorParams`.

------------------------------------------------------------------

@subsection pg3_drivemapping Mapping

@subsubsection pg_maps_3_0 Maps

The object model for map access has been significantly refactored for release 3.0.

Instead of directly initializing a `dwMapHandle_t` with `dwMaps_initialize()` application
should instead first initialize a `::dwMapProviderHandle_t` instance and subsequently use the
Map Provider API to request Maps appropriately for scenario-specific needs.

An important change to the programming model is that Map resources are no longer owned
directly by the application. Maps are only "requested" from the Map Provider and must
be released back to the Map Provider via `dwMapProvider_releaseMap()` instead of
being directly released by `dwMaps_release()`.

Enum `dwMapsLaneGeometry` is changed to have mapping enum entry consistent with
the 'side' concept at `dwSide` eg., DW_MAPS_LANE_GEOMETRY_LANEDIVIDERLEFT = DW_SIDE_LEFT.

`layers/ImageLayer.h` is now available and contains 3 modules:
  - Image Layer module, to work with `dwMapsContentLayerType` of type `DW_MAPS_CONTENT_LAYER_TYPE_IMAGES`
      - Image layer types available defined in enum `dwMapsImageType`
      - `dwMapsImageLayer_checkType` to check if contentLayer is an image type.
      - `dwMapsImageLayer_getProperties` to get a handle to `dwMapsImageLayerProperties` of loaded image layer
  - Image Layer View module to extend the layer views of content layers
      - `dwMapsImageLayerView_getImageCount` get number of image layers in layer view
      - `dwMapsImageLayerView_getLocalImages` get references to image layers from layer view into a buffer
  - Maps Image Layer Storage module for loading and serving maps images
      - `dwMapsImageStorage_initialize` to create a maps image storage handle
      - `dwMapsImageStorage_release` to release a maps image storage handle
      - `dwMapsImageStorage_getMapsImage` to request a specific maps image with a road segment ID
      - `dwMapsImageStorage_returnMapsImage` to release memory for a specific maps image
      - `dwMapsImageStorage_getProperties` to get image properties of the images being served.


DriveWorks SDK documentation contains example for correct Map request and release code
following the Map Provider programming model.

This release also replaces the `dwMapsLaneWaitCondition` structure with `dwMapsLaneWaitElement` structure. 
The structure had limitations in holding required information such as map rules, ego, contender lane information etc.
The new wait element structures are more inline with world model wait element objects and contain more information.

The previous APIs `dwLanePlan_getLaneForNextWaitConditionAt()`, `dwLanePlan_getLaneForNextWaitCondition()` are not 
supported through direct API as the has changed to use via local map.  The usage to get new wait elements is through 
the LocalMap interface `dwLocalMap_initialize()`, `dwLocalMap_bindOutput()`, 
`dwLocalMap_computeLaneGraphFromRelativePose()` and `dwLocalMap_bindWaitElementGroupsBuffer()`. 
Clients can process new wait elements ego lane graph information find the next wait element in the lane plan. 

------------------------------------------------------------------

@subsubsection pg3_localization Localization

This release adds the functionality of localization with radar, lidar, and the fusion of localization
with any combination of camera, radar, and liar sensors. To facilitate these new functions, we have
renamed the following old localization types and functions to be for camera only:

- Configuration parameters:
  - `dwLocalizationParameters` -> `dwLocalizationCameraParameters`

- Functions:
  - `dwLocalization_initParamsFromRig` -> `dwLocalization_cameraInitParamsFromRig()`
  - `dwLocalization_initialize` -> `dwLocalization_cameraInitialize()`
  - `dwLocalization_localize` -> `dwLocalization_cameraLocalize()`
  - `dwLocalization_setCameraExtrinsics` -> `dwLocalization_cameraSetExtrinsics()`

The following functions have been generalized to be used for camera, radar, and lidar localization:
  - `dwLocalization_setMap()`
  - `dwLocalization_reset()`
  - `dwLocalization_release()`
  - `dwLocalization_setCUDAStream()`
  - `dwLocalization_getCUDAStream()`

The following new types and functions have been added:

- Configuration parameters:
  - `dwLocalizationRadarParameters()` for radar localization parameters.
  - `dwLocalizationLidarParameters()` for lidar localization parameters.

- Radar functions:
  - `dwLocalization_radarInitialize()` for initialization of radar localization.
  - `dwLocalization_radarLocalize()` for localization with radar.

- Lidar functions:
  - `dwLocalization_lidarInitialize()` for initialization of lidar localization.
  - `dwLocalization_lidarLocalize()` for localization with lidar.

- Localization fusion:
  - `::dwLocalizationFusionHandle_t` as fusion localization handle.
  - `dwLocalizationFusion_initialize()` for initialization of localization fusion.
  - `dwLocalizationFusion_localize()` for localization fusion.
  - `dwLocalizationFusion_setMap()` for set a new map for localization fusion.
  - `dwLocalizationFusion_reset()` for resetting localization fusion.
  - `dwLocalizationFusion_release()` for release the resource acquired by localization fusion.

We have also renamed, reordered, and added variables in `::dwLocalizationResult` struct:
  - `absolutePos` -> `::dwLocalizationResult.globalPosition`.
  - `absoluteOriQuat` -> `::dwLocalizationResult.globalOrientation`.
  - `relativePos` -> `::dwLocalizationResult.relativePosition`.
  - `relativeOriQuat` -> `::dwLocalizationResult.relativeOrientation`.
  - `isRelativePoseValid` -> `::dwLocalizationResult.isMapAvailable`.
  - `translationUncertainty` -> `::dwLocalizationResult.positionUncertainty`.
  - `rotationUncertainty` -> `::dwLocalizationResult.globalOrientation`.
  - Add `::dwLocalizationResult.indicator` and `::dwLocalizationResult.confidence`.


------------------------------------------------------------------

@subsection pg3_driveplanning Planning

@subsubsection pg3_laneplanner Lane Planner

The `dwLanePlanner_run()` function now accepts a list of target lane pointers instead of a target road segment. To get the same behavior, a list of pointers to the lanes in the road segment can be passed as target lanes:
```
std::vector<const dwMapsLane*> targetLanes;
for (uint32_t laneIdx = 0; laneIdx < targetRS->laneCount; ++laneIdx)
{
    targetLanes.push_back(&targetRoadSegment->lanes[laneIdx]);
}
dwLanePlanner_run(... , targetLanes.data(), static_cast<uint32_t>(targetLanes.size()), ...)
```