Defines the PilotNet detector module to utilize DNN based PilotNet path predictor.
Data Structures | |
struct | dwPilotNetDetectorInputSensor |
Holds the camera properties for PilotNetDetector. More... | |
struct | dwPilotNetDetectorOutput |
Holds the output trajectories and other predictions of PilotNetDetector. More... | |
struct | dwPilotNetDetectorParams |
Holds the parameters to initialize the PilotNetDetector module. More... | |
struct | dwPilotNetDetectorState |
Holds the input state values for PilotNetDetector. More... | |
struct | dwPilotNetLaneDivider |
Holds the left and right lane dividers for a single lane type. More... | |
Macros | |
#define | DW_PILOTNET_MAX_CAMERA_COUNT 7 |
The maximum supported ROI Views. More... | |
#define | DW_PILOTNET_MAX_ROI_VIEWS DW_PILOTNET_MAX_CAMERA_COUNT* DW_PILOTNET_MAX_ROI_VIEWS_PER_CAMERA |
#define | DW_PILOTNET_MAX_ROI_VIEWS_PER_CAMERA 3 |
#define | DW_PILOTNET_MAX_TRAJ_LEN 200 |
The maximum trajectory length. More... | |
Typedefs | |
typedef struct dwPilotNetDetectorObject * | dwPilotNetDetectorHandle_t |
Handle for PilotNetDetector. More... | |
Enumerations | |
enum | dwPilotNetCameraType { DW_PILOTNET_FRONT_CENTER_60FOV, DW_PILOTNET_FRONT_CENTER_120FOV, DW_PILOTNET_FRONT_LEFT_120FOV, DW_PILOTNET_FRONT_RIGHT_120FOV, DW_PILOTNET_CROSS_LEFT_120FOV, DW_PILOTNET_CROSS_RIGHT_120FOV } |
The cameras supported by the PilotNetDetector module. More... | |
enum | dwPilotNetDrivingMode { DW_PILOTNET_LANE_STABLE, DW_PILOTNET_LANE_CHANGE_LEFT_FIRST_HALF, DW_PILOTNET_LANE_CHANGE_LEFT_SECOND_HALF, DW_PILOTNET_LANE_CHANGE_RIGHT_FIRST_HALF, DW_PILOTNET_LANE_CHANGE_RIGHT_SECOND_HALF, DW_PILOTNET_LANE_SPLIT_LEFT, DW_PILOTNET_LANE_SPLIT_RIGHT, DW_PILOTNET_TURN_LEFT, DW_PILOTNET_TURN_RIGHT, DW_PILOTNET_LANE_LEFT, DW_PILOTNET_LANE_RIGHT, DW_PILOTNET_MAX_DRIVING_MODES } |
Defines the different driving trajectories produced by the PilotNet network. More... | |
Functions | |
DW_API_PUBLIC dwStatus | dwPilotNetDetector_bindOutput (dwPilotNetDetectorOutput *output, dwPilotNetDetectorHandle_t pilotnet) |
Bind the memory of the output to a pre-allocated memory in the application. More... | |
DW_API_PUBLIC dwStatus | dwPilotNetDetector_convertPointsRigToSensor (dwVector2f *points, size_t numPoints, const dwVector3f *trajPoints, size_t numTrajPoints, uint32_t sensorId, dwPilotNetDetectorHandle_t pilotnet) |
Get the trajectory points based on the network output in image space. More... | |
DW_API_PUBLIC dwStatus | dwPilotNetDetector_getAvailableDrivingModes (bool *modes, dwPilotNetDetectorHandle_t pilotnet) |
Getting supported driving modes from PilotNetDetector. More... | |
DW_API_PUBLIC dwStatus | dwPilotNetDetector_getDrivingModeString (const char **string, dwPilotNetDrivingMode drivingMode) |
Getting the driving mode enum as a string. More... | |
DW_API_PUBLIC dwStatus | dwPilotNetDetector_getPatchU8 (const dwImageCUDA **dwImg, uint32_t *numROI, uint32_t sensorId, dwPilotNetDetectorHandle_t pilotnet) |
Get a pointer to the CUDA patch buffer. More... | |
DW_API_PUBLIC dwStatus | dwPilotNetDetector_getROIParams (dwRectf *roi, uint32_t *numROI, uint32_t sensorId, dwPilotNetDetectorHandle_t pilotnet) |
Get the parameters for the region of interest (ROI) calculated by dwPilotNetDetector_processFrames. More... | |
DW_API_PUBLIC dwStatus | dwPilotNetDetector_getVisMaskU8 (const dwImageCUDA **dwImg, uint32_t *numROI, uint32_t sensorId, dwPilotNetDetectorHandle_t pilotnet) |
Gets a pointer to the CUDA visualization mask buffer. More... | |
DW_API_PUBLIC dwStatus | dwPilotNetDetector_infer (dwPilotNetDetectorHandle_t pilotnet) |
Runs the inference on the patch produced by dwPilotNetDetector_processFrames to output the driving trajectory. More... | |
DW_API_PUBLIC dwStatus | dwPilotNetDetector_initialize (dwPilotNetDetectorHandle_t *pilotnet, const dwPilotNetDetectorParams *params, dwPilotNetHandle_t pilotnetDNN) |
Creates and initializes the PilotNetDetector module with the given dwPilotNetHandle that is used for predicting driving commands for a vehicle based on an input image. More... | |
DW_API_PUBLIC dwStatus | dwPilotNetDetector_initParams (dwPilotNetDetectorParams *params, dwPilotNetHandle_t pilotnetDNN) |
Initializes the PilotNetDetector module params with the given dwPilotNetHandle. More... | |
DW_API_PUBLIC dwStatus | dwPilotNetDetector_processFrames (dwPilotNetDetectorHandle_t pilotnet) |
Processes the input image(s) by calculating a patch for the region of interest (ROI) by compensating for calibration and other pre-processing based on the model params. More... | |
DW_API_PUBLIC dwStatus | dwPilotNetDetector_processOutput (dwPilotNetDetectorHandle_t pilotnet) |
Gets the driving state (mode, commands) for the last processed image. More... | |
DW_API_PUBLIC dwStatus | dwPilotNetDetector_release (dwPilotNetDetectorHandle_t pilotnet) |
Releases the PilotNetDetector module. More... | |
DW_API_PUBLIC dwStatus | dwPilotNetDetector_reset (dwPilotNetDetectorHandle_t pilotnet) |
Resets the PilotNetDetector module. More... | |
DW_API_PUBLIC dwStatus | dwPilotNetDetector_setCameraFrame (const dwImageCUDA *inputFrame, uint32_t sensorId, dwPilotNetDetectorHandle_t pilotnet) |
Sets the input camera frame for inferencing. More... | |
DW_API_PUBLIC dwStatus | dwPilotNetDetector_setSensorExtrinsics (const dwTransformation3f *tx, const uint32_t sensorId, dwPilotNetDetectorHandle_t pilotnet) |
Setting PilotNetDetector extrinsic camera calibration parameters . More... | |
DW_API_PUBLIC dwStatus | dwPilotNetDetector_setState (dwPilotNetDetectorState inputState, dwPilotNetDetectorHandle_t pilotnet) |
Sets the driving state for the current frame. More... | |
struct dwPilotNetDetectorInputSensor |
Data Fields | ||
---|---|---|
dwPilotNetCameraType | camera | PilotNetDetector camera type. |
dwTransformation3f | cameraExtrinsics | Camera Extrinsics. |
dwCameraModelHandle_t | cameraModel | Camera model. |
float32_t | fps | Update rate of the data from this sensor. |
dwRect | inDimensions | Input Image dimensions of the data from this sensor. |
uint32_t | sensorId | Sensor ID of the sensor from the vehicle rig. |
cudaStream_t | stream | Stream to be used for a particular sensor. |
struct dwPilotNetDetectorOutput |
Data Fields | ||
---|---|---|
float32_t | cameraPitch | Predicted camera pitch. |
bool | cameraPitchValid | If set to true, camera pitch is valid. |
dwTime_t | commandTimeStamp | Timestamp when the output values are filled in. |
float32_t | confidence[DW_PILOTNET_MAX_DRIVING_MODES][DW_PILOTNET_MAX_TRAJ_LEN] |
Confidence for each trajectory point represented as a number in [0,1]. 0 represents no confidence, and 1.0 represents 100% confidence. |
dwTime_t | inputTimestamp | Timestamp of the input image. |
dwPilotNetLaneDivider | laneDividers[DW_PILOTNET_MAX_DRIVING_MODES] | Lane Divider outputs in car coordinates. |
float32_t | lcStatus[DW_PILOTNET_MAX_DRIVING_MODES] | CNN output indicating the progress of lane change, left (pos) and right (neg). |
bool | lcStatusValid[DW_PILOTNET_MAX_DRIVING_MODES] | If set to true, lcStatus is valid. |
float32_t | rotation | Off-orientation position (yaw) of vehicle in degrees. |
bool | rotationValid | If set to true, off-orientation output is valid. |
dwVector3f | trajectory[DW_PILOTNET_MAX_DRIVING_MODES][DW_PILOTNET_MAX_TRAJ_LEN] | Trajectory output in car coordinates. |
size_t | trajectoryLen[DW_PILOTNET_MAX_DRIVING_MODES] | Trajectory length. |
bool | trajectoryValid[DW_PILOTNET_MAX_DRIVING_MODES] | Trajectory validity. |
struct dwPilotNetDetectorParams |
Data Fields | ||
---|---|---|
uint32_t | numSensors | PilotNetDetector camera type. |
dwPilotNetDetectorInputSensor | sensors[DW_PILOTNET_MAX_CAMERA_COUNT] | PilotNetDetector Sensor params. |
struct dwPilotNetDetectorState |
struct dwPilotNetLaneDivider |
Data Fields | ||
---|---|---|
dwVector3f | leftLaneDivider[DW_PILOTNET_MAX_TRAJ_LEN] | Left lane divider. |
bool | leftLaneDividerValid | Left lane divider validity. |
dwVector3f | rightLaneDivider[DW_PILOTNET_MAX_TRAJ_LEN] | Right lane divider. |
bool | rightLaneDividerValid | Right lane divider validity. |
#define DW_PILOTNET_MAX_CAMERA_COUNT 7 |
The maximum supported ROI Views.
Definition at line 65 of file PilotNetDetector.h.
#define DW_PILOTNET_MAX_ROI_VIEWS DW_PILOTNET_MAX_CAMERA_COUNT* DW_PILOTNET_MAX_ROI_VIEWS_PER_CAMERA |
Definition at line 67 of file PilotNetDetector.h.
#define DW_PILOTNET_MAX_ROI_VIEWS_PER_CAMERA 3 |
Definition at line 66 of file PilotNetDetector.h.
#define DW_PILOTNET_MAX_TRAJ_LEN 200 |
The maximum trajectory length.
Definition at line 62 of file PilotNetDetector.h.
typedef struct dwPilotNetDetectorObject* dwPilotNetDetectorHandle_t |
Handle for PilotNetDetector.
Definition at line 219 of file PilotNetDetector.h.
enum dwPilotNetCameraType |
The cameras supported by the PilotNetDetector module.
The nominal extrensic values and tolerances of each camera is mentioned for each camera type
Definition at line 93 of file PilotNetDetector.h.
Defines the different driving trajectories produced by the PilotNet network.
Definition at line 73 of file PilotNetDetector.h.
DW_API_PUBLIC dwStatus dwPilotNetDetector_bindOutput | ( | dwPilotNetDetectorOutput * | output, |
dwPilotNetDetectorHandle_t | pilotnet | ||
) |
Bind the memory of the output to a pre-allocated memory in the application.
[in] | output | The address to store the pilotNet result. |
[in] | pilotnet | Handle for the PilotNetDetector module. |
DW_API_PUBLIC dwStatus dwPilotNetDetector_convertPointsRigToSensor | ( | dwVector2f * | points, |
size_t | numPoints, | ||
const dwVector3f * | trajPoints, | ||
size_t | numTrajPoints, | ||
uint32_t | sensorId, | ||
dwPilotNetDetectorHandle_t | pilotnet | ||
) |
Get the trajectory points based on the network output in image space.
[out] | points | Pointer to the array holding the coordinates in image space. |
[in] | numPoints | Number of trajectory points to plot. |
[in] | trajPoints | points from network output. |
[in] | numTrajPoints | of trajectory points from network output. |
[in] | sensorId | Sensor Id according to car rig. |
[in] | pilotnet | Handle for the PilotNetDetector module. |
DW_API_PUBLIC dwStatus dwPilotNetDetector_getAvailableDrivingModes | ( | bool * | modes, |
dwPilotNetDetectorHandle_t | pilotnet | ||
) |
Getting supported driving modes from PilotNetDetector.
[out] | modes | Boolean indicator for supported PilotNetDetector modes. Pre-allocated array of size DW_PILOTNET_MAX_DRIVING_MODES. |
[in] | pilotnet | Handle for PilotNetDetector module. |
DW_API_PUBLIC dwStatus dwPilotNetDetector_getDrivingModeString | ( | const char ** | string, |
dwPilotNetDrivingMode | drivingMode | ||
) |
Getting the driving mode enum as a string.
[out] | string | String of the enum if available, else nullptr |
[in] | drivingMode | Driving mode enum to be converted to string |
DW_API_PUBLIC dwStatus dwPilotNetDetector_getPatchU8 | ( | const dwImageCUDA ** | dwImg, |
uint32_t * | numROI, | ||
uint32_t | sensorId, | ||
dwPilotNetDetectorHandle_t | pilotnet | ||
) |
Get a pointer to the CUDA patch buffer.
[out] | dwImg | Pointer to an array of image pointers to return the ROIpatch used by PilotNetDetector for inference. |
[out] | numROI | Number of ROI patches present. |
[in] | sensorId | Sensor Id according to car rig. |
[in] | pilotnet | Handle for PilotNetDetector module |
DW_API_PUBLIC dwStatus dwPilotNetDetector_getROIParams | ( | dwRectf * | roi, |
uint32_t * | numROI, | ||
uint32_t | sensorId, | ||
dwPilotNetDetectorHandle_t | pilotnet | ||
) |
Get the parameters for the region of interest (ROI) calculated by dwPilotNetDetector_processFrames.
The area is an approximate rectangle enclosing the ROI with coordinates normalized with the dimensions of the input image.
[out] | roi | Pointer to an array of rectangle that defines the ROI. |
[out] | numROI | Number of ROI in the given sensor. |
[in] | sensorId | Sensor Id according to car rig. |
[in] | pilotnet | Handle for the PilotNetDetector module. |
DW_API_PUBLIC dwStatus dwPilotNetDetector_getVisMaskU8 | ( | const dwImageCUDA ** | dwImg, |
uint32_t * | numROI, | ||
uint32_t | sensorId, | ||
dwPilotNetDetectorHandle_t | pilotnet | ||
) |
Gets a pointer to the CUDA visualization mask buffer.
[out] | dwImg | Pointer to an array of image pointers to return the visualization mask generated by PilotNetDetector. |
[out] | numROI | Number of visualization masks present. |
[in] | sensorId | Sensor Id according to car rig. |
[in] | pilotnet | Handle for PilotNetDetector module. |
DW_API_PUBLIC dwStatus dwPilotNetDetector_infer | ( | dwPilotNetDetectorHandle_t | pilotnet | ) |
Runs the inference on the patch produced by dwPilotNetDetector_processFrames to output the driving trajectory.
[in] | pilotnet | Handle for the PilotNetDetector module. |
DW_API_PUBLIC dwStatus dwPilotNetDetector_initialize | ( | dwPilotNetDetectorHandle_t * | pilotnet, |
const dwPilotNetDetectorParams * | params, | ||
dwPilotNetHandle_t | pilotnetDNN | ||
) |
Creates and initializes the PilotNetDetector module with the given dwPilotNetHandle that is used for predicting driving commands for a vehicle based on an input image.
[out] | pilotnet | Pointer to the PilotNetDetector handle. |
[in] | params | Properies created by dwPilotNetDetector_initParams and then filled by the application. |
[in] | pilotnetDNN | Handle to the pilotnet module initialized with the right version and precision. |
DW_API_PUBLIC dwStatus dwPilotNetDetector_initParams | ( | dwPilotNetDetectorParams * | params, |
dwPilotNetHandle_t | pilotnetDNN | ||
) |
Initializes the PilotNetDetector module params with the given dwPilotNetHandle.
[out] | params | Properies of the module based on the model being loaded. This needs to completed before calling dwPilotNetDetector_initialize. |
[in] | pilotnetDNN | Handle to the pilotnet module initialized with the right version and precision. |
DW_API_PUBLIC dwStatus dwPilotNetDetector_processFrames | ( | dwPilotNetDetectorHandle_t | pilotnet | ) |
Processes the input image(s) by calculating a patch for the region of interest (ROI) by compensating for calibration and other pre-processing based on the model params.
[in] | pilotnet | Handle for the PilotNetDetector module. |
DW_API_PUBLIC dwStatus dwPilotNetDetector_processOutput | ( | dwPilotNetDetectorHandle_t | pilotnet | ) |
Gets the driving state (mode, commands) for the last processed image.
[in] | pilotnet | Handle for the PilotNetDetector module. |
DW_API_PUBLIC dwStatus dwPilotNetDetector_release | ( | dwPilotNetDetectorHandle_t | pilotnet | ) |
Releases the PilotNetDetector module.
[in] | pilotnet | Handle of an already initialized PilotNetDetector module to release. |
DW_API_PUBLIC dwStatus dwPilotNetDetector_reset | ( | dwPilotNetDetectorHandle_t | pilotnet | ) |
Resets the PilotNetDetector module.
[in] | pilotnet | Handle for an already initialized PilotNetDetector module to reset. |
DW_API_PUBLIC dwStatus dwPilotNetDetector_setCameraFrame | ( | const dwImageCUDA * | inputFrame, |
uint32_t | sensorId, | ||
dwPilotNetDetectorHandle_t | pilotnet | ||
) |
Sets the input camera frame for inferencing.
[in] | inputFrame | Sensor Image for inferencing with the following properties format: DW_IMAGE_FORMAT_RGB_UINT8_PLANAR, DW_IMAGE_FORMAT_YUV420_UINT8_PLANAR or DW_IMAGE_FORMAT_RGB_FLOAT16_PLANAR memoryLayot: DW_IMAGE_MEMORY_TYPE_PITCH |
[in] | sensorId | sensor ID of the corresponding frame |
[in] | pilotnet | Handle for the PilotNetDetector module. |
DW_API_PUBLIC dwStatus dwPilotNetDetector_setSensorExtrinsics | ( | const dwTransformation3f * | tx, |
const uint32_t | sensorId, | ||
dwPilotNetDetectorHandle_t | pilotnet | ||
) |
Setting PilotNetDetector extrinsic camera calibration parameters .
[in] | tx | Calibration transformation matrix. |
[in] | sensorId | Sensor Id according to car rig. |
[in] | pilotnet | Handle for PilotNetDetector module. |
DW_API_PUBLIC dwStatus dwPilotNetDetector_setState | ( | dwPilotNetDetectorState | inputState, |
dwPilotNetDetectorHandle_t | pilotnet | ||
) |
Sets the driving state for the current frame.
[in] | inputState | Driving state to set in module. |
[in] | pilotnet | Handle for the PilotNetDetector module. |