- Note
- SW Release Applicability: This module is available in NVIDIA DRIVE Software releases.
About This Module
DriveNet is the NVIDIA proprietary object detection deep neural network: DriveNet.
DriveWorks comes with four DriveNet models:
- front camera model: DW_DRIVENET_MODEL_FRONT
- side camera mode: DW_DRIVENET_MODEL_SIDE
- stateful temporal model with urgency prediction: DW_STATEFUL_TEMPORAL_DRIVENET_MODEL
- stateless temporal model with urgency prediction: DW_STATELESS_TEMPORAL_DRIVENET_MODEL
Inputs
DriveNet consumes FP16 planar frames with a resolution of 480x240 (side camera model) or 960x480 (front camera model) pixels from AR0231 cameras (revision >= 4).
- Note
- resizing the input frame is internally handled by Object Detector as part of the normal workflow.
DW_DRIVENET_MODEL_FRONT model is trained to support any of the following six camera configurations:
- Front camera location with a 30° field of view
- Front camera location with a 60° field of view
- Front camera location with a 120° field of view
- Rear camera location with a 60° field of view
- Front-left camera location with a 120° field of view
- Front-right camera location with a 120° field of view
- Rear-left camera location with a 120° field of view
- Rear-right camera location with a 120° field of view
- Warning
- Current models are not fully trained on the 30° field of view camera, resulting in functions being less validated.
DW_DRIVENET_MODEL_SIDE is trained to support side camera location with a 120° field of view
Temporal models (DW_STATEFUL_TEMPORAL_DRIVENET_MODEL and DW_STATELESS_TEMPORAL_DRIVENET_MODEL ) support front camera with 60° field of view.
Outputs
DriveNet allows to detect a range of different objects as 2D bounding boxes.
The following classes are currently supported on all models:
- Car
- Bicycle
- Pedestrian
- Traffic sign
- Traffic light
- Note
- Trucks are also detected, however they are currently classified as cars.
- Motorbikes are classified as bicycles.
Detections coming from DriveNet are associated with:
- a confidence value in the [0,1] range
- a depth value [m]
- an urgency value [1/s] which is the inverse of time to collision
Frame rate is internally considered for the computation of the urgency.
- Note
dwDriveNet_getClassProperties()can be used to query if depth and urgency are available a particular model / class.
Additional Information
While by default inference on DriveNet models is executed on GPUs, it is also possible to configure them to use
the Deep Learning Accelerators available on Drive AGX boards.
- Warning
- Currently, the DriveNet DNN has limitations that could affect its performance:
- It is optimized for daytime, clear-weather data. As a result, it does not perform well in dark or rainy conditions.
- It is trained primarily on data collected in the United States. As a result, it may have reduced accuracy in other locales,
particularly for road sign shapes that do not exist in the U.S.
- Note
- For more information on running inference using DriveNet see Object Perception.