The PilotNet module provides the interfaces to use PilotNet, NVIDIA's proprietary end-to-end neural network for path prediction. The PilotNet neural network predicts the optimum path/rail for various driving maneuvers such as lane stable, lane changes, forks, turns, etc. All the predictions are in the world coordinate system. The APIs enable users to choose one of the default models defined in dwPilotNetModel or supply a user-trained PilotNet model. This module is used as an input to the Pilotnet Detector module.
PilotNet consumes a FP32 planar frame(s) from AR0231 camera(s). For best performance use images with an aspect ratio of 8:5 and with a minimum resolution of 960x604 pixels. The pre-processing of images is performed in Pilotnet Detector. Depending on the network, multiple cameras and auxiliary inputs such as speed and pose might also be needed.
PilotNet predicts the trajectory to follow for various driving decisions such as lane stable, lane changes and lane splits. The supported modes are defined in dwPilotNetDrivingMode and the predictions are in world coordinate system. Depending on the model, it also predicts laneDividers, confidence and percentage of lane change completed. All the outputs and their validity are defined in dwPilotNetDetectorOutput struct.
Additionally the network also outputs a visualization mask to highlight the areas of the input image that are of interest to the network.