Description

The PilotNet sample demonstrates how to use the NVIDIA^® proprietary deep neural network called PilotNet to perform path prediction. The PilotNet sample streams RAW frames from a video file and runs the PilotNet DNN inference on each frame to predict paths for lane stable, lane change and lane fork maneuvers in world coordinates. Based on the prediction, the sample app plots the trajectories in the image space.

The sample model uses a video captured by a forward-facing Sekonix Camera Module(AR0231 RCCB sensor) with a 60-degree field of view and a resolution of 940x604 pixels. The associated camera in the rig file is called the pilot camera.

Running the Sample

The PilotNet sample, sample_pilotnet, accepts the following optional parameters. If none are specified, it will perform path perception on pre-recorded video.

./sample_pilotnet --loop=[0|1]
                  --patchDir=[path/to/output/directory]
                  --printOutput=[0|1]
                  --rig=[path/to/rigfile]
                  --video=[path/to/video]

Where:

--loop=[0|1]
    Setting the parameter to 0 will exit the app once the video is completed.
    Setting the parameter to 1 will run the video in an infinite loop until the user exits the app by pressing the [Esc] key.
    Default value: 1

--patchDir=[path/to/output/directory]
    Directory for storing PilotNet ROI as a ppm image for each frame.
    If not provided, no ROI image is stored.
    Default value: ""

--printOutput=[0|1]
    Print the network prediction for each trajectory.
    Default value: 0

--rig=[path/to/rigfile]
    Path to the rig file containg the calibration for the video specified by --video.
    Name of the camera must be pilot
    Default value: path/to/data/samples/pilotnet/rig.json

--video=[path/to/video]
    Path to video file to be used for replay.
    Default value: path/to/data/samples/raw/rccb.raw

Examples

The sample is setup to run a default RAW video in a loop.

./sample_pilotnet

Use the –video and –rig option to run a custom RAW video file and rig file. Make sure that the rig file corresponds to the car from which the video was recorded from.

./sample_pilotnet --video=[path/to/video] --rig=[path/to/rigfile]

Use the –loop option to enable/disable looping of video file.

./sample_pilotnet --loop=[0|1]

Use the –printOutput option to print the world space trajectory coordinates predicted by PilotNet.

./sample_pilotnet --printOutput=[0|1]

Output

PilotNet Sample replays a video and predicts the paths for different maneuvers which are color-coded as follows

Red: Lane Stable
Purple: Lane Change Left First Half
Blue: Lane Change Right First Half
Light Green: Lane Change Left Second Half
White: Lane Change Right Second Half
Yellow: Lane Split Left
Dark Green: Lane Split Right

Additionally, the green highlighting in the video specifies the areas of the ROI that are of interest to the network.

PilotNet Sample

Additional Information

For more details see PilotNet and Pilotnet Detector.

Table of Contents

Description

Running the Sample

Examples

Output

Additional Information