1 # Copyright (c) 2019-2020 NVIDIA CORPORATION. All rights reserved.
3 @page dwx_path_perception_sample Path Perception Sample (PathNet)
6 @note SW Release Applicability: This sample is available in **NVIDIA DRIVE Software** releases.
8 @section dwx_path_perception_sample_description Description
10 The Path Perception sample demonstrates how to use the NVIDIA<sup>®</sup> proprietary deep neural network
11 to perform path perception on the road. It detects the path you are in (ego-path), as well
12 as the left and right adjacent paths when they are present. PathNet has been trained with RCB
13 images and its performance is invariant to RGB encoded H.264 videos.
15 This sample streams an H.264 or RAW video, computing paths for each frame. The network
16 directly computes the path vertices and a confidence value for each path.
17 A user assigned threshold value sets the minimum confidence for each path to be considered valid.
18 The sample can also be operated with cameras.
22 The image datasets used to train Pathnet have been captured by a View Sekonix Camera Module (SS3323) with
23 AR0231 RCCB sensor with a 60 degree field of view. The camera is mounted high up at the rear view mirror position.
24 Demo videos are captured at 2.3 MP and down-sampled to 960 x 604.
26 To achieve the best path perception performance, NVIDIA<sup>®</sup> recommends to adopt a similar camera setup and align
27 the video center vertically with the horizon before recording new videos. Also, the detection will perform best
28 with a 60 degree field of view camera.
30 @section dwx_path_perception_sample_running Running the Sample
32 The Path Perception sample, sample_path_perception, accepts the following optional parameters. If none are specified, it will perform path perception on pre-recorded video.
34 ./sample_path_perception --camera-type=[camera]
35 --camera-group=[a|b|c|d]
36 --camera-index=[0|1|2|3]
38 --input-type=[video|camera]
39 --video=[path/to/video]
41 --fps=<integer number in (1, 120)>
42 --detectionThreshold=<floating-point number in (0, 1)>
43 --temporalSmoothingFactor=<floating-point number in (0, 1)>
44 --lookAheadDistance=<floating-point number in (0.0, 110.0)>
45 --roi.x=<integer number in (0, image_width)>
46 --roi.y=<integer number in (0, image_height)>
47 --roi.width=<integer number in (0, image_width)>
48 --roi.height=<integer number in (0, image_height)>
49 --precision=[int8|fp16|fp32]
54 --camera-type=[camera]
55 Is a supported AR0231 `RCCB` sensor.
56 Only applicable if --input-type=camera.
57 Default value: ar0231-rccb-bae-sf3324
59 --camera-group=[a|b|c|d]
60 Is the group where the camera is connected to.
61 Only applicable if --input-type=camera.
64 --camera-index=[0|1|2|3]
65 Indicates the camera index on the given port.
69 Setting this parameter to 1 when running the sample on Xavier B allows to access a camera that
70 is being used on Xavier A. Only applicable if --input-type=camera.
73 --input-type=[video|camera]
74 Defines if the input is from live camera or from a recorded video.
75 Live camera is only supported on NVIDIA<sup>®</sup> DRIVE platform.
78 --detectionThreshold=[fp_number]
79 The detection threshold parameter is used to determine the validity of a path generated
80 by the network. If there is no path with a confidence above this value, then no paths will be displayed.
81 By default, the value is 0.5, which provides the best accuracy based on the NVIDIA<sup>®</sup> test data set.
82 Decrease the threshold value if path polylines flicker or cover shorter distance.
84 --temporalSmoothingFactor=[fp_number]
85 The temporal smoothing factor is used to take a weighted average of the model predictions from the current
86 frame and the immediately preceding frame. The average is computed as
87 x'(t) = (1 - temporalSmoothingFactor) * x(t) + temporalSmoothingFactor * x(t-1). This means that the higher
88 the factor, the less the impact of the current prediction on the final output. A factor of 1 would never update
89 the output and a factor of 0 would never consider the past input.
90 By default, the value is 0.1, which provides the best accuracy based on the NVIDIA<sup>®</sup> test data set.
91 Increase the factor value if path polylines flicker.
93 --lookAheadDistance=[fp_number]
94 The maximum allowable distance from car in world coordinates over which detected points are ignored.
95 By default, the value is set to 80.0, which provides the best accuracy based on the NVIDIA<sup>®</sup> test data set.
98 The top left x image coordinate in the input frame that is to be cropped and passed into the network.
99 By default, the value is set to 0, which provides the best accuracy based on the NVIDIA<sup>®</sup> test data set.
102 The top left y image coordinate in the input frame that is to be cropped and passed into the network.
103 By default, the value is set to 400, which provides the best accuracy based on the NVIDIA<sup>®</sup> test data set.
106 The width of our ROI.
107 By default, the value is set to 1920, which provides the best accuracy based on the NVIDIA<sup>®</sup> test data set.
110 The height of our ROI.
111 By default, the value is set to 800, which provides the best accuracy based on the NVIDIA<sup>®</sup> test data set.
113 --video=[path/to/video]
114 Specifies the absolute or relative path of recording.
115 Only applicable if --input-type=video
116 Default value: path/to/samples/pathDetection/video_paths.h264
119 Rig file containing all information about vehicle sensors and calibration.
120 Default value: path/to/data/samples/pathDetection/rig.json
123 Whether to show the default view or the debug view, which includes fishbone lines connecting the predicted points of the network.
125 --precision=[int8|fp16|fp32]
126 Specifies the precision for the PathNet model.
130 Setting this parameter to 1 runs PathNet DNN inference by CUDAGraph if the hardware supports.
133 @subsection dwx_path_perception_sample_examples Examples
135 #### To run the sample on Linux
137 ./sample_path_perception --video=<video file.h264> --detectionThreshold=<floating-point number in (0,1)>
140 ./sample_path_perception --video=<video file.raw> --detectionThreshold=<floating-point number in (0,1)>
142 #### To run the sample on an NVIDIA DRIVE platform with cameras:
144 ./sample_path_perception --input-type=camera --camera-type=<camera_type> --camera-group=<camera_group> --detectionThreshold=<floating-point number in (0,1)>
146 where `<camera type>` is a supported `RCCB` sensor.
147 See @ref supported_sensors for the list of supported cameras for each platform.
149 @note Path perception sample directly resizes video frames to the network
150 input resolution. Therefore, to get the best performance, it is suggested to
151 use videos with similar aspect ratio to the demo video. Or you can set Region
152 of Interest (ROI) to perform inference on a sub-window of the full frame.
154 @section dwx_path_perception_sample_output Output
156 PathNet creates a window, displays a video, and overlays a collection of polylines for each detected path.
157 The path center line is displayed as a thick polyline, with its lateral extent shown as thin polylines.
159 The colors of the polylines represent the path marking position types and the path attribute that
160 it detects, as follows:
163 - Blue: Left adjacent path
164 - Green: Right adjacent path
165 - Dark Red: Ego path fork-left
166 - Purple: Ego path fork-right
167 - Dark Green: Right adjacent path fork-right
168 - Dark Blue: Left adjacent path fork-left
169 - White: Opposite traffic direction
171 
172 
174 @section dwx_path_perception_sample_more Additional Information
176 For more details see @ref pathperception_mainsection_pathdetector.