1 # Copyright (c) 2019-2020, NVIDIA CORPORATION. All rights reserved.
3 @page dwx_sign_classification_sample Traffic Sign Classification Sample (SignNet)
6 @note SW Release Applicability: This sample is available in **NVIDIA DRIVE Software** releases.
8 @section dwx_sign_classification_description Description
10 The Traffic Sign Classification sample demonstrates how to use the NVIDIA<sup>®</sup>
11 proprietary SignNet deep neural network (DNN) to perform traffic sign classification.
12 It outputs the class of the traffic signs from images captured by the ego car.
14 SignNet currently supports RCB images. RGBA images are not supported. SignNet models currently cover three geographical regions. For one of such regions, the United States (US), two models are provided: `US_V1` and `US_V2`. For another supported region, the European Union (EU), there are also two available models: `EU_V1` and `EU_V2`. For the last supported region, Japan, there is only one supported model - `JP_V1`. The main difference between different model versions for the EU and US regions is in the number of supported classes. Models version 2 for each region have significantly more traffic sign classes covered than version 1 models.
16 The default model for the sample app is `US_V2` for the US. In order to use the European traffic sign models, the Japan model, or to use the US model version 1, one would need to explicitly select it with a corresponding command line parameter when running the sample.
18 This sample shows a simple implementation of traffic sign classification built around the NVIDIA SignNet DNN. The classification is done by first detecting the traffic signs with the help of NVidia DriveNet DNN and then classification of found image-crops with the help of SignNet DNN. There is no tracking of traffic signs applied, so one may notice some flickering of detections.
19 For more information on the SignNet DNN and how to customize it for your applications, consult your NVIDIA sales or business representative.
21 @subsection dwx_sign_classification_sensor_details Sensor Details
23 The image datasets used to train SignNet have been captured by a View Sekonix Camera Module (SF3325) with
24 AR0231 RCCB sensor. The camera is mounted on a rig on top of the vehicle. Demo videos are
25 captured at 2.3 MP and downsampled to 960 x 604. Eight cameras were used to collect the data for training the provided SignNet models.
26 The following list shows the setup position and field of view (FOV) of each such camera:
27 - Center Front 60° FOV
28 - Center Front 120° FOV
29 - Center Front 30° FOV
30 - Center Right 120° FOV
31 - Center Left 120° FOV
32 - Rear Left 120° FOV
33 - Rear Center 120° FOV
34 - Rear Center 60° FOV
36 To achieve the best traffic sign detection performance, NVIDIA recommends to adopt a camera setup similar to one or more cameras from the list above and align the video center vertically with the horizon before recording new videos.
38 @subsection dwx_sign_classification_sample_limitations Limitations
40 @warning Currently, the SignNet DNN has limitations that could affect its performance:
41 - It was trained mostly for bright day-light, overcast, twilight, non-rain visibility conditions.
42 Training for artificial light, night-light conditions, and rainy-weather visibility was limited and, thus, the performance of the classifier may suffer in rain or in constrained illumination.
43 - The classification performance of SignNet depends on the size of the traffic signs detected in an image frame.
44 Good classification performance is observed when the height of the traffic signs is 20 pixels or more. Predictions for very small signs may be unreliable.
45 - The provided SignNet models were trained on data collected in the United States, Japan, and the countries comprising European Union.
46 As a result, SignNet models may not be suitable for other geographical regions. However, the use of the EU model may be appropriate for other countries which adopted the Vienna convention for traffic signs. But the specific sign classes available in those countries should be reviewed case by cases basis against the ones available in the provided model.
48 Even though the SignNet DNN was trained with data from cameras setup pointing in various direction of the sensor rig (see the list above), it is recommeded to use it for the following directional and FOV setup::
49 * Center-front camera location with a 60° FOV.
50 * Center-front camera location with a 120° FOV.
52 @section dwx_sign_classification_running Running the Sample
54 ./sample_sign_classifier --input-type=[video|camera]
55 --model=[US_V1|US_V2|EU_V1|EU_V2|JP_V1]
56 --video=[path/to/video]
57 --camera-type=[camera]
58 --camera-group=[a|b|c|d]
60 --camera-index=[0|1|2|3]
61 --precision=[fp16|fp32]
65 --input-type=[video|camera]
66 Defines if the input is from live camera or from a recorded video.
67 Live camera is supported only on NVIDIA DRIVE(tm) platforms.
68 It is not supported on Linux (x86 architecture) host systems.
71 --video=[path/to/video]
72 Specifies the absolute or relative path of a raw, lraw or h264 recording.
73 Only applicable if --input-type=video
74 Default value: samples/waitcondition/traffic_signs.h264
76 --model=[US_V1|US_V2|EU_V1|EU_V2]
77 Specifies which regional signnet model and its version to use for classifying the signs.
78 Default value: US_V2 (USA model, version 2).
80 --camera-type=[camera]
81 Specifies a supported AR0231 `RCCB` sensor.
82 Only applicable if --input-type=camera.
83 Default value: ar0231-rccb-bae-sf3324
85 --camera-group=[a|b|c|d]
86 Is the group where the camera is connected to.
87 Only applicable if --input-type=camera.
91 Setting this parameter to 1 when running the sample on Xavier B accesses the camera
93 Applicable only when --input-type=camera.
96 --camera-index=[0|1|2|3]
97 Specifies the camera index within the camera group.
100 --precision=[fp16|fp32]
101 Specifies the precision to be used for the SignNet DNN model.
105 Setting this parameter to 1 runs SignNet DNN inference by CUDAGraph if the hardware supports.
108 ### To run the sample on Linux
110 ./sample_sign_classifier --video=<video file.raw>
112 ./sample_sign_classifier --video=<video file.lraw>
114 ./sample_sign_classifier --video=<video file.h264>
116 ### To run the sample on video with european signs classifier version 2:
118 ./sample_sign_classifier --video=<video file.h264> --signnet-model EU_V2
120 ### To run the sample on a camera on NVIDIA DRIVE platforms
122 ./sample_sign_classifier --input-type=camera --camera-type=<rccb camera type> --camera-group=<camera-group> --camera-index=<camera idx on camera group> --model <model_name>
124 where `<rccb camera type>` is one of the following:
126 - `ar0231-rccb-bae-sf3324`
127 - `ar0231-rccb-bae-sf3325`
130 @subsection dwx_sign_classification_output Output
132 The sample creates a window, displays a video, and overlays bounding boxes for detected traffic signs.
133 The class of the sign is displayed with the text label above the bounding box.
135 The following table describes the models provided as part of the package. Follow the hyperlinks to the the full list of classes
136 supported by each model.
138 | MODEL NAME | MODEL IDENTIFIER | SUPPORTED REGION | # OF MODEL'S OUTPUTS | # OF DISTINCT CLASS LABELS | DESCRIPTION |
139 |-------------------|------------------------|---------------------------|----------------------|----------------------------|----------------|
140 | @ref signnet_US_v1_0 | US_V1 | United States of America | 30 | 24 | Minimal USA sign model with focus on speed limit signs.|
141 | @ref signnet_US_v2_0 | US_V2 | United States of America | 312 | 272 | Advanced USA model with expanded sign coverage. |
142 | @ref signnet_EU_v1_0 | EU_V1 | European Union | 31 | 28 | Minimal EU sign model with focus on speed limit signs. |
143 | @ref signnet_EU_v2_0 | EU_V2 | European Union | 176 | 159 | Advanced EU model with expanded sign coverage. |
144 | @ref signnet_JP_v1_0 | JP_V1 | Japan | 184 | 151 | Japan sign model.
146 Note, the EU SignNet models may be appropriate to classify road signs from other non-EU countries that follow Vienna convention.
148 
150 @section dwx_sign_classification_more Additional Information
152 For more information, see @ref signnet_mainsection.