perception/object/camera/signnet/docs/signnet_mainsection.md

Go to the documentation of this file.

# Copyright (c) 2019-2020 NVIDIA CORPORATION.  All rights reserved.

@page signnet_mainsection SignNet

@note SW Release Applicability: This module is available in **NVIDIA DRIVE Software** releases.

## About This Module
This module provides the APIs to initialize, query, and release the NVIDIA proprietary traffic sign classification deep neural network: **SignNet**. The data structures include `::dwTrafficSignalClass`, which defines the traffic sign state that the SignNet is trained to detect. `::dwSignNetModel` allows user to choose among SignNet models. And `dwSignNetParams`, in addition to `::dwSignNetModel`, defines the SignNet model variant with specific precision and processor optimization to be loaded.

 SignNet models currently cover three geographical regions: United States, Euroopean Unition, and Japan. For one of such regions, the United States (US), two models are provided: `::DW_SIGNNET_MODEL_US_V1` and `::DW_SIGNNET_MODEL_US_V2`. For EU, there are also two available models: `::DW_SIGNNET_MODEL_EU_V1` and `::DW_SIGNNET_MODEL_EU_V2`. For the remaining region, Japan, only one model is available: `::DW_SIGNNET_MODEL_JP_V1`. The main difference between different model versions for EU and US regions is in the number of supported classes. Models version 2 for each region have significantly more traffic sign classes covered than version 1 models.

## Inputs

The image datasets used to train SignNet have been captured by a View Sekonix Camera Module (SF3325) with
AR0231 RCCB sensor. The camera is mounted on a rig on top of the vehicle. Demo videos are
captured at 2.3 MP and downsampled to 960 x 604. Eight cameras were used to collect the data for training the provided SignNet models.
The following list shows the setup position and field of view (FOV) of each such camera:
- Center Front 60° FoV
- Center Front 120° FoV
- Center Front 30° FoV
- Center Right 120° FoV
- Center Left 120° FoV
- Rear Left 120° FoV
- Rear Center 120° FoV
- Rear Center 60° FoV

Even though the SignNet DNN was trained with data from cameras setup pointing in various direction of the sensor rig it is recommeded to use it for the following directional and FoV setup:
* Center-front camera location with a 60° FoV.
* Center-front camera location with a 120° FoV.

## Output

For a list of traffic signs detected by each model see the pages below.

| Model Name           |   Description |
|----------------------|----------------------|
| @ref signnet_US_v1_0 | SignNet model for US (v1.0) with 30 outputs, and 24 distinct text labels |
| @ref signnet_EU_v1_0 | SignNet model for EU (v1.0) with 31 outputs, and 28 distinct text labels |
| @ref signnet_US_v2_0 | SignNet model for US (v2.0) with 312 outputs, and 272 distinct text labels |
| @ref signnet_EU_v2_0 | SignNet model for EU (v2.0) with 176 outputs, and 159 distinct text labels |
| @ref signnet_JP_v1_0 | SignNet model for Japan (v1.0) with 184 outputs, and 151 distinct text labels |

### Additional Information

@warning Currently, the SignNet DNN has limitations that could affect its performance:
- It was trained mostly for bright day-light, overcast, twilight, non-rain visibility conditions.
  Training for artificial light, night-light conditions, and rainy-weather visibility was limited and, thus, the performance of the classifier may suffer in rain or in constrained illumination.
- The classification performance of SignNet depends on the size of the traffic signs detected in an image frame.
  Good classification performance is observed when the height of the traffic signs is 20 pixels or more. Predictions for very small signs may be unreliable.
- The provided SignNet models were trained on data collected in the United States, Japan, and countries comprising European Union.
  As a result, SignNet models may not be suitable for other geographical regions. However, the use of the EU model may be appropriate for other countries which adopted the Vienna convention for traffic signs. But the specific sign classes available in those countries should be reviewed case by cases basis against the ones available in the provided model.
- SignNet currently supports RCB images. RGBA images are not supported.

## Relevant Tutorials

- @ref signnet_usecase1

## APIs

- @ref signnet_group