perception/object/camera/lightnet/docs/lightnet_mainsection.md

Go to the documentation of this file.

# Copyright (c) 2019-2020 NVIDIA CORPORATION.  All rights reserved.

@page lightnet_mainsection LightNet

@note SW Release Applicability: This module is available in **NVIDIA DRIVE Software** releases.

## About This Module
This module provides the APIs to initialize, query, and release the NVIDIA proprietary traffic light classification deep neural network: **LightNet**. <br>

The data structures include:
 - `::dwTrafficLightClass`: defines the traffic light states that LightNet is trained to classify.
 - `::dwLightNetModel`: allows user to choose among LightNet models.
 - `::dwLightNetParams`: in addition to `::dwLightNetModel`, defines the LightNet model variant with specific precision and processor optimization to be loaded.

DriveWorks comes with the LightNet model:
 - Front camera model:  ::DW_LIGHTNET_MODEL_FRONT_V5: Supported on GPU, DLA
 - Front camera model:  ::DW_LIGHTNET_MODEL_FRONT_V6: Supported on GPU
 - Front camera model:  ::DW_LIGHTNET_MODEL_FRONT_V7: Supported on GPU

### Inputs

LightNet consumes FP16 planar frames with a resolution of 960x604 pixels from AR0231 cameras (revision >= 4).

### Outputs

LightNet currently supports 21 traffic light classes:

| #   | TRAFFIC LIGHT STATE NAME    |              LIGHT ICON                            | DESCRIPTION                                                                      |
|-----|-----------------------------|----------------------------------------------------|:---------------------------------------------------------------------------------|
| 1   | TLGreenArrowDown            |  ![ ](images/lightnet_us/green_down.png)           | Green down arrow.                                                                |
| 2   | TLGreenArrowLeft            |  ![ ](images/lightnet_us/green_arr_left.png)       | Green left arrow.                                                                |
| 3   | TLGreenArrowRight           |  ![ ](images/lightnet_us/green_arr_right.png)      | Green right arrow.                                                               |
| 4   | TLGreenArrowStraight        |  ![ ](images/lightnet_us/green_up.png)             | Green straight arrow.                                                            |
| 5   | TLGreenLightGreenArrowLeft  |  ![ ](images/lightnet_us/green_green_arr_left.png) | Green solid along with a green left arrow.                                       |
| 6   | TLGreenLightGreenArrowRight |  ![ ](images/lightnet_us/green_green_arr_right.png)| Green solid along with a green right arrow.                                      |
| 7   | TLGreenLightRedArrowLeft    |  ![ ](images/lightnet_us/green_red_arr_left.png)   | Green solid along with a red left arrow.                                         |
| 8   | TLGreenLightRedArrowRight   |  ![ ](images/lightnet_us/green_red_arr_right.png)  | Green solid along with a red right arrow.                                        |
| 9   | TLRedArrowLeft              |  ![ ](images/lightnet_us/red_arr_left.png)         | Red left arrow.                                                                  |
| 10  | TLRedArrowRight             |  ![ ](images/lightnet_us/red_arr_right.png)        | Red left arrow.                                                                  |
| 11  | TLRedLight                  |  ![ ](images/lightnet_us/red_solid.png)            | Red solid.                                                                       |
| 12  | TLRedLightGreenArrowLeft    |  ![ ](images/lightnet_us/red_green_arr_left.png)   | Red solid along with a green left arrow.                                         |
| 13  | TLRedLightGreenArrowRight   |  ![ ](images/lightnet_us/red_green_arr_right.png)  | Red solid along with a green right arrow.                                        |
| 14  | TLRedLightRedArrowLeft      |  ![ ](images/lightnet_us/red_red_arr_left.png)     | Red solid along with a red left arrow.                                           |
| 15  | TLRedLightRedArrowRight     |  ![ ](images/lightnet_us/red_red_arr_right.png)    | Red solid along with a red right arrow.                                          |
| 16  | TLYellowArrowLeft           |  ![ ](images/lightnet_us/yellow_arr_left.png)      | Yellow left arrow.                                                               |
| 17  | TLYellowArrowRight          |  ![ ](images/lightnet_us/yellow_arr_right.png)     | Yellow right arrow.                                                              |
| 18  | TLYellowLight               |  ![ ](images/lightnet_us/yellow_solid.png)         | Yellow solid.                                                                    |
| 19  | TrafficLightGreen           |  ![ ](images/lightnet_us/green_solid.png)          | Green solid.                                                                     |
| 20  | TLStatelessOrSide           |  ![ ](images/lightnet_us/stateless.png)            | The traffic light shows no light, is seen from the side, or not activated.    |
| 21  | TLLaneClosed                |  ![ ](images/lightnet_us/lane_closed.png)          | Red cross signal.                                                                 |
| 22  | Invalid                     |  ![ ](images/lightnet_us/invalid.png)              | Not a traffic light object.                                                                   |


### Additional Information
@warning The LightNet model currently deployed is mostly trained with United States traffic lights. It might experience
reduced accuracy in other regions, particularly for light patterns not present in the US.

## Relevant Tutorials

- @ref lightnet_usecase1

## APIs

- @ref lightnet_group