waitcondition/camera/docs/mainsection.md

Go to the documentation of this file.

# Copyright (c) 2019-2020 NVIDIA CORPORATION.  All rights reserved.

@page waitcondition_mainsection Wait Conditions Detection and Classification

@note SW Release Applicability: This module is available in **NVIDIA DRIVE Software** releases.

## About This Module

Wait Conditions detector module detects intersections, traffic lights and traffic signs from camera images. NVIDIA proprietary DNN named WaitNet will return the detected bounding box of intersections, traffic signs and traffic lights.

Wait Conditions classifier module classifies detected traffic signs and traffic lights from camera images. NVIDIA proprietary DNN named LightNet will return the state (color(s) and shape(s)) of a detected traffic light. NVIDIA proprietary DNN named SignNet will return the class of a detected traffic sign.

### Wait Conditions Intersection Detector Module Initialization
Wait Conditions detector module can create default parameters:

```{.cpp}
dwStatus dwWaitConditionsDetector_initDefaultParams(dwWaitConditionsDetectorParams* detectorParams);
```

With the generated default parameters, Wait Conditions detector module can be initialized by:

```{.cpp}
dwStatus dwWaitConditionsDetector_initializeFromWaitNetNew(dwWaitConditionsDetectorHandle_t* obj,
                                                           dwWaitNetHandle_t waitnet,
                                                           const dwWaitConditionsDetectorParams* detectorParams,
                                                           const dwTransformation3f* cam2Rig,
                                                           dwCameraModelHandle_t calibratedCamObj,
                                                           dwEgomotionConstHandle_t motionObj,
                                                           dwContextHandle_t ctx);
```

where dwWaitNetHandle_t specifies the handle to the WaitNet module, dwWaitConditionsDetectorParams is pointer to WaitConditionsDetector parameters, dwTransformation3f is camera extrinsic transformation, dwCameraModelHandle_t is camera intrinsic transformation, dwEgomotionConstHandle_t is egomotion module handle and dwContextHandle_t specifies the handle to the drivework context. The camera extrinsic can be set by:

```{.cpp}
dwStatus dwWaitConditionsDetector_setCameraExtrinsics(uint32_t cameraID, const dwTransformation3f* cam2Rig, dwWaitConditionsDetectorHandle_t obj);
```

The output of the module need to be created by the application and bind to the detector module by:

```{.cpp}
dwStatus dwWaitConditionsDetector_bindOutput(const dwWaitConditionsDetectorOutput* output,
                                             dwWaitConditionsDetectorHandle_t obj);
```

where dwWaitConditionsDetectorOutput holds the output.

CUDA stream can be set and get by:

```{.cpp}
dwStatus dwWaitConditionsDetector_setCUDAStream(cudaStream_t stream, dwWaitConditionsDetectorHandle_t obj);
```

```{.cpp}
dwStatus dwWaitConditionsDetector_getCUDAStream(cudaStream_t* stream, dwWaitConditionsDetectorHandle_t obj);
```

### Wait Conditions Intersection Detector Module Execution

After module initialization, the network to detect the intersection, traffic light and traffic sign can be inferred by:

```{.cpp}
dwStatus dwWaitConditionsDetector_inferDeviceAsync(const dwImageCUDA* const* imageArray,
                                                   uint32_t numImages, dwWaitConditionsDetectorHandle_t obj);
```

where imageArray stores pointers to images where the detector is to be applied. The inferred result can be interpreted by:

```{.cpp}
dwStatus dwWaitConditionsDetector_interpret(uint32_t numImages, dwWaitConditionsDetectorHandle_t obj);
```

After the interpretation, the detection result can be accessed in the output bound to the detector.

### Wait Conditions Intersection Detector Module Release

The module can be released by:

```{.cpp}
dwStatus dwWaitConditionsDetector_release(dwWaitConditionsDetectorHandle_t obj);
```

### Wait Conditions Traffic Light/Sign Classifier Module Initialization
Wait Conditions classifier module initialization takes either a user customized sign detection network module handle or a user customized light detection network module handle as an input. To initialize NVIDIA Traffic Light/Sign Classifier module:

With the initialized LightNet module handle, Wait Conditions classifier module can be initialized by:
```

dwTrafficLightSignClassifier_initializeFromLightNet(dwTrafficLightSignClassifierHandle_t* obj,
                                          dwLightNetHandle_t dnn,
                                          dwContextHandle_t ctx);
```

With the initialized SignNet module handle, Wait Conditions classifier module can be initialized by:
```

dwTrafficLightSignClassifier_initializeFromSignNet(dwTrafficLightSignClassifierHandle_t* obj,
                                         dwSignNetHandle_t dnn,
                                         dwContextHandle_t ctx);
```

### Wait Conditions Classification Execution
After module initialization, the sequential API calls to classify detected input traffic signs and traffic lights are as follows.
To perform network inference on GPU:
```
dwTrafficLightSignClassifier_inferDeviceAsync(const dwImageCUDA* const* imageArray,
                                    dwObject* objectList,
                                    uint32_t numObjects,
                                    dwObjectClass type,
                                    dwTrafficLightSignClassifierHandle_t obj);
```

To interpret the network inferred results into traffic light states or traffic sign classes:
```
dwTrafficLightSignClassifier_interpretHost(uint32_t numObjects,
                                 dwObjectClass type,
                                 dwTrafficLightSignClassifierHandle_t obj);
```

To get the per frame interpreted results in the format of `subClassId` field on each traffic sign or traffic light dwObject:
```
dwTrafficLightSignClassifier_getClassifiedObjects(dwObject* objectList,
                                        uint32_t numObjects,
                                        dwTrafficLightSignClassifierHandle_t obj);
```

### Wait Conditions Traffic Light/Sign Classifier Module Release
To release the Wait Conditions classifier module, the module itself needs to be released first by:
```
dwTrafficLightSignClassifier_release(dwTrafficLightSignClassifierHandle_t obj);
```

then the SignNet module:
```
dwSignNet_release(dwSignNetHandle_t obj);

```

or the LightNet module:
```
dwLightNet_release(dwLightNetHandle_t obj);

```

## Relevant Tutorials

- @ref waitcondition_usecase1
- @ref waitcondition_usecase2
- @ref waitcondition_usecase3
- @ref waitcondition_usecase4