perception/safety/camera/clearsightnet/docs/clearsightnet_usecase1.md

Go to the documentation of this file.

# Copyright (c) 2018-2020, NVIDIA CORPORATION.  All rights reserved.

@page clearsightnet_usecase1 ClearSightNet Workflow

## Initialization

The ClearSightNet and ClearSightNetDetector modules create handles of types `dwClearSightNetHandle_t` and `dwBlindnessDetectorHandle_t` respectively. Both of these handles are owned by the application and must be released by the application when not needed anymore.

An instance of type `dwClearSightNetParams` encodes parameters that configure the ClearSightNet module. To initialize default ClearSightNet parameters:

```{.cpp}
dwStatus dwClearSightNet_initDefaultParams(dwClearSightNetParams* clearSightNetParams);
```

To initialize a pointer to a `dwClearSightNetHandle_t` instance with default parameter values:

```{.cpp}
dwStatus dwClearSightNet_initialize(dwClearSightNetHandle_t* clearSightNetHandle,
                                    const dwClearSightNetParams* clearSightNetParams,
                                    dwContextHandle_t ctx);
```

An instance of type `dwBlindnessDetectorParams` encapsulates several configuration parameters and encapsulates a handle to ClearSightNet (of type `dwClearSightNetHandle_t`) that configure the BlindnessDetector module.

To initialize in instance of `dwBlindnessDetectorParams` with default values:

```{.cpp}
dwStatus dwBlindnessDetector_initDefaultParams(dwBlindnessDetectorParams *clearSightNetDetectorParams);
```

This initializes the members `dwBlindnessDetectorParams.temporalFilterWindow`, `dwBlindnessDetectorParams.numRegionsX` and `dwBlindnessDetectorParams.numRegionsY` to 5, 1 and 1 respectively.

To initialize a pointer to a `dwBlindnessDetectorHandle_t` instance with an initialized ClearSightNet handle and default parameter values:

```{.cpp}
dwStatus dwBlindnessDetector_initialize(dwClearSightNetDetectorHandle_t* clearSightNetDetectorHandle,
                                        dwBlindnessDetectorParams *clearSightNetDetectorParams,
                                        dwContextHandle_t ctx);
```

An example code snippet to initialize required modules with default values to perfrom inference on GPU:

```{.cpp}
// Initialize ClearSightNet
dwClearSightNetParams m_clearSightNetParams;
dwBlindnessDetectorParams m_detectorParams;
CHECK_DW_ERROR(dwClearSightNet_initDefaultParams(&m_clearSightNetParams));
CHECK_DW_ERROR(dwClearSightNet_initialize(&m_detectorParams.clearSightNetHandle, 
                                          &m_clearSightNetParams, 
                                          m_sdk));

dwBlindnessDetectorHandle_t m_clearSightNetDetector;
CHECK_DW_ERROR(dwBlindnessDetector_initDefaultParams(&clearSightNetDetectorParams));
CHECK_DW_ERROR(dwBlindnessDetector_initialize(&m_clearSightNetDetector,
                                              &m_detectorParams,
                                              m_sdk));
```

where `m_sdk` is an initialized instance of `dwContextHandle_t`.

Another code snippet to initialize required modules with parameters that enable 3 sub-regions in each direction:

```{.cpp}
// Initialize ClearSightNet
dwClearSightNetParams m_clearSightNetParams;
dwBlindnessDetectorParams m_detectorParams;
CHECK_DW_ERROR(dwClearSightNet_initDefaultParams(&m_clearSightNetParams));
CHECK_DW_ERROR(dwClearSightNet_initialize(&m_detectorParams.clearSightNetHandle, 
                                          &m_clearSightNetParams, 
                                          m_sdk));

dwBlindnessDetectorHandle_t m_clearSightNetDetector;
CHECK_DW_ERROR(dwBlindnessDetector_initDefaultParams(&clearSightNetDetectorParams));
m_detectorParams.numRegionsX = 3U;
m_detectorParams.numRegionsY = 3U;
m_detectorParams.regionDividersX[0] = 0.2f; m_detectorParams.regionDividersX[1] = 0.8f;
m_detectorParams.regionDividersY[0] = 0.2f; m_detectorParams.regionDividersY[1] = 0.8f;
CHECK_DW_ERROR(dwBlindnessDetector_initialize(&m_clearSightNetDetector,
                                              &m_detectorParams,
                                              m_sdk));
```

Note that values for parameters `dwBlindnessDetectorParams.regionDividersX` and `dwBlindnessDetectorParams.regionDividersY` are specified in image fractions. This will create sub-region dividers at 20% and 80% of image width and height.

## Query Network Specs

With initialized handle to the ClearSightNet module, the following functions can be used to query size of expected input blob, size of output blob and network metadata respectively:

```{.cpp}
dwStatus dwClearSightNet_getInputBlobSize(dwBlobSize* inputBlobsize, dwClearSightNetHandle_t clearSightNetHandle);
```

```{.cpp}
dwStatus dwClearSightNet_getOutputBlobSize(dwBlobSize* outputBlobsize, dwClearSightNetHandle_t clearSightNetHandle);
```

```{.cpp}
dwStatus dwClearSightNet_getDNNMetaData(dwDNNMetaData* metaData, dwClearSightNetHandle_t clearSightNetHandle);
```

## Use for Inference

With initialized handle to the ClearSightNetDetector module, the following functions can be used to perform inference on an image and retrieve the processed output:

```{.cpp}
dwStatus dwBlindnessDetector_detect(const dwImageCUDA* image,
                                    dwBlindnessDetectorHandle_t clearSightNetDetectorHandle);
```

```{.cpp}
dwStatus dwBlindnessDetector_getOutput(dwBlindnessDetectionOutput *outputs,
                                       dwBlindnessDetectorHandle_t clearSightNetDetectorHandle);
```

The `dwBlindnessDetector_detect()` function performs inference asynchronously. This means that an inference call can be queued using `dwBlindnessDetector_detect()` and then other computation can be performed and `dwBlindnessDetector_getOutput()`, whenever called, will automatically wait for the inference to finish before returning the output. To this end, the following functions can be used to set and get CUDA the stream used by the module:

```{.cpp}
dwStatus dwBlindnessDetector_setCUDAStream(const cudaStream_t stream, dwBlindnessDetectorHandle_t clearSightNetDetectorHandle);
```

```{.cpp}
dwStatus dwBlindnessDetector_getCUDAStream(cudaStream_t *stream, dwBlindnessDetectorHandle_t clearSightNetDetectorHandle);
```

## Reset and Release

The following functions can be used to reset or release the initialized module handles whenever necessary:

```{.cpp}
dwStatus dwClearSightNet_reset(dwClearSightNetHandle_t clearSightNetHandle);
```

```{.cpp}
dwStatus dwClearSightNet_release(dwClearSightNetHandle_t clearSightNetHandle);
```

```{.cpp}
dwStatus dwBlindnessDetector_reset(dwBlindnessDetectorHandle_t clearSightNetDetectorHandle);
```

```{.cpp}
dwStatus dwBlindnessDetector_release(dwBlindnessDetectorHandle_t clearSightNetDetectorHandle);
```

For usage of a sample application, please refer to @ref dwx_clearsightnet_sample.