src/dw/sensors/sensormanager/docs/mainsection.md

Go to the documentation of this file.

# Copyright (c) 2019-2021 NVIDIA CORPORATION.  All rights reserved.

@page sensormanager_mainsection Sensor Manager

## About This Module

Sensor Manager uses underlying SAL (Sensor Abstraction Layer) APIs to provide applications with a higher-level API to access sensor events. It is a thread-less module, whose purpose is to provide time-sorted stream of events from an aggregate set of sensors. Key features include:

- Configuration: instantiation of sensors based on JSON configuration data.
- Sensor Event Access: strictly time-sorted event stream from all sensors in case of "virtual" sensors (i.e. sensor data playback from file); and weakly-sorted event stream from all sensors if running in live mode (with real sensors).

Implementation details:

- Blocking poll operations with timeout support.
- Support for multiple event consumers.

## Sensor Abstraction Layer

In general, the NVIDIA<sup>&reg;</sup> DriveWorks SAL implements sensor functionality into two modular layers:

- Device type: DriveWorks provides implementations for supported sensor interfaces (e.g. Radar, Lidar, Camera, etc.)
- Device-specific decoder: handles parsing and interpretation of raw data for a given sensor model.

In addition to the device decoders already implemented within the DriveWorks library, customers may implement additional device decoders using plug-in interfaces. The list of interfaces to be implemented is defined in an interface header file that is specific to sensor type. As part of application deployment, this pluggable interface is compiled as a shared object and loaded by DriveWorks at runtime.

The current API definition for the SAL can be found in the DriveWorks SDK documentation.

## Sensor Manager

Sensor Manager is agnostic to specific sensor type, instead, it provides unified access to all types of sensors that are supported by SAL. Sensors instantiation is handled through the `::dwRigHandle_t` (see @ref rig_mainsection). An example configuration file is shown below. This configuration defines a one-camera setup (ar0231 RCCB cameras), CAN bus, and a GPS sensor.

```
{
    "rig": {
        "sensors": [
            {
                "name": "camera:front:center:60fov",
                "parameter": "camera-type=ar0231-rccb,camera-group=a,camera-count=1,camera-mask=0001,siblingIndex=0",
                "properties": {
                    "Model": "ftheta",
                    "width": "1920",
                    "height": "1208",
                    "cx": "960",
                    "cy": "604",
                    "bw-poly": "0.0 0.000545421498827636 -1.6216719633103e-10 -4.64720492990289e-12 2.85224527762934e-16"
                },
                "protocol": "camera.gmsl",
                "nominalSensor2Rig": {
                    "quaternion": [-0.502444, 0.507493, -0.497444, 0.492494],
                    "t": [1.749, -0.1, 1.47]
                },
                "sensor2Rig": {
                    "quaternion": [-0.502444, 0.507493, -0.497444, 0.492494],
                    "t": [1.749, -0.1, 1.47]
                }
            },
            {
                "name": "can:vehicle",
                "nominalSensor2Rig": { "quaternion": [ 0.0, 0.0, 0.0, 1.0 ],
                                       "t": [ 0.0, 0.0, 0.0 ] },
                "parameter": "device=can0",
                "properties": null,
                "protocol": "can.socket",
                "sensor2Rig": { "quaternion": [ 0.0, 0.0, 0.0, 1.0 ],
                                "t": [ 0.0, 0.0, 0.0 ] }
            },
            {
                "name": "gps:dataspeed",
                "nominalSensor2Rig": {
                    "quaternion": [0.0, 0.0, 0.0, 1.0],
                    "t": [-0.2, -0.1, 0.4]
                },
                "parameter": "can-proto=can.socket,can-params=device=can0",
                "properties": null,
                "protocol": "gps.dataspeed",
                "sensor2Rig": {
                    "quaternion": [0.0, 0.0, 0.0, 1.0],
                    "t": [-0.2, -0.1, 0.4]
                }
            }
        ],
    },
    "version": 2
}
```

## Grouping camera sensors

One of the features of the module is a grouping events from synchronized cameras with the ability to request them in scope 
with single acquiring call. In order to combine multiple cameras into a group, the user reflects this in configuration file. 
`camera-group` sensor parameter indicates to which named group camera should be refered. In the following example both cameras 
are assigned to the same group. Therefore, each acquired event will contain two frames, one for each camera. 

```
{
    "rig": {
        "sensors": [
            {
                "name": "camera:front:center:60fov",
                "parameter": "file=front_camera.mp4,camera-group=main_group",
                "protocol": "camera.virtual"
            },
            {
                "name": "camera:rear:center:60fov",
                "parameter": "file=rear_camera.mp4,camera-group=main_group",
                "protocol": "camera.virtual"
            },
        ],
    },
    "version": 2
}
```

If the user wants to split the events, it is enough to specify different names in `camera-group` parameter:

```
{
    "rig": {
        "sensors": [
            {
                "name": "camera:front:center:60fov",
                "parameter": "file=front_camera.mp4,camera-group=separate-group-1",
                "protocol": "camera.virtual"
            },
            {
                "name": "camera:rear:center:60fov",
                "parameter": "file=rear_camera.mp4,camera-group=separate-group-2",
                "protocol": "camera.virtual"
            },
        ],
    },
    "version": 2
}
```

In this case acquired event will contain single frame from single camera with minimum timestamp.

It is assumed that all virtual cameras being processed by the module are synchronized. Therefore
by default they all combined into a single group. That is, the first example is the same as 
the following:

```
{
    "rig": {
        "sensors": [
            {
                "name": "camera:front:center:60fov",
                "parameter": "file=front_camera.mp4",
                "protocol": "camera.virtual"
            },
            {
                "name": "camera:rear:center:60fov",
                "parameter": "file=rear_camera.mp4",
                "protocol": "camera.virtual"
            },
        ],
    },
    "version": 2
}
```

#### Input

The following are the input requirements to the Sensor Manager:

- All sensors used by the Sensor Manager (defined in a JSON configuration file) must be supported by the Sensor Abstraction Layer (SAL).
- All sensor data must be timestamped (as provided by SAL).

#### Output

The only output requirements for the Sensor Manager is to provide the respective sensor data as specified at the given event when the function to acquire the next event is called.

## Relevant Tutorials

- @ref sensormanager_usecase1