AI-NVR Mobile Application Internals

Device Access

The mobile app has two launcher entry points. One is for direct access to the AI-NVR device where the device access will be based on <device ip>:port rather than through the cloud endpoint. The second entry point is for remote access to the device through the cloud.

../_images/device_access_flow.png

User Login

On app launch with cloud based access, the first thing the app does is checking for the logged-in user with a valid token. In the case of first launch or expired token - a login flow is launched as described below.

../_images/user_login_flow.png

Authorization

../_images/user_authorization.png

Communication with the AI-NVR device

The app uses MVVM pattern.

Every request is initiated from UI classes (usually fragments or dialogs) while an Observer is configured to receive the response from the ViewModel and handle it.

The call to an API is routed to the module repository through the ViewModel of the corresponding module and executes the request using Retrofit. The response is delivered back to the ViewModel, which calls for the UI update using the LiveData and triggers the event on the Observer.

Video Storage Toolkit (VST)

VST API is responsible for connected cameras, streams management, and actions. See the below list of VST APIs which are being used use in the app.

Sensor:

  • GET sensor/list: Get list of sensors/cameras with some details

  • GET sensor/streams: Get list of sensor streams information

Streams:

  • POST replay/stream/seek: Seek forward (+10s), seek backward (-10s), rewind and fast forward

  • POST replay/stream/pause: Pause the recorded stream playback

  • POST replay/stream/resume: Resume the recorded stream playback

  • POST replay/stream/stop: Stop webrtc streaming

  • POST replay/stream/swap: Switch stream type from live to recorded

  • POST replay/stream/start: Start webrtc streaming

  • GET replay/stream/status: Get stream error status and playback status like PLAYING, PAUSED, NOT PLAYING

  • GET replay/stream/stats: Get stream stats like frame rate, total encoded and decoded frames

  • GET replay/stream/query: Get stream last played timestamp and related metadata, if available

The following flow demonstrates the fetching of connected cameras.

../_images/fetch_camera_list.png

Playing/Pausing a camera from the list is triggered by setClickListenerState() in CameraStackAdapter class, which overrides onRequestStreamState interface and eventually calls to a function from a CameraStackViewModel class. Every Media Player requests which changes the webrtc player states (Play/Pause/Stop) is initiated from UI classes (usually fragments or dialogs) -> ViewModel -> RtcClient -> Service class -> Module which executing the request using Retrofit. The following flow demonstrates play of one of the streams from a CameraStackAdapter.

../_images/play_camera_stream.png

eMDAT APIs

eMDAT API is responsible for managing gems - FOV (Field of View), Tripwire, and ROI (Region of Interest). It is also responsible for retrieving analytical data from gem data. See the below list of eMDAT API functions which are used in the app.

FOV:

  • GET metrics/occupancy/fov/histogram: List count of people in camera field of view

Tripwire:

  • GET metrics/tripwire/histogram: List trip event count and alerts

  • GET config/tripwire: List all tripwire configs

  • POST config/tripwire: Create tripwires

  • DELETE config/tripwire: Delete tripwires

  • GET config/tripwire/history: List historical tripwire configs

  • GET config/rule/alerts/tripwire: List tripwire alert rules

  • POST config/rule/alerts/tripwire: Create tripwire alert rules

  • DELETE config/rule/alerts/tripwire: Delete tripwire alert rule

  • GET config/rule/alerts/tripwire/history: List historical tripwire alert rule

ROI:

  • GET metrics/occupancy/roi/histogram: List roi count and alerts

  • GET config/roi: List all roi configs

  • POST config/roi: Create rois

  • DELETE config/roi: Delete ROI

  • GET config/roi/history: List historical roi configs

  • GET config/rule/alerts/roi: List roi alert rules

  • POST config/rule/alerts/roi: Create roi alert rules

  • DELETE config/rule/alerts/roi: Delete ROI alert rule

  • GET config/rule/alerts/roi/history: List historical roi alert rule

Alerts:

  • GET alerts: List all alerts

The following diagram demonstrates a simplified flow of selecting a tripwire and fetching occupancy data.

../_images/fetch_gems.png

Enable/Disable real-time alerts

These APIs on the reference cloud are responsible for enabling and disabling real-time alerts for an Jetson-mobile app cloud-based connection.

  • POST api/enable-notification: Enable real-time alerts.

  • POST api/disable-notification: Disable real-time alerts.

The following diagram demonstrates a simplified flow of enabling/disabling and receiving real-time alerts for both direct device and cloud-based access.

../_images/alert_notification.png