Analytics Data

Overview

The Metropolis Microservices data flow initiates with video input from cameras or pre-recorded footage.

This media is streamed via VST, the microservice for media management & storage, which utilizes the Video Storage Toolkit (VST).

These RTSP streams are then processed by the Perception microservice, built on the DeepStream SDK. Key functions of the Perception microservice include object detection, tracking, and generating embeddings.

This output is subsequently relayed to Kafka, a message broker facilitating real-time data streaming and exchange. Kafka’s role is pivotal in distributing this data to various microservices for advanced analytics. These microservices, in turn, generate new messages and feed them back to Kafka for downstream use. In scenarios where users require custom real-time analytics, they have the option to either consume data from Kafka or produce data to Kafka through their microservices.

Data from Kafka is channeled to Logstash, which then indexes it into Elasticsearch. Elasticsearch, a versatile datastore, enables on-demand data querying.

A specific case is the Multi-Camera Tracking (MTMC) reference application, where the Behavior Analytics microservice archives behavior embeddings in Milvus, a dedicated vector database.

Both Elasticsearch and Milvus databases are accessible for queries through their respective client interfaces.

Finally, the Analytics and Tracking API provides an accessible means to retrieve and understand the analytics performed on the data. While these API endpoints cover a range of analytics, for tailored analytics needs, users can directly query the databases using the provided clients.

Streaming Data

  • Kafka topics used by various microservices are as follows:

    Parameters

    Name

    Description

    mdx-raw

    The topic where the Perception microservice sends frame metadata.

    mdx-behavior

    The topic for behavior data, handled by the Spark streaming pipeline. Refer to Behavior Processing for details.

    mdx-behavior-plus

    Used by the Spark streaming pipeline to send enhanced behavior data, like assigning cluster details to behaviors.

    mdx-alerts

    For all anomaly data sent by the Spark streaming pipeline.

    mdx-tripwire

    Handles all tripwire-related events, also managed by the Spark streaming pipeline.

    mdx-frames

    This topic is for enriched frames metadata, which may include additional metrics and analytics, such as FOV & ROI occupancy. Its usage by the Spark streaming pipeline varies.

    mdx-mtmc

    Used by the Multi-Camera Tracking (MTMC Analytics) microservice for sending MTMC-related data.

    mdx-rtls

    Used by the Real Time Location System (RTLS) microservice for sending RTLS-related data.

    mdx-amr

    Used to send Autonomous Mobile Robot’s (AMR) location and route change data.

    mdx-notification

    For sending various notifications such as calibration updates, configuration changes, and sensor management.

Note

  • For more details of Kafka, check the Kafka quickstart guide.

  • You can use this Kafka Python library to produce and consume messages. You can find more details regarding the library and sample code over here.

On-demand Data

  • Elasticsearch indices created for the Metropolis Microservices include:

    Parameters

    Index Pattern

    Description

    mdx-raw-*

    Index pattern for mdx-raw data, with daily data in separate indices named as mdx-raw-YYYY-MM-dd.

    mdx-behavior-*

    Similar daily index pattern for mdx-behavior data, mdx-behavior-YYYY-MM-dd.

    mdx-cluster-labels

    Stores cluster IDs, model versions, and cluster names.

    mdx-alerts-*

    Daily indexing pattern for mdx-alerts data.

    mdx-tripwire-*

    Daily indices for mdx-tripwire data.

    mdx-occupancy-reset

    Stores occupancy reset data for various locations.

    mdx-frames-*

    Daily indexing pattern for mdx-frames data.

    mdx-mtmc-*

    Daily indices for mdx-mtmc data.

    mdx-rtls-*

    Daily indexing pattern for mdx-rtls data.

    mdx-amr-location-*

    Daily indices for mdx-amr-location data.

    mdx-route-change-*

    Daily indexing pattern for mdx-route-change data.

    mdx-calibration

    Stores current calibration states.

    mdx-calibration-audit

    Logs each calibration update.

    mdx-calibration-images

    Stores metadata of calibration images.

    mdx-configs

    Holds current configurations of various microservices.

    mdx-configs-audit

    Logs each update made to the microservices’ configurations.

Note

  • For more details of Elasticsearch, check the Elasticsearch quickstart guide.

  • You can use this Elasticsearch Python library to query the indices. You can find more details regarding the library and sample code over here.

  • Milvus Collections created for the Metropolis Microservices include:

    Parameters

    Collection

    Description

    mdxBehavior

    Stores mdx-behaviors along with their embeddings.

Note