Metropolis Microservices for Jetson

Metropolis Microservices for Jetson (MMJ) is a platform that simplifies development, deployment and management of Edge AI applications on NVIDIA Jetson. It provides a modular & extensible architecture for developers to distill large complex applications into smaller modular microservice with APIs to integrate into other apps & services. To that extent, the platform leverages existing pieces in the Metropolis ecosystem such as Video Storage Toolkit (VST) and DeepStream while adding new ones (analytics) to provide several building blocks to quickly assemble fully functional edge AI systems. Addressing system maturity is another objective by enabling functionality needed for productization including security, monitoring and alerts, IoT/API gateways, and deployment - all available as out of the box functionality that developers can readily leverage.

_images/moj_overview.png

Software Stack

The Metropolis Microservices stack diagram is shown below. It is based on Jetpack 6.0 running on Orin AGX & NX systems. On top of Jetpack are three main classes of software: platform services, Metropolis microservices and applications.

_images/moj_stack.jpg

Metropolis Microservices

Metropolis Microservices provide the foundation for building video and sensor processing applications at the edge, and comprise:

  • Video Storage Toolkit: discovery, ingestion and management of camera and sensor streams, video and sensor data storage, video streaming

  • DeepStream: Real-time processing of multiple video (camera) and sensor streams through deep learning based inference using pluggable models, tracking and custom CV algorithms

  • Analytics: streaming analytics of inference metadata to produce metrics and alerts based on gem capabilities

At the core of Metropolis on Jetson is the use of REST APIs. Microservices present REST APIs to enable configuration and retrieval of results. Developers create end applications to address real world use cases by building on top of these APIs. Device APIs are presented external to the system through the API Gateway (a.k.a Ingress) Platform Services (see below) that can be configured and used out of the box.

Applications are built by configuring and instantiating the microservices along with overlying use case specific logic that interacts with microservices using the APIs. Use of REST APIs allows clients to interact with systems remotely and securely through HTTP. Microservices and application logic run as containers on the system, and deployed as a bundle using docker compose. Containerized deployment allows software to be packaged and independently and deployed remotely.

This release provides a reference application for AI network video recorder (NVR) bundling various microservices together. comprising together through the API mechanism allows microservices functionality to be also accessed remotely. Microservices and application containers are bundled together and deployed through an “app bundle” based on docker-compose utility.

Platform services

Platform software are domain agnostic services supporting commonly needed system functionality including:

  • Storage: to provision external storage and allocate to various microservices

  • Monitoring: visualize both system and application metrics using dashboards

  • Firewall: control ingress and egress network traffic to the system

  • IoT gateway: remote, secure access of device APIs to enable rich clients such as mobile apps

Reference Cloud

To enable remote, secure access of the APIs in an Metropolis Microservices based Jetson system, a reference cloud is provided with capabilities below:

  • “Always on” TCP connection from device to this cloud through secure provisioning

  • API gateway to enable calls to be routed to the appropriate devices based on ID using the TCP connection

  • User account support with capability for users to claim devices for access

  • Authorization support to control access

Reference Application

The release includes a reference Android application to visualize how an end user client can interact with the system. The app showcases various capabilities of Metropolis microservices including live and recorded video streaming through webRTC, analytics creation and visualization and alerts (see images below). The app interacts with the system solely using REST APIs, either accessed directly (for quick start purposes), or through the cloud securely (for maturation).

_images/app_metrics.png _images/app_streaming.png

Disaggregation

While the canonical form of the software stack is that both Metropolis Microservices and Platform Services are installed in completion for full functionality, the general philosophy is that various pieces in the software stack could be installed in a disaggregated manner as motivated by specific use cases.

To understand cases for adding these in a piecemeal manner, a developer could start with base Jetpack 6.0 installation and directly install one or more Metropolis Microservices. Choice of specific microservices would depend on functionality relevant to their particular case and whether they have custom functionality. For instance, developers bringing their non DeepStream based inference can leverage VST for ingesting videos but use RTSP streams output out of that for feeding their inference block.

Documentation Organization

The rest of the documentation is organized as below. We recommend following the documentation in the same order.

  • The Quick Start Guide provides instructions for getting started with deploying the Metropolis Microservices stack on Orin hardware, showcasing streaming video, inference and analytics capabilities.

  • Refer to the Application Microservices sections to understand various Metropolis building blocks:

    • VST: Video Storage Toolkit (VST) for camera discovery, video storage, video streaming

    • DeepStream: DeepStream based real-time inference of video using PeopleNet deep learning model

    • Overview: People analytics functionality

  • Refer to the Platform Services for a description of Platform Services that provide a collection of general, reusable Linux services necessary for building mature, production grade systems on Jetson.

  • Refer to AI NVR for a description of extending the ‘Quick Start’ deployment to build a performant, mature AI application in the form of a Network Video Recorder using the platform. Initially deploy without cloud integration for simplicity.

  • Refer to AI-NVR Mobile Application User Guide for a rich Android based client to interact with the AI-NVR system and view video, analytics and alerts generated by the Metropolis Microservices.

  • Refer to Reference Cloud for Jetson devices running Metropolis Microservices for information about using the reference cloud design to make your system production ready by addressing security, user accounts, remote access etc.

  • Redeploy AI-NVR using AI NVR setup instructions but with reference cloud integration enabled. This showcases the fully-featured deployment of the AI-NVR reference example using the complete Metropolis Microservices software offering.