DeepStream Perception

Overview

Jetson Platform Services offers DeepStream 7.0 based perception service supporting optimized, real-time, and multi-stream out of the box object detection and tracking support. In combination with the SDR and VST microservices, this also supports automatic addition of camera streams. The processing includes object detection using the PeopleNet model from NVIDIA by default or the optional use of YOLOv8s as described below, followed by object tracking using the NvDCF tracker plugin available in DeepStream. The output of the application is metadata based on the Metropolis schema sent to the Redis message bus using the msgbroker plugin.

The modified Jetson Platform Services DeepStream 7.0 container has the following changes in comparison to the standard samples container:

• Precompiled version of Service Maker deepstream-test5 app

• PeopleNet2.6 model and calibration files as well as many precompiled engine files for various Jetson platforms and batch sizes

• Precompiled tracker engine files

• Precompiled YOLOv8s DeepStream plugin

• OpenCV install necessary for YOLOv8s

Jetson Platform Services uses the sample deepstream-test5 app based on the new Service Maker framework supported as part of DeepStream 7.0 to programmatically assemble DeepStream pipelines, source code for which is available as part of the native DeepStream install and as part of the DeepStream Docker container. Options passed into this app are as follows:

• -s option specifies a source list config file. If using a config with nvmultiurisrcbin as the reference configs provided with AI-NVR do, the add/remove API within DeepStream is enabled.

• -c option specifies a pipeline config file containing the various components and connections within your programs pipeline.

• -l option specifies the label file for your model.

• --perf-measurement-interval-sec option specifies how frequently in seconds to print fps info.

  #example for running on Orin AGX
  $ /opt/nvidia/deepstream/deepstream-7.0/service-maker/sources/apps/deepstream_test5_app/build/deepstream-test5-app -s /ds-config-files/pn26/service-maker/source-list-0_agx.yaml,/ds-config-files/pn26/service-maker/source-list-1_agx.yaml -c /ds-config-files/pn26/service-maker/ds-config-0_agx.yaml,/ds-config-files/pn26/service-maker/ds-config-1_agx.yaml -l /ds-config-files/pn26/labels.txt --perf-measurement-interval-sec 5
  

In order to run multiple pipelines, you must specify multiple source list and config files separated by commas as seen in the above example.

On Orin AGX and Orin NX16, these pipelines use DLA 0 & 1 through their gie config files (config/deepstream/pn26/config_infer_primary_*):

enable-dla=1 # 0 => disable, 1 => enable
use-dla-core=0 # dla core number => 0, 1, etc.

On Orin NX8, only a single DLA is available and thus only one is used, while on Orin Nano GPU is utilized.

Note that user can alternatively incorporate the legacy test5 application (from DeepStream 6.4 and included by default in the DeepStream 7.0 container) as part of their Jetson Platform Services based systems provided the integration points with rest of the Jetson Platform Services architecture in terms of stream addition API and metadata output in the Metropolis schema are preserved.

DeepStream config files can be found in the config/deepstream/pn26 folder of the Docker Compose repo. Each config is marked with the device type at the end of the filename - agx, nx16, nx8, or nano. This folder is mounted in the DeepStream Docker container. Depending on which compose file run (compose_agx.yaml for Orin AGX, compose_nx16.yaml for Orin NX16, compose_nx8.yaml for Orin NX8, compose_nano.yaml for Orin Nano), the corresponding DeepStream config files are passed into the DeepStream application. Configs for both the Service Maker and legacy versions of the DeepStream test5 app are provided for all supported systems.

An additional container called SDR is used to automatically add and remove streams from the DeepStream application based on events that occur in VST. An overview of how DeepStream and VST interface with other components can be seen below:

GitHub Repo

As a reference, the Dockerfile for the JPS DeepStream container and initialization script for YOLOv8s is open sourced and available on GitHub on the jetson-platform-services repository under the inference/perception folder.

Configuration

No changes should need to be made to the DeepStream config files. Up to 8 streams per DLA are supported (16 total) on Orin AGX, 4 streams per DLA are supported (8 total) on Orin NX16, 4 streams total on Orin NX8, and 4 streams total on Orin Nano. Notable points about the configuration include:

• Use of PeopleNet 2.6 (PN2.6) unpruned model for superior accuracy or YOLOv8s for multi class support

• use of DLA for PN2.6 to offload inference from GPU on Orin AGX, Orin NX16, and Orin NX8

• Inference done every alternate frame based on the interval parameter within the primary-gie section

• Use of NvDCF multi-object tracker that supports running on PVA backend for PN2.6 (on Orin AGX, NX16, NX8) for further reducing the load on GPU

• Use of Redis message broker for metadata output

• Use of dynamic stream addition to add streams using the SDR microservices (see below)

In order to increase or reduce the number of streams per pipeline from the default values, the WDM_WL_THRESHOLD value under the compose yaml file will need to be changed. Similarly, the max_devices_supported value in the VST config (config/vst/vst_config.json) may also need to be edited to support the necessary number of streams. For best performance, batch size options set in the DeepStream configs should be modified, as well as generating and using engine files specific to the new batch size and available hardware (ie. DLAs).

SDR (Sensor Distribution and Routing)

Stream addition to DeepStream is based on a dedicated microservice named SDR to discover streams from VST and add them to DeepStream on the fly. SDR is used to automatically add and remove streams from DeepStream using the dynamic stream addition feature in DeepStream, triggered based on events from VST. When a stream is added to VST and is able to stream, an event is sent by VST to Redis with the following format:

{
    "alert_type": "camera_status_change",
    "created_at": "2023-11-20T23:51:04Z",
    "event": {
        "camera_id": "MOJD7",
        "camera_name": "36398191-d48a-4260-b574-239fd59f156f",
        "camera_url": "rtsp://172.17.170.143/live/36398191-d48a-4260-b574-239fd59f156f",
        "change": "camera_streaming"
    },
    "source": "vst"
}

SDR picks this up, then determines which pipeline in DeepStream currently has availability and adds it to that.

Sometimes when SDR starts up, VST may already have streams added and will not resend a message on Redis. This can occur for instance if SDR was restarted. To solve this, on startup SDR queries the VST live streams endpoint to fetch streams that have already been added. It then adds these streams as well to DeepStream.

SDR keeps track of which pipeline each stream was added to. This is so that when a stream is deleted, it knows which pipeline to send the delete request to. Each DeepStream pipeline has a different port that needs to be specified when using the add or remove API call.

To add a stream to DeepStream, a POST request must be made to the http://localhost:9010/api/v1/stream/add endpoint. Note that localhost may need to be changed to the proper IP and the port from 9010 to something else - by default port 9010 is used for pipeline 1 and port 9011 for pipeline 2, this is specified in the DeepStream config files. The following JSON must also be included in the request:

{
    "alert_type": "camera_status_change",
    "created_at": "2024-05-01T07:49:16Z",
    "event": {
        "camera_id": "Amcrest_3",
        "camera_name": "0227ed9f-de24-495e-b46f-b9d2b8122dd5",
        "camera_url": "rtsp://172.17.170.143/live/0227ed9f-de24-495e-b46f-b9d2b8122dd5",
        "change": "camera_streaming"
    },
    "source": "vst"
}

To remove a stream from DeepStream, a POST request must be made to the http://localhost:9010/api/v1/stream/remove endpoint. Again, localhost may need to be changed to the proper IP and specify the correct port. The following JSON must also be included in the request:

{
    "alert_type": "camera_status_change",
    "created_at": "2024-05-01T07:49:16Z",
    "event": {
        "camera_id": "Amcrest_3",
        "camera_name": "0227ed9f-de24-495e-b46f-b9d2b8122dd5",
        "camera_url": "rtsp://172.17.170.143/live/0227ed9f-de24-495e-b46f-b9d2b8122dd5",
        "change": "camera_remove"
    },
    "source": "vst"
}

Note that only the camera_id, camera_name, camera_url, and change values are necessary, everything else is optional. Also note that these two API endpoints are interfacing directly with DeepStream and thus if used manually, will skip over SDR and may cause issues. Internally, SDR uses these endpoints and JSON formats to interface with DeepStream. These add and remove endpoints are provided in the default DeepStream test5 application when using [source-list] in the legacy config file or using nvmultiurisrcbin in the Service Maker config.

SDR also supports reconciliation in the event some streams are missed from VST. The SDR sidecar container occasionally queries both the SDR cache and current streams in VST to make sure they are in sync and calls the add/remove function as necessary in SDR in the event they are out of sync.

There are various SDR configuration options that can be set via container environment variables and can be seen used in the Docker Compose yaml files. These options are specified below. Note that the sample docker compose files do override some of the default values specified here.

Variable	Description	Default Value
PORT	Port for the SDR REST API server	4000
WDM_WL_SPEC	Local cache file SDR uses in case of a crash/restart	“./tests/data_wl.yaml”
WDM_CLUSTER_CONFIG_FILE	IP/port values for each DeepStream pipeline	“docker_cluster_config.json”
WDM_MSG_KEY	Redis key SDR should listen on	“vst.event”
WDM_WL_REDIS_MSG_FIELD	key within the Redis JSON to parse	“sensor.id”
WDM_WL_ADD_URL	Service add endpoint	“/api/v1/stream/add”
WDM_WL_DELETE_URL	Service remove endpoint	“/api/v1/stream/remove”
WDM_WL_HEALTH_CHECK_URL	Service health check endpoint	“/api/v1/stream/add”
WDM_WL_CHANGE_ID_ADD	“change” type to listen for on Redis to know when to add a stream to DeepStream	“camera_streaming”
WDM_PRELOAD_WORKLOAD	Config file with streams which should be auto loaded on startup other than those already available on VST	“./event_pre-roll.json”
WDM_CLEAR_DATA_WL	True if the local cache should be cleared at startup	False
WDM_KFK_ENABLE	True if Kafka should be enabled	True
WDM_DS_SWAP_ID_NAME	True if ID and Name should be swapped from the VST Redis message before adding to DeepStream	False
WDM_VALIDATE_BEFORE_ADD	True if the add JSON should be validated before being sent to DeepStream (ie., make sure name, ID, and URL are set)	False
WDM_PRELOAD_DELAY_FOR_DS_API	True if SDR should wait for the service health check to pass before starting to add streams	False
WDM_WL_THRESHOLD	Maximum number of streams SDR should add per pipeline	8
WDM_CLUSTER_TYPE	Cluster type being used	“docker”
WDM_POD_WATCH_DOCKER_DELAY	How frequently in seconds SDR should check for container restarts and update stream distribution	0.05
WDM_DS_STATUS_CHECK	True if DeepStream return status code should be read rather than ignored	False
WDM_RESTART_DS_ON_ADD_FAIL	True if the DeepStream should be restarted on multiple failed add/remove events	False
WDM_DISABLE_WERKZEUG_LOGGING	True if Werkzeug logging should be disabled to reduce overall logging	False
WDM_WL_OBJECT_NAME	The workload object name for SDR events sent to Redis and must be unique across instances of SDR running on the same system	“testapp”
WDM_CONSUMER_GRP_ID	The Redis consumer group ID and must be unique across instances of SDR running on the same system	“consumer-grp-id-3”
WDM_CLUSTER_CONTAINER_NAMES	The container names to watch for restarts/crashes for the SDR recovery mechanism	“["sdr", "deepstream", "vst"]”
WDM_NIMS_FORMAT	True if the NIMS format JSON (needed for GenAI) should be sent out rather than the default	False

the docker_cluster_config.json file specifies details on available DeepStream pipelines for SDR to send streams to. This file is pointed to by the WDM_CLUSTER_CONFIG_FILE environment variable and must be in the following format:

{
    "moj-ds-01": {
        "provisioning_address": "localhost:9010",
        "process_type": "docker"
    },
    "moj-ds-02": {
        "provisioning_address": "localhost:9011",
        "process_type": "docker"
    }
}

Where:

• The number of keys are equal to the total number of services you want to distribute to.

• The keys for all must be unique.

• provisioning_address must be set to the IP/port of each service (ie. in the case of DeepStream, the IP is localhost but the port differs for the two pipelines).

• process_type must be set to docker if using Docker Compose.

SDR will fill up each in order. In other words, SDR will first add WDM_WL_THRESHOLD sources to “moj-ds-01” before adding sources to “moj-ds-02”. In the event a source is removed from “moj-ds-01”, that spot will be filled by the next source added.

PeopleNet Based Inference

The DeepStream module uses NVIDIA PeopleNet version 2.6 for object detection model. Pre-generated engine files are used to reduce startup time. Engine files vary by device used (Orin AGX, Orin NX16, Orin NX8, Orin Nano) as does batch size, hence the varying DeepStream configs depending on device. Tracking distance of 1 is used, which means inference is run every alternate frames to limit inference utilization. Hybrid clustering is used to process network output.

Note that PeopleNet 2.6 engine file used with DeepStream to be run on DLA with TensorRT in Jetpack 6.0 need to generated offline due to a known issue documented in the TensorRT release notes, captured here for convenience:

When building TensorRT engines for DLA, there is a known issue that entire DLA subgraphs listed in “Layers Running on DLA” (seen with TensorRT’s verbose mode) cannot be built/eventually fall back to GPU with the message “{ForeignNode[…]} cannot be compiled by DLA, falling back to GPU”. This has been observed with the two ResNet-based models: PeopleNet v2.6 and TrafficCamNet from TAO. In both cases, this issue can be fixed by changing TensorRT’s default DLA SRAM pool size of 1 MiB to 0.5 MiB. Using trtexec, this can be achieved by adding the argument --memPoolSize=dlaSRAM:0.5 when building the TensorRT engine.

YOLOv8s Based Inference

DeepStream inference service now supports the YOLO (v8s) model for object detection. YOLO is a state of the art object detection model supporting several features such as low latency, efficient GPU resource utilization thereby supporting large stream count and multi class support. For more background of various YOLO models released over the years, refer this link: https://arxiv.org/abs/2304.00501

Due to licensing, YOLOv8s based engine and model files are not distributed as part of our DeepStream container. Rather, a script is provided for the user to download the model file and generate engine and calibration files on their own. By default, this script and provided configs only support Orin AGX and Orin NX16. Unlike PeopleNet2.6, YOLOv8s uses legacy test5 rather than the Service Maker version. DLA is also not utilized with YOLOv8s. See YOLOv8s Setup below for details on setting up YOLOv8s.

NvDCF based tracker

The NvDCF tracker used in DeepStream has been configured for inference being run every alternate frame. NvDCF is particularly suitable for people analytics in indoor, retail settings based on its visual tracking capability that makes it resilient to full or partial occlusions. Refer to the ll-config-file variable definition in the DeepStream config file for tracker configurations being used.

The default configs for Orin AGX, NX16, and NX8 make use of a PVA backend for the tracker via the VPI unified API in order to offload computation from the GPU. Sub-batching is also utilized to process multiple streams per PVA instance. On Orin AGX, a CUDA backend is used in conjunction with PVA for some streams due to the high stream count supported. The Orin Nano does not have the PVA accelerator and thus tracker computation is done on GPU.

Redis

Detection events from DeepStream are outputted by the MessageBroker to the Redis test stream. The latest event can be viewed using the following command:

sudo docker exec -it redis redis-cli

xinfo STREAM test

Events are in the following format:

"{\n  \"version\" : \"4.0\",\n  \"id\" : \"65704\",\n  \"@timestamp\" : \"2024-04-25T21:56:43.715Z\",\n  \"sensorId\" : \"MOJD7_2\",\n  \"objects\" : [\n    \"655|1364.74|230.769|1487.88|431.31|Person|-0.1|#||||||\",\n    \"607|137.03|204.356|282.938|583.155|Person|-0.1|#||||||\"\n  ]\n}"

These messages contain information about the latest timestamp processed, sensorId, and object detections. The Analytics Microservice then consumes these messages as its input. View the Redis page for more information.

Logs

DeepStream logs are saved in the /data/logging-volume/deepstream.log file by default. Live logs can be viewed using the command tail -f /data/logging-volume/deepstream.log. The current streams being processed by DeepStream can be viewed in this file as well as the current fps values for each stream. Generally, these should be around 30fps with default config options, but may be lower if the input stream is a lower fps value. A sample snippet of the DeepStream logs can be seen below:

Active sources : 4
**PERF:  FPS 3 (Avg)    FPS 2 (Avg)     FPS 1 (Avg)     FPS 0 (Avg)
Thu Apr 25 17:26:26 2024
**PERF:
MOJD2_2mbps[e9ad9678-0506-4c3a-8621-9dc865bb1cb1] 30.03 (30.15) MOJD3[e728c793-cf9c-4b6c-b0c3-3565d001d9d9] 30.03 (30.86)       MOJD2_5mbps[9dab9be9-b1b0-4892-b6b9-7f7adfe08bb7] 30.03 (30.40)    MOJD6[555f23f8-4b8f-457d-8246-207af6e06263] 30.03 (30.92)
Active sources : 0
Thu Apr 25 17:26:26 2024

It specifies the number of sources for each pipeline, the current date/time, fps values, stream IDs, and stream names. The first fps is the current fps value while the number in parenthesis is the average for that stream. Stream name is set when adding a stream via VST while stream ID is randomly generated by VST as a unique identifier.

SDR logs are saved in the /data/logging-volume/sdr-deepstream.log file by default. Live logs can be viewed using the command tail -f /data/logging-volume/sdr-deepstream.log. The current streams added to DeepStream via SDR can be viewed in this file as well as the latest relevant messages SDR picks up over Redis.

Deployment

Example DeepStream config files and container information can be found on NGC as part of the reference AI-NVR application. PeopleNet2.6 configs can be found under the ai_nvr/config/deepstream/pn26 while YOLOv8s configs can be found under ai_nvr/config/deepstream/yolov8s. Different docker compose configs are provided for easy out of the box use of either PeopleNet2.6 or YOLOv8s. Keep in mind that only the PeopleNet2.6 engine and model files are included by default in the provided DeepStream container. Commands to deploy using the default PeopleNet2.6 model with the rest of the sample AI-NVR reference application are provided in the Quick Start guide. Before deployment, first follow the Quick Start Guide to set up your system with Jetson Platform Services.

Modifying DeepStream Configs

You may modify DeepStream configs as necessary for your use case. This may include things such as changing batch size, changing the tracker configs, changing primary-gie configs to modify models used or hardware accelerators used, enabling or disabling various sinks such as display outputs or different message brokers, etc. A few things to keep in mind while modifying these configs:

• If changing batch size or configs for the primary-gie, you may also need to rename (and possibly generate) new engine files with the new batch size. Similarly, you may need to change the WDM_WL_THRESHOLD SDR environment variable as defined in the docker compose yaml config to let SDR know the max number of streams it should add to a given DeepStream pipeline.

• Having various width/height options between different components of the DeepStream pipeline can cause large performance hits. Same goes with different batch sizes across different components. It is recommended to keep these in sync across each piece of the DeepStream pipeline.

• SDR uses the DeepStream “nvmultiurisrcbin” source-bin to add/remove streams. Analytics uses the DeepStream Redis output as defined by the “msgbroker” sink. Monitoring DeepStream FPS reporting in Grafana uses logs outputted to /data/logging-volume/deepstream.log on the host. If any of these aspects are modified, some other components of the reference AI-NVR application may not function as expected.

Additional considerations for the two reference models are provided below.

PeopleNet2.6 Deployment

If you decide to modify the example configs, you may need to regenerate engine files to support your new configurations. As described in the above PeopleNet2.6 section, this may need to be done manually using a trtexec command due to a known issue. An example command is provided below to generate the DLA0 batch size 8 config file for Orin AGX:

/usr/src/tensorrt/bin/trtexec --uff=./peoplenet_V2.6_resnet34_hr_noreg_ph2_dla.uff --int8 --uffInput=input_1,3,544,960 --output=output_bbox/BiasAdd,output_cov/Sigmoid --calib=./peoplenet_V2.6_resnet34_hr_noReg_ph2_dla.cache --batch=8 --duration=100 --useDLACore=0 --allowGPUFallback --memPoolSize=dlaSRAM:0.5 --verbose --saveEngine=./dla0_pn26_jp6_halfmem_bs8-8623.engine

This can be run from the /pn26-files directory of the provided Jetson Platform Services DeepStream container. This is where the necessary PeopleNet2.6 dependencies (uff and calib files) are located. Options passed into trtexec can be modified as needed.

Commands to deploy PeopleNet2.6 with the AI-NVR reference application can be found under the Run IVA Application section of the quick start guide.

YOLOv8s Deployment

Note

For each dataset an user elects to use, the user is responsible for checking if the dataset license is fit for the intended purpose.

YOLOv8s Setup

Before deployment, you must run the initialization script. Again, note that by default, this script and provided configs only support Orin AGX and Orin NX16. Run the below command from the ai-nvr folder in order to start the script:

Orin AGX: sudo docker run -v ./yolov8s:/yolov8s -v ./config/deepstream:/ds-config-files --runtime nvidia nvcr.io/nvidia/jps/deepstream:7.0-jps-v1.1.1 /yolov8s-files/yolov8s_setup.sh --agx

Orin NX16: sudo docker run -v ./yolov8s:/yolov8s -v ./config/deepstream:/ds-config-files --runtime nvidia nvcr.io/nvidia/jps/deepstream:7.0-jps-v1.1.1 /yolov8s-files/yolov8s_setup.sh --nx16

This command starts the DeepStream docker container, then runs the yolov8s_setup.sh script. This script does the following:

1. Creates required directories

2. Downloads the yolov8s.pt model file from Ultralytics (AGPL-3.0 license), the conversion script from this GitHub repo (MIT license), and the val2017 dataset from cocodataset.org (Creative Commons Attribution 4.0 License)

3. The python3-dev apt package is also installed, as well as the onnx, onnxsim, onnxruntime, ultralytics, and torch pip packages

4. 1000 random images are selected from the val2017 dataset to use for int8 calibration

5. The the .pt model file is converted to ONNX

6. DeepStream is run to generate the engine file and int8 calibration file

All files created by this command will be saved into the ./yolov8s folder. If you want to change the output directory, simply change ./yolov8s:/yolov8s in the run command replacing the first ./yolov8s with your desired directory on the host. If you do not keep this in the default location, provided configs to run with the rest of the AI-NVR stack may also need to be edited. In the event some error occurs, these generated files may need to be manually deleted before retrying. In this case, delete this folder and recreate it then retry running the script using the command above.

YOLOv8s default configs can be found under the config/deepstream/yolov8s directory. There are a few different config files here:

• cfg_redis.txt - Redis config file for DeepStream

• config_infer_* - the DeepStream GIE configs

• config_tracker_* - the DeepStream tracker configs

• labels.txt - labels file used with the model

• setup_config_* - DeepStream configs used in the initialization script to generate the engine and calibration files

• yolov8s-ds-config_* - the main DeepStream configs used when running the AI-NVR reference application with YOLOv8s

Different commands for Orin AGX and Orin NX16 are needed as the maximum stream count and batch size with expected performance (30fps on all streams) varies based on device type. On Orin AGX this is 8 streams, while on Orin NX16 it is 4 streams. Note that the device type selected here must also match the device type when running the docker compose up command below, otherwise a mismatch in batch size will occur causing engine and config file issues.

This process will take 15-30 minutes to complete. 4 key files will be needed to run YOLO, the first three of which are generated by the initialization script:

1. ./yolov8s/calib.table - the calibration file

2. ./yolov8s/model_b4_gpu0_int8.engine or ./yolov8s/model_b8_gpu0_int8.engine based on batch sizes specified in configs which by default are different for AGX and NX16 - these are the model engine files

3. ./yolov8s/yolov8s-dependencies/yolov8s.onnx - the converted .pt model file

4. ./config/deepstream/yolov8s/labels.txt - the labels file containing all model classes

If any of these are missing, go through the logs outputted by the script and look for any errors. The engine file name may be different if you modified any of the options (such as batch size) in the DeepStream configs.

Note that this script could also be run directly on the Jetson without the need for running in a docker container. In this case, some of the paths referenced in the script may need to change and the host system will need DeepStream installed alongside CUDA, openCV and the DeepStream function to generate the TRT engine from this repo. You can also use the model and calibration file generated here for your own DeepStream use case outside of AI-NVR by adding a primary-gie section similar to that found in the example configs.

Once these model files are compiled, you may copy them to other devices as well. Keep in mind that if these are copied to other devices, the same configs must also be run on the other system - in other words, if you compiled the engine files for a batch size of 4 (such as the default for NX16), you must run the engine/config files with the corresponding batch size of 4 otherwise you may run into errors or performance issues.

To generate new engine files with modified batch size or other similar options, simply modify the setup_config_agx.txt or setup_config_nx16.txt files under ai_nvr/config/deepstream/yolov8s, modify the batch size set in the initialization script, and rerun the script. Make sure to also modify the “model-engine-file” otherwise you risk overwriting the old engine file. Once done, edit your deployment config with similar modifications and deploy.

Running YOLOv8s with AI-NVR Reference Application

Docker compose configs are provided to run this with the rest of the AI-NVR stack. To run, use the commands below:

Orin AGX: sudo docker compose -f compose_agx_yolov8s.yaml up -d --force-recreate

Orin NX16: sudo docker compose -f compose_nx16_yolov8s.yaml up -d --force-recreate

To stop:

Orin AGX: sudo docker compose -f compose_agx_yolov8s.yaml down --remove-orphans

Orin NX16: sudo docker compose -f compose_nx16_yolov8s.yaml down --remove-orphans