> For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.nvidia.com/vss/dev/system-components/middleware/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/vss/_mcp/server. # Middleware The Middleware components provide the storage, messaging, schema, and observability infrastructure that the VSS microservices and agents depend on. * [Video IO & Storage (VIOS)](/system-components/middleware/video-io-storage-vios) - Video ingest, storage, and retrieval services. * [API Gateway / MCP](/system-components/middleware/api-gateway-mcp) - Programmatic access to video analytics data and capabilities via REST and MCP. * [Message Broker](/system-components/middleware/message-broker) - Exchanges data and control messages between services. * [NvSchema](/system-components/middleware/nvschema) - The metadata schema used across the pipeline. * [Database](/system-components/middleware/database) - Persistence for analytics, incidents, and verified results. * [Observability](/system-components/middleware/observability) - The Grafana, Prometheus, cAdvisor, DCGM Exporter, and Node Exporter stack deployed with the Blueprint. ## Docs - [Video IO & Storage (VIOS)](https://docs.nvidia.com/vss/dev/system-components/middleware/video-io-storage-vios.md): - [VIOS Microservices](https://docs.nvidia.com/vss/dev/system-components/middleware/video-io-storage-vios/vios-microservices.md): - [NVStreamer](https://docs.nvidia.com/vss/dev/system-components/middleware/video-io-storage-vios/nvstreamer.md): - [VST API Reference](https://docs.nvidia.com/vss/dev/system-components/middleware/video-io-storage-vios/vst-api-reference.md): - [VST Sensor Management API](https://docs.nvidia.com/vss/dev/system-components/middleware/video-io-storage-vios/vst-api-reference/vst-sensor-management-api.md): - [VST Sensor Management API Reference](https://docs.nvidia.com/vss/dev/system-components/middleware/video-io-storage-vios/vst-api-reference/vst-sensor-management-api/vst-sensor-management-api-reference.md): The Sensor Management Service is a comprehensive system designed to facilitate the administration and control of sensor-related functionalities. It streamlines sensor operations, - [VST Live Stream Management API](https://docs.nvidia.com/vss/dev/system-components/middleware/video-io-storage-vios/vst-api-reference/vst-live-stream-management-api.md): - [VST Live Stream Management API Reference](https://docs.nvidia.com/vss/dev/system-components/middleware/video-io-storage-vios/vst-api-reference/vst-live-stream-management-api/vst-live-stream-management-api-reference.md): The Live Stream Service is designed to facilitate the live or real time streaming of sensors. This service encompasses: - Live streaming: Live streaming of sensors using WebRTC pr - [VST Replay Stream Management API](https://docs.nvidia.com/vss/dev/system-components/middleware/video-io-storage-vios/vst-api-reference/vst-replay-stream-management-api.md): - [VST Replay Stream Management API Reference](https://docs.nvidia.com/vss/dev/system-components/middleware/video-io-storage-vios/vst-api-reference/vst-replay-stream-management-api/vst-replay-stream-management-api-reference.md): The Replay Stream Service supports recorded sensor streaming using WebRTC, listing recorded streams, capturing snapshots, monitoring stream status in real time, playback controls (fast forward, rewind, seeking, play, and pause), performance statistics, and queries for stream position and metadata. - [VST Record Stream Management API](https://docs.nvidia.com/vss/dev/system-components/middleware/video-io-storage-vios/vst-api-reference/vst-record-stream-management-api.md): - [VST Record Stream Management API Reference](https://docs.nvidia.com/vss/dev/system-components/middleware/video-io-storage-vios/vst-api-reference/vst-record-stream-management-api/vst-record-stream-management-api-reference.md): VST Stream Recorder MS is used to manage recording of sensors NOTE: All JSON responses from this service are emitted with HTTP header `Content-Type: text/plain`. The body itself is - [VST Proxy Stream Management API](https://docs.nvidia.com/vss/dev/system-components/middleware/video-io-storage-vios/vst-api-reference/vst-proxy-stream-management-api.md): - [VST Proxy Stream Management API Reference](https://docs.nvidia.com/vss/dev/system-components/middleware/video-io-storage-vios/vst-api-reference/vst-proxy-stream-management-api/vst-proxy-stream-management-api-reference.md): VST Replay Stream MS is used to manage proxy RTSP URLs sensors NOTE: All JSON responses from this service are emitted with HTTP header `Content-Type: text/plain`. The body itself i - [VST Storage Management API](https://docs.nvidia.com/vss/dev/system-components/middleware/video-io-storage-vios/vst-api-reference/vst-storage-management-api.md): - [VST Storage Management API Reference](https://docs.nvidia.com/vss/dev/system-components/middleware/video-io-storage-vios/vst-api-reference/vst-storage-management-api/vst-storage-management-api-reference.md): The Storage Management is designed to facilitate the management of media files. For nvstreamer the media files are static media files that are scanned from the specified directory - [API Gateway / MCP](https://docs.nvidia.com/vss/dev/system-components/middleware/api-gateway-mcp.md): - [Video Analytics API](https://docs.nvidia.com/vss/dev/system-components/middleware/api-gateway-mcp/video-analytics-api.md): - [Video Analytics API Reference](https://docs.nvidia.com/vss/dev/system-components/middleware/api-gateway-mcp/video-analytics-api/video-analytics-api-reference.md): Request and response schemas for the Video Analytics API - [Video Analytics MCP](https://docs.nvidia.com/vss/dev/system-components/middleware/api-gateway-mcp/video-analytics-mcp.md): - [Message Broker](https://docs.nvidia.com/vss/dev/system-components/middleware/message-broker.md): - [Kafka](https://docs.nvidia.com/vss/dev/system-components/middleware/message-broker/kafka.md): - [Redis](https://docs.nvidia.com/vss/dev/system-components/middleware/message-broker/redis.md): - [NvSchema](https://docs.nvidia.com/vss/dev/system-components/middleware/nvschema.md): - [JSON Schema](https://docs.nvidia.com/vss/dev/system-components/middleware/nvschema/json-schema.md): - [Protobuf Schema](https://docs.nvidia.com/vss/dev/system-components/middleware/nvschema/protobuf-schema.md): - [Database](https://docs.nvidia.com/vss/dev/system-components/middleware/database.md): - [ELK](https://docs.nvidia.com/vss/dev/system-components/middleware/database/elk.md): - [Observability](https://docs.nvidia.com/vss/dev/system-components/middleware/observability.md): ## API Docs - Video Analytics API Reference > metrics [Get average speed](https://docs.nvidia.com/vss/dev/system-components/middleware/api-gateway-mcp/video-analytics-api/video-analytics-api-reference/metrics/controller-metrics-get-average-speed.md) - Video Analytics API Reference > metrics [Get flow rate](https://docs.nvidia.com/vss/dev/system-components/middleware/api-gateway-mcp/video-analytics-api/video-analytics-api-reference/metrics/controller-metrics-get-flowrate.md) - Video Analytics API Reference > metrics [Get average speed and flow rate](https://docs.nvidia.com/vss/dev/system-components/middleware/api-gateway-mcp/video-analytics-api/video-analytics-api-reference/metrics/controller-metrics-get-average-speed-with-flowrate.md) - Video Analytics API Reference > metrics [Get average speed and travel time](https://docs.nvidia.com/vss/dev/system-components/middleware/api-gateway-mcp/video-analytics-api/video-analytics-api-reference/metrics/controller-metrics-get-average-speed-with-travel-time.md) - Video Analytics API Reference > metrics [Get tripwire counts](https://docs.nvidia.com/vss/dev/system-components/middleware/api-gateway-mcp/video-analytics-api/video-analytics-api-reference/metrics/controller-metrics-get-tripwire-counts.md) - Video Analytics API Reference > metrics [Get tripwire-based occupancy](https://docs.nvidia.com/vss/dev/system-components/middleware/api-gateway-mcp/video-analytics-api/video-analytics-api-reference/metrics/controller-metrics-get-tripwire-based-occupancy.md) - Video Analytics API Reference > metrics [Reset place occupancy](https://docs.nvidia.com/vss/dev/system-components/middleware/api-gateway-mcp/video-analytics-api/video-analytics-api-reference/metrics/controller-metrics-reset-occupancy.md) - Video Analytics API Reference > metrics [Get tripwire count histogram](https://docs.nvidia.com/vss/dev/system-components/middleware/api-gateway-mcp/video-analytics-api/video-analytics-api-reference/metrics/controller-metrics-get-tripwire-histogram.md) - Video Analytics API Reference > metrics [Get FOV occupancy](https://docs.nvidia.com/vss/dev/system-components/middleware/api-gateway-mcp/video-analytics-api/video-analytics-api-reference/metrics/controller-metrics-get-fov-occupancy.md) - Video Analytics API Reference > metrics [Get FOV occupancy histogram](https://docs.nvidia.com/vss/dev/system-components/middleware/api-gateway-mcp/video-analytics-api/video-analytics-api-reference/metrics/controller-metrics-get-fov-histogram.md) - Video Analytics API Reference > metrics [Get ROI occupancy](https://docs.nvidia.com/vss/dev/system-components/middleware/api-gateway-mcp/video-analytics-api/video-analytics-api-reference/metrics/controller-metrics-get-roi-occupancy.md) - Video Analytics API Reference > metrics [Get ROI occupancy histogram](https://docs.nvidia.com/vss/dev/system-components/middleware/api-gateway-mcp/video-analytics-api/video-analytics-api-reference/metrics/controller-metrics-get-roi-histogram.md) - Video Analytics API Reference > metrics [Get mutually exclusive ROI occupancy](https://docs.nvidia.com/vss/dev/system-components/middleware/api-gateway-mcp/video-analytics-api/video-analytics-api-reference/metrics/controller-metrics-get-occupancy-based-on-mutually-exclusive-roi.md) - Video Analytics API Reference > metrics [Get tracker-based occupancy](https://docs.nvidia.com/vss/dev/system-components/middleware/api-gateway-mcp/video-analytics-api/video-analytics-api-reference/metrics/controller-metrics-get-occupancy-tracker.md) - Video Analytics API Reference > metrics [Get tracker occupancy histogram](https://docs.nvidia.com/vss/dev/system-components/middleware/api-gateway-mcp/video-analytics-api/video-analytics-api-reference/metrics/controller-metrics-get-tripwire-occupancy-histogram.md) - Video Analytics API Reference > metrics [Get space utilization histogram](https://docs.nvidia.com/vss/dev/system-components/middleware/api-gateway-mcp/video-analytics-api/video-analytics-api-reference/metrics/controller-metrics-get-space-utilization-histogram.md) - Video Analytics API Reference > metrics [Get last processed timestamp](https://docs.nvidia.com/vss/dev/system-components/middleware/api-gateway-mcp/video-analytics-api/video-analytics-api-reference/metrics/controller-metrics-get-last-processed-timestamp.md) - Video Analytics API Reference > metrics [Get road-segment speeds](https://docs.nvidia.com/vss/dev/system-components/middleware/api-gateway-mcp/video-analytics-api/video-analytics-api-reference/metrics/controller-road-network-average-segment-speed.md) - Video Analytics API Reference > tracker [Get unique object count](https://docs.nvidia.com/vss/dev/system-components/middleware/api-gateway-mcp/video-analytics-api/video-analytics-api-reference/tracker/controller-tracker-get-unique-object-count.md) - Video Analytics API Reference > tracker [Get unique object count and locations](https://docs.nvidia.com/vss/dev/system-components/middleware/api-gateway-mcp/video-analytics-api/video-analytics-api-reference/tracker/controller-tracker-get-unique-object-count-with-locations.md) - Video Analytics API Reference > tracker [Get tracked objects](https://docs.nvidia.com/vss/dev/system-components/middleware/api-gateway-mcp/video-analytics-api/video-analytics-api-reference/tracker/controller-tracker-get-unique-objects.md) - Video Analytics API Reference > tracker [Get behavior locations](https://docs.nvidia.com/vss/dev/system-components/middleware/api-gateway-mcp/video-analytics-api/video-analytics-api-reference/tracker/controller-tracker-get-behavior-locations.md) - Video Analytics API Reference > tracker [Get the latest RTLS or AMR record](https://docs.nvidia.com/vss/dev/system-components/middleware/api-gateway-mcp/video-analytics-api/video-analytics-api-reference/tracker/controller-tracker-get-last-record.md) - Video Analytics API Reference > frames [Get raw frame metadata](https://docs.nvidia.com/vss/dev/system-components/middleware/api-gateway-mcp/video-analytics-api/video-analytics-api-reference/frames/controller-frames-get-raw-frames.md) - Video Analytics API Reference > frames [Get enhanced frame metadata](https://docs.nvidia.com/vss/dev/system-components/middleware/api-gateway-mcp/video-analytics-api/video-analytics-api-reference/frames/controller-frames-get-enhanced-frames.md) - Video Analytics API Reference > frames [Get BEV frame metadata](https://docs.nvidia.com/vss/dev/system-components/middleware/api-gateway-mcp/video-analytics-api/video-analytics-api-reference/frames/controller-frames-get-bev-frames.md) - Video Analytics API Reference > frames [Get frame-based alerts](https://docs.nvidia.com/vss/dev/system-components/middleware/api-gateway-mcp/video-analytics-api/video-analytics-api-reference/frames/controller-frames-get-alerts.md) - Video Analytics API Reference > frames [Get high-confidence frame objects](https://docs.nvidia.com/vss/dev/system-components/middleware/api-gateway-mcp/video-analytics-api/video-analytics-api-reference/frames/controller-frames-get-high-confidence-objects.md) - Video Analytics API Reference > frames [Get proximity clusters](https://docs.nvidia.com/vss/dev/system-components/middleware/api-gateway-mcp/video-analytics-api/video-analytics-api-reference/frames/controller-frames-get-proximity-clusters.md) - Video Analytics API Reference > frames [Get frame PTS](https://docs.nvidia.com/vss/dev/system-components/middleware/api-gateway-mcp/video-analytics-api/video-analytics-api-reference/frames/controller-frames-get-pts.md) - Video Analytics API Reference > behavior [Get behaviors](https://docs.nvidia.com/vss/dev/system-components/middleware/api-gateway-mcp/video-analytics-api/video-analytics-api-reference/behavior/controller-behavior-get-behavior.md) - Video Analytics API Reference > behavior [Get behavior PTS](https://docs.nvidia.com/vss/dev/system-components/middleware/api-gateway-mcp/video-analytics-api/video-analytics-api-reference/behavior/controller-behavior-get-pts.md) - Video Analytics API Reference > alerts [Get alerts](https://docs.nvidia.com/vss/dev/system-components/middleware/api-gateway-mcp/video-analytics-api/video-analytics-api-reference/alerts/controller-alerts-get-alerts.md) - Video Analytics API Reference > alerts [Get severe alerts by place or sensor](https://docs.nvidia.com/vss/dev/system-components/middleware/api-gateway-mcp/video-analytics-api/video-analytics-api-reference/alerts/controller-alerts-get-severe-alerts.md) - Video Analytics API Reference > incidents [Get incidents](https://docs.nvidia.com/vss/dev/system-components/middleware/api-gateway-mcp/video-analytics-api/video-analytics-api-reference/incidents/controller-incidents-get-incidents.md) - Video Analytics API Reference > incidents [Get severe incidents by place or sensor](https://docs.nvidia.com/vss/dev/system-components/middleware/api-gateway-mcp/video-analytics-api/video-analytics-api-reference/incidents/controller-incidents-get-severe-incidents.md) - Video Analytics API Reference > clustering [Get behavior clusters](https://docs.nvidia.com/vss/dev/system-components/middleware/api-gateway-mcp/video-analytics-api/video-analytics-api-reference/clustering/controller-clustering-get-clustering-results.md) - Video Analytics API Reference > clustering [Label a behavior cluster](https://docs.nvidia.com/vss/dev/system-components/middleware/api-gateway-mcp/video-analytics-api/video-analytics-api-reference/clustering/controller-clustering-add-cluster-label.md) - Video Analytics API Reference > events [Get tripwire events](https://docs.nvidia.com/vss/dev/system-components/middleware/api-gateway-mcp/video-analytics-api/video-analytics-api-reference/events/controller-events-get-tripwire-events.md) - Video Analytics API Reference > events [Get AMR events](https://docs.nvidia.com/vss/dev/system-components/middleware/api-gateway-mcp/video-analytics-api/video-analytics-api-reference/events/controller-events-get-amr-events.md) - Video Analytics API Reference > events [Get ROI events](https://docs.nvidia.com/vss/dev/system-components/middleware/api-gateway-mcp/video-analytics-api/video-analytics-api-reference/events/controller-events-get-roi-events.md) - Video Analytics API Reference > config [Upload a configuration file](https://docs.nvidia.com/vss/dev/system-components/middleware/api-gateway-mcp/video-analytics-api/video-analytics-api-reference/config/controller-config-upload-file.md) - Video Analytics API Reference > config [Update service configuration](https://docs.nvidia.com/vss/dev/system-components/middleware/api-gateway-mcp/video-analytics-api/video-analytics-api-reference/config/controller-config-update.md) - Video Analytics API Reference > config [Get configuration update status](https://docs.nvidia.com/vss/dev/system-components/middleware/api-gateway-mcp/video-analytics-api/video-analytics-api-reference/config/controller-config-get-update-status.md) - Video Analytics API Reference > config [Upsert sensor calibration](https://docs.nvidia.com/vss/dev/system-components/middleware/api-gateway-mcp/video-analytics-api/video-analytics-api-reference/config/controller-config-upsert-calibration.md) - Video Analytics API Reference > config [Delete sensor calibration](https://docs.nvidia.com/vss/dev/system-components/middleware/api-gateway-mcp/video-analytics-api/video-analytics-api-reference/config/controller-config-delete-sensor-calibration.md) - Video Analytics API Reference > config [Get calibration configuration](https://docs.nvidia.com/vss/dev/system-components/middleware/api-gateway-mcp/video-analytics-api/video-analytics-api-reference/config/controller-config-get-calibration-config.md) - Video Analytics API Reference > config [Get calibration modification timestamp](https://docs.nvidia.com/vss/dev/system-components/middleware/api-gateway-mcp/video-analytics-api/video-analytics-api-reference/config/controller-config-get-calibration-last-modified-timestamp.md) - Video Analytics API Reference > config [Get road-network configuration](https://docs.nvidia.com/vss/dev/system-components/middleware/api-gateway-mcp/video-analytics-api/video-analytics-api-reference/config/controller-config-get-road-network-config.md) - Video Analytics API Reference > config [Get USD asset configuration](https://docs.nvidia.com/vss/dev/system-components/middleware/api-gateway-mcp/video-analytics-api/video-analytics-api-reference/config/controller-config-get-usd-assets-config.md) - Video Analytics API Reference > config [Upload calibration images](https://docs.nvidia.com/vss/dev/system-components/middleware/api-gateway-mcp/video-analytics-api/video-analytics-api-reference/config/controller-config-upload-calibration-images.md) - Video Analytics API Reference > config [Get a calibration image](https://docs.nvidia.com/vss/dev/system-components/middleware/api-gateway-mcp/video-analytics-api/video-analytics-api-reference/config/controller-config-get-calibration-image.md) - Video Analytics API Reference > config [Get calibration image metadata](https://docs.nvidia.com/vss/dev/system-components/middleware/api-gateway-mcp/video-analytics-api/video-analytics-api-reference/config/controller-config-get-calibration-image-metadata.md) - Video Analytics API Reference > config [Delete calibration images](https://docs.nvidia.com/vss/dev/system-components/middleware/api-gateway-mcp/video-analytics-api/video-analytics-api-reference/config/controller-config-delete-calibration-images.md) - Video Analytics API Reference > sensor [Find sensors covering a floor-plan coordinate](https://docs.nvidia.com/vss/dev/system-components/middleware/api-gateway-mcp/video-analytics-api/video-analytics-api-reference/sensor/controller-sensor-get-sensor-lookup.md) - Video Analytics API Reference > livez [Check service health](https://docs.nvidia.com/vss/dev/system-components/middleware/api-gateway-mcp/video-analytics-api/video-analytics-api-reference/livez/controller-livez-is-alive.md) ## OpenAPI Specification The raw OpenAPI 3.1 specification for this API is available at: - [OpenAPI JSON](https://docs.nvidia.com/vss/openapi.json) - [OpenAPI YAML](https://docs.nvidia.com/vss/openapi.yaml)