Alert Verification Performance | NVIDIA Blueprint for Video Search and Summarization

Overview

The VSS Alert Bridge verification pipeline processes live RTSP streams through CV detection and analytics (RTVI-CV and video-analytics microservices) and a VLM stage to classify and confirm security alerts. Benchmarks measure end-to-end (E2E) latency, VLM and VST stage latency, and verified alert throughput across the NVIDIA B200, DGX H100 SXM, L40S, RTX PRO 4500, RTX PRO 6000, DGX Spark, and AGX Thor platforms using RT-DETR and Grounding DINO (GDINO) with Cosmos3 Reasoner (CR3) Nano, Super, and Edge model variants. All platforms are reported for single-stream and multi-stream operation. Stream counts differ by platform and configuration, see the tables below.

Test Configuration

Parameter	Value
VSS Release	3.3.0
Platforms tested	NVIDIA B200, DGX H100 SXM, L40S, RTX PRO 4500, RTX PRO 6000, DGX Spark, AGX Thor
CV models	RT-DETR, Grounding DINO (GDINO)
VLM	Cosmos3 Reasoner (CR3) Nano, Super, and Edge (platform- and model-dependent)
VLM precision	BF16, FP8, NVFP4 (platform- and model-dependent)
Stream concurrency	1 to 80 on dGPU platforms (1, 20, 30, 40, 50, 60, and 80, platform-dependent), 1 to 14 on edge platforms; see tables below
Configured stream FPS	30
Video resolution	1920x1080 (1080p)
OSL	1 and 100
Test duration	1 hour per configuration

Performance by Platform

B200

DGX H100 SXM

L40S

RTX PRO 4500

RTX PRO 6000

DGX Spark

AGX Thor

OSL 1 (alert verification)

CV Model	VLM	Precision	Streams	E2E Avg (s)	E2E P90 (s)	E2E P99 (s)	VLM P90 (s)	VST P90 (s)	Verified/min/stream
RT-DETR	Nano	BF16	1	0.69	1.00	2.35	0.90	0.09	5.38
RT-DETR	Nano	BF16	20	0.61	1.00	2.34	0.82	0.09	5.31
RT-DETR	Nano	BF16	40	0.76	1.70	2.43	0.93	0.09	5.24
RT-DETR	Nano	FP8	1	0.67	0.99	2.34	0.91	0.09	5.42
RT-DETR	Nano	FP8	20	0.61	0.99	2.34	0.81	0.09	5.30
RT-DETR	Nano	FP8	40	0.76	1.64	2.42	0.93	0.09	5.24
RT-DETR	Nano	NVFP4	1	0.68	1.00	2.34	0.90	0.09	5.38
RT-DETR	Nano	NVFP4	20	0.63	1.00	2.35	0.82	0.09	5.31
RT-DETR	Nano	NVFP4	40	0.77	1.69	2.43	0.94	0.09	5.24
RT-DETR	Super	BF16	1	0.75	0.99	2.33	0.90	0.09	5.38
RT-DETR	Super	BF16	20	0.75	1.79	2.43	1.24	0.09	5.30
RT-DETR	Super	BF16	40	0.99	2.16	2.48	1.60	0.09	5.24
RT-DETR	Super	FP8	1	0.72	0.98	2.32	0.91	0.09	5.77
RT-DETR	Super	FP8	20	0.67	1.54	2.40	0.88	0.09	5.30
RT-DETR	Super	FP8	40	0.86	1.99	2.46	1.34	0.09	5.24
RT-DETR	Super	NVFP4	1	0.68	0.98	2.32	0.90	0.09	5.38
RT-DETR	Super	NVFP4	20	0.63	0.99	2.33	0.81	0.09	5.30
RT-DETR	Super	NVFP4	40	0.80	1.84	2.44	0.96	0.09	5.24
RT-DETR	Edge	BF16	1	0.86	1.75	2.42	0.94	0.09	7.08
RT-DETR	Edge	BF16	20	0.95	2.16	2.47	1.67	0.09	5.42
RT-DETR	Edge	BF16	40	1.18	2.26	2.50	1.79	0.09	5.38
GDINO	Nano	BF16	1	7.62	36.11	57.61	0.94	0.09	3.58
GDINO	Nano	BF16	60	7.75	21.21	58.18	1.27	0.09	0.70
GDINO	Nano	BF16	80	8.80	34.26	57.43	1.66	0.09	0.18
GDINO	Nano	FP8	1	7.64	36.11	57.61	0.94	0.09	3.58
GDINO	Nano	FP8	60	10.02	14.15	14.91	0.69	0.09	0.03
GDINO	Nano	FP8	80	6.63	9.52	13.63	0.96	0.09	1.61
GDINO	Nano	NVFP4	1	7.60	36.11	57.61	0.94	0.09	3.58
GDINO	Nano	NVFP4	80	8.87	35.17	57.52	0.99	0.09	0.13
GDINO	Super	BF16	1	7.77	36.22	57.62	0.94	0.09	3.57
GDINO	Super	BF16	60	8.92	34.04	57.40	1.81	0.09	0.12
GDINO	Super	BF16	80	5.52	9.12	9.97	0.93	0.09	1.68
GDINO	Super	FP8	1	7.65	36.11	57.61	0.93	0.09	3.58
GDINO	Super	NVFP4	1	7.65	36.11	57.61	0.93	0.09	3.58
GDINO	Super	NVFP4	60	10.16	14.25	14.92	0.49	0.09	0.02
GDINO	Super	NVFP4	80	6.31	9.52	9.97	0.93	0.09	1.18
GDINO	Edge	BF16	1	7.05	33.33	57.33	0.95	0.09	4.00
GDINO	Edge	BF16	60	10.07	14.23	14.92	0.73	0.09	0.03
GDINO	Edge	BF16	80	5.44	9.12	9.97	0.94	0.09	1.71

OSL 100 (captioning)

CV Model	VLM	Precision	Streams	E2E Avg (s)	E2E P90 (s)	E2E P99 (s)	VLM P90 (s)	VST P90 (s)	Verified/min/stream
RT-DETR	Nano	BF16	1	2.06	3.17	4.82	0.94	0.09	7.55
RT-DETR	Nano	BF16	20	2.99	4.71	8.07	1.97	0.09	7.47
RT-DETR	Nano	FP8	1	1.98	2.98	4.80	0.94	0.09	7.55
RT-DETR	Nano	FP8	20	2.63	4.55	5.85	1.81	0.09	7.48
RT-DETR	Nano	NVFP4	1	2.02	3.25	4.83	0.94	0.09	6.65
RT-DETR	Nano	NVFP4	20	2.63	4.53	4.99	1.80	0.09	7.47
RT-DETR	Super	BF16	1	4.96	9.09	9.91	2.86	0.09	7.52
RT-DETR	Super	BF16	20	19.68	29.95	56.94	25.80	0.09	6.54
RT-DETR	Super	FP8	1	2.46	4.56	4.96	1.73	0.09	6.62
RT-DETR	Super	FP8	20	7.71	14.03	27.52	8.13	0.09	7.47
RT-DETR	Super	NVFP4	1	2.33	4.33	4.93	1.50	0.09	7.53
RT-DETR	Super	NVFP4	20	3.86	7.54	9.80	2.89	0.09	7.47
RT-DETR	Edge	BF16	1	2.48	4.54	4.95	1.79	0.09	7.52
RT-DETR	Edge	BF16	20	4.89	8.89	9.94	4.02	0.09	7.47
GDINO	Nano	BF16	1	8.77	27.17	56.46	0.96	0.09	5.32
GDINO	Nano	BF16	60	11.86	14.51	22.43	0.92	0.09	0.08
GDINO	Nano	BF16	80	9.82	26.16	56.23	0.96	0.09	0.07
GDINO	Nano	FP8	1	8.93	27.17	56.46	1.13	0.09	5.32
GDINO	Nano	FP8	60	11.49	14.41	14.94	0.93	0.09	0.03
GDINO	Nano	NVFP4	1	8.82	26.71	56.34	1.61	0.09	5.28
GDINO	Nano	NVFP4	80	10.07	26.31	56.22	1.33	0.09	0.13
GDINO	Super	BF16	1	9.52	26.71	56.34	1.88	0.09	5.28
GDINO	Super	BF16	60	12.34	14.33	14.93	1.85	0.09	0.00
GDINO	Super	FP8	1	10.48	26.71	56.34	1.90	0.09	5.28
GDINO	Super	FP8	60	12.63	14.74	29.32	0.97	0.09	0.07
GDINO	Super	NVFP4	1	8.94	26.71	56.34	1.59	0.09	5.28
GDINO	Super	NVFP4	60	12.01	14.47	14.98	0.96	0.09	0.05
GDINO	Super	NVFP4	80	11.20	38.70	110.08	1.88	0.09	0.26
GDINO	Edge	BF16	1	9.19	27.17	56.46	1.78	0.09	5.32
GDINO	Edge	BF16	80	11.41	46.10	111.99	1.91	0.09	0.26

Configurations whose E2E average latency exceeds 30 s are not shown, because the stream count is beyond what the platform can sustain for that configuration.

All benchmarks use input streams at 1080p, H.264 input.

For dGPU platform tests (NVIDIA B200, DGX H100 SXM, L40S, RTX PRO 4500, and RTX PRO 6000), two GPUs were used for the Nano and Edge models: one GPU dedicated to RTVI-CV and one GPU dedicated to RTVI-VLM. Super-model runs used three GPUs. On L40S and RTX PRO 4500, the LLM used by the pipeline is served remotely rather than on the local GPUs.

Results are shown for OSL 1 (alert verification) and OSL 100 (captioning). GDINO end-to-end latency is dominated by the upstream CV and analytics stage, so GDINO E2E P90 and P99 are much higher than RT-DETR at the same stream count. Verified/min/stream is the number of verified alerts per minute for each active stream.

As stream counts increase, the average CV frame rate per stream may decrease, which causes a drop in Verified/min/stream.