> This page is for version 3.3.0 (default).
> For other versions, use one of these documentation indexes:
> - 3.3.0 (default): https://docs.nvidia.com/vss/3.3.0/llms.txt
> - Dev: https://docs.nvidia.com/vss/dev/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/vss/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/vss/_mcp/server.

# Alert Verification Performance

## Overview

The VSS Alert Bridge verification pipeline processes live RTSP streams through CV detection
and analytics (RTVI-CV and video-analytics microservices) and a VLM stage to classify and
confirm security alerts. Benchmarks measure end-to-end (E2E) latency, VLM and VST stage
latency, and verified alert throughput across the NVIDIA B200, DGX H100 SXM, L40S,
RTX PRO 4500, RTX PRO 6000, DGX Spark, and AGX Thor platforms using RT-DETR and
Grounding DINO (GDINO) with Cosmos3 Reasoner (CR3) Nano, Super, and Edge model
variants. All platforms are reported for single-stream and multi-stream
operation. Stream counts differ by platform and configuration, see the tables
below.

## Test Configuration

| Parameter             | Value                                                                                                                      |
| --------------------- | -------------------------------------------------------------------------------------------------------------------------- |
| VSS Release           | 3.3.0                                                                                                                      |
| Platforms tested      | NVIDIA B200, DGX H100 SXM, L40S, RTX PRO 4500, RTX PRO 6000, DGX Spark, AGX Thor                                           |
| CV models             | RT-DETR, Grounding DINO (GDINO)                                                                                            |
| VLM                   | Cosmos3 Reasoner (CR3) Nano, Super, and Edge (platform- and model-dependent)                                               |
| VLM precision         | BF16, FP8, NVFP4 (platform- and model-dependent)                                                                           |
| Stream concurrency    | 1 to 80 on dGPU platforms (1, 20, 30, 40, 50, 60, and 80, platform-dependent), 1 to 14 on edge platforms; see tables below |
| Configured stream FPS | 30                                                                                                                         |
| Video resolution      | 1920x1080 (1080p)                                                                                                          |
| OSL                   | 1 and 100                                                                                                                  |
| Test duration         | 1 hour per configuration                                                                                                   |

## Performance by Platform

#### B200

**OSL 1 (alert verification)**

| CV Model | VLM   | Precision | Streams | E2E Avg (s) | E2E P90 (s) | E2E P99 (s) | VLM P90 (s) | VST P90 (s) | Verified/min/stream |
| -------- | ----- | --------- | ------- | ----------- | ----------- | ----------- | ----------- | ----------- | ------------------- |
| RT-DETR  | Nano  | BF16      | 1       | 0.69        | 1.00        | 2.35        | 0.90        | 0.09        | 5.38                |
| RT-DETR  | Nano  | BF16      | 20      | 0.61        | 1.00        | 2.34        | 0.82        | 0.09        | 5.31                |
| RT-DETR  | Nano  | BF16      | 40      | 0.76        | 1.70        | 2.43        | 0.93        | 0.09        | 5.24                |
| RT-DETR  | Nano  | FP8       | 1       | 0.67        | 0.99        | 2.34        | 0.91        | 0.09        | 5.42                |
| RT-DETR  | Nano  | FP8       | 20      | 0.61        | 0.99        | 2.34        | 0.81        | 0.09        | 5.30                |
| RT-DETR  | Nano  | FP8       | 40      | 0.76        | 1.64        | 2.42        | 0.93        | 0.09        | 5.24                |
| RT-DETR  | Nano  | NVFP4     | 1       | 0.68        | 1.00        | 2.34        | 0.90        | 0.09        | 5.38                |
| RT-DETR  | Nano  | NVFP4     | 20      | 0.63        | 1.00        | 2.35        | 0.82        | 0.09        | 5.31                |
| RT-DETR  | Nano  | NVFP4     | 40      | 0.77        | 1.69        | 2.43        | 0.94        | 0.09        | 5.24                |
| RT-DETR  | Super | BF16      | 1       | 0.75        | 0.99        | 2.33        | 0.90        | 0.09        | 5.38                |
| RT-DETR  | Super | BF16      | 20      | 0.75        | 1.79        | 2.43        | 1.24        | 0.09        | 5.30                |
| RT-DETR  | Super | BF16      | 40      | 0.99        | 2.16        | 2.48        | 1.60        | 0.09        | 5.24                |
| RT-DETR  | Super | FP8       | 1       | 0.72        | 0.98        | 2.32        | 0.91        | 0.09        | 5.77                |
| RT-DETR  | Super | FP8       | 20      | 0.67        | 1.54        | 2.40        | 0.88        | 0.09        | 5.30                |
| RT-DETR  | Super | FP8       | 40      | 0.86        | 1.99        | 2.46        | 1.34        | 0.09        | 5.24                |
| RT-DETR  | Super | NVFP4     | 1       | 0.68        | 0.98        | 2.32        | 0.90        | 0.09        | 5.38                |
| RT-DETR  | Super | NVFP4     | 20      | 0.63        | 0.99        | 2.33        | 0.81        | 0.09        | 5.30                |
| RT-DETR  | Super | NVFP4     | 40      | 0.80        | 1.84        | 2.44        | 0.96        | 0.09        | 5.24                |
| RT-DETR  | Edge  | BF16      | 1       | 0.86        | 1.75        | 2.42        | 0.94        | 0.09        | 7.08                |
| RT-DETR  | Edge  | BF16      | 20      | 0.95        | 2.16        | 2.47        | 1.67        | 0.09        | 5.42                |
| RT-DETR  | Edge  | BF16      | 40      | 1.18        | 2.26        | 2.50        | 1.79        | 0.09        | 5.38                |
| GDINO    | Nano  | BF16      | 1       | 7.62        | 36.11       | 57.61       | 0.94        | 0.09        | 3.58                |
| GDINO    | Nano  | BF16      | 60      | 7.75        | 21.21       | 58.18       | 1.27        | 0.09        | 0.70                |
| GDINO    | Nano  | BF16      | 80      | 8.80        | 34.26       | 57.43       | 1.66        | 0.09        | 0.18                |
| GDINO    | Nano  | FP8       | 1       | 7.64        | 36.11       | 57.61       | 0.94        | 0.09        | 3.58                |
| GDINO    | Nano  | FP8       | 60      | 10.02       | 14.15       | 14.91       | 0.69        | 0.09        | 0.03                |
| GDINO    | Nano  | FP8       | 80      | 6.63        | 9.52        | 13.63       | 0.96        | 0.09        | 1.61                |
| GDINO    | Nano  | NVFP4     | 1       | 7.60        | 36.11       | 57.61       | 0.94        | 0.09        | 3.58                |
| GDINO    | Nano  | NVFP4     | 80      | 8.87        | 35.17       | 57.52       | 0.99        | 0.09        | 0.13                |
| GDINO    | Super | BF16      | 1       | 7.77        | 36.22       | 57.62       | 0.94        | 0.09        | 3.57                |
| GDINO    | Super | BF16      | 60      | 8.92        | 34.04       | 57.40       | 1.81        | 0.09        | 0.12                |
| GDINO    | Super | BF16      | 80      | 5.52        | 9.12        | 9.97        | 0.93        | 0.09        | 1.68                |
| GDINO    | Super | FP8       | 1       | 7.65        | 36.11       | 57.61       | 0.93        | 0.09        | 3.58                |
| GDINO    | Super | NVFP4     | 1       | 7.65        | 36.11       | 57.61       | 0.93        | 0.09        | 3.58                |
| GDINO    | Super | NVFP4     | 60      | 10.16       | 14.25       | 14.92       | 0.49        | 0.09        | 0.02                |
| GDINO    | Super | NVFP4     | 80      | 6.31        | 9.52        | 9.97        | 0.93        | 0.09        | 1.18                |
| GDINO    | Edge  | BF16      | 1       | 7.05        | 33.33       | 57.33       | 0.95        | 0.09        | 4.00                |
| GDINO    | Edge  | BF16      | 60      | 10.07       | 14.23       | 14.92       | 0.73        | 0.09        | 0.03                |
| GDINO    | Edge  | BF16      | 80      | 5.44        | 9.12        | 9.97        | 0.94        | 0.09        | 1.71                |

**OSL 100 (captioning)**

| CV Model | VLM   | Precision | Streams | E2E Avg (s) | E2E P90 (s) | E2E P99 (s) | VLM P90 (s) | VST P90 (s) | Verified/min/stream |
| -------- | ----- | --------- | ------- | ----------- | ----------- | ----------- | ----------- | ----------- | ------------------- |
| RT-DETR  | Nano  | BF16      | 1       | 2.06        | 3.17        | 4.82        | 0.94        | 0.09        | 7.55                |
| RT-DETR  | Nano  | BF16      | 20      | 2.99        | 4.71        | 8.07        | 1.97        | 0.09        | 7.47                |
| RT-DETR  | Nano  | FP8       | 1       | 1.98        | 2.98        | 4.80        | 0.94        | 0.09        | 7.55                |
| RT-DETR  | Nano  | FP8       | 20      | 2.63        | 4.55        | 5.85        | 1.81        | 0.09        | 7.48                |
| RT-DETR  | Nano  | NVFP4     | 1       | 2.02        | 3.25        | 4.83        | 0.94        | 0.09        | 6.65                |
| RT-DETR  | Nano  | NVFP4     | 20      | 2.63        | 4.53        | 4.99        | 1.80        | 0.09        | 7.47                |
| RT-DETR  | Super | BF16      | 1       | 4.96        | 9.09        | 9.91        | 2.86        | 0.09        | 7.52                |
| RT-DETR  | Super | BF16      | 20      | 19.68       | 29.95       | 56.94       | 25.80       | 0.09        | 6.54                |
| RT-DETR  | Super | FP8       | 1       | 2.46        | 4.56        | 4.96        | 1.73        | 0.09        | 6.62                |
| RT-DETR  | Super | FP8       | 20      | 7.71        | 14.03       | 27.52       | 8.13        | 0.09        | 7.47                |
| RT-DETR  | Super | NVFP4     | 1       | 2.33        | 4.33        | 4.93        | 1.50        | 0.09        | 7.53                |
| RT-DETR  | Super | NVFP4     | 20      | 3.86        | 7.54        | 9.80        | 2.89        | 0.09        | 7.47                |
| RT-DETR  | Edge  | BF16      | 1       | 2.48        | 4.54        | 4.95        | 1.79        | 0.09        | 7.52                |
| RT-DETR  | Edge  | BF16      | 20      | 4.89        | 8.89        | 9.94        | 4.02        | 0.09        | 7.47                |
| GDINO    | Nano  | BF16      | 1       | 8.77        | 27.17       | 56.46       | 0.96        | 0.09        | 5.32                |
| GDINO    | Nano  | BF16      | 60      | 11.86       | 14.51       | 22.43       | 0.92        | 0.09        | 0.08                |
| GDINO    | Nano  | BF16      | 80      | 9.82        | 26.16       | 56.23       | 0.96        | 0.09        | 0.07                |
| GDINO    | Nano  | FP8       | 1       | 8.93        | 27.17       | 56.46       | 1.13        | 0.09        | 5.32                |
| GDINO    | Nano  | FP8       | 60      | 11.49       | 14.41       | 14.94       | 0.93        | 0.09        | 0.03                |
| GDINO    | Nano  | NVFP4     | 1       | 8.82        | 26.71       | 56.34       | 1.61        | 0.09        | 5.28                |
| GDINO    | Nano  | NVFP4     | 80      | 10.07       | 26.31       | 56.22       | 1.33        | 0.09        | 0.13                |
| GDINO    | Super | BF16      | 1       | 9.52        | 26.71       | 56.34       | 1.88        | 0.09        | 5.28                |
| GDINO    | Super | BF16      | 60      | 12.34       | 14.33       | 14.93       | 1.85        | 0.09        | 0.00                |
| GDINO    | Super | FP8       | 1       | 10.48       | 26.71       | 56.34       | 1.90        | 0.09        | 5.28                |
| GDINO    | Super | FP8       | 60      | 12.63       | 14.74       | 29.32       | 0.97        | 0.09        | 0.07                |
| GDINO    | Super | NVFP4     | 1       | 8.94        | 26.71       | 56.34       | 1.59        | 0.09        | 5.28                |
| GDINO    | Super | NVFP4     | 60      | 12.01       | 14.47       | 14.98       | 0.96        | 0.09        | 0.05                |
| GDINO    | Super | NVFP4     | 80      | 11.20       | 38.70       | 110.08      | 1.88        | 0.09        | 0.26                |
| GDINO    | Edge  | BF16      | 1       | 9.19        | 27.17       | 56.46       | 1.78        | 0.09        | 5.32                |
| GDINO    | Edge  | BF16      | 80      | 11.41       | 46.10       | 111.99      | 1.91        | 0.09        | 0.26                |

#### DGX H100 SXM

**OSL 1 (alert verification)**

| CV Model | VLM   | Precision | Streams | E2E Avg (s) | E2E P90 (s) | E2E P99 (s) | VLM P90 (s) | VST P90 (s) | Verified/min/stream |
| -------- | ----- | --------- | ------- | ----------- | ----------- | ----------- | ----------- | ----------- | ------------------- |
| RT-DETR  | Nano  | BF16      | 1       | 0.93        | 2.16        | 2.47        | 1.75        | 0.09        | 4.48                |
| RT-DETR  | Nano  | BF16      | 20      | 0.92        | 2.10        | 2.46        | 1.56        | 0.09        | 5.31                |
| RT-DETR  | Nano  | BF16      | 40      | 1.72        | 3.60        | 4.86        | 2.58        | 0.09        | 5.23                |
| RT-DETR  | Nano  | FP8       | 1       | 0.90        | 2.13        | 2.46        | 1.64        | 0.09        | 4.48                |
| RT-DETR  | Nano  | FP8       | 20      | 0.94        | 2.12        | 2.46        | 1.57        | 0.09        | 5.30                |
| RT-DETR  | Nano  | FP8       | 40      | 1.40        | 2.39        | 4.52        | 1.89        | 0.09        | 5.24                |
| RT-DETR  | Super | FP8       | 1       | 1.02        | 2.21        | 2.47        | 1.76        | 0.09        | 5.38                |
| RT-DETR  | Super | FP8       | 20      | 1.20        | 2.27        | 2.48        | 1.80        | 0.09        | 5.31                |
| RT-DETR  | Super | FP8       | 40      | 2.73        | 4.62        | 7.57        | 4.40        | 0.09        | 5.34                |
| RT-DETR  | Edge  | BF16      | 1       | 1.13        | 2.28        | 2.48        | 1.84        | 0.09        | 5.45                |
| RT-DETR  | Edge  | BF16      | 20      | 1.28        | 2.28        | 2.77        | 1.85        | 0.09        | 4.98                |
| RT-DETR  | Edge  | BF16      | 40      | 1.83        | 4.00        | 4.90        | 2.79        | 0.09        | 5.32                |
| GDINO    | Nano  | BF16      | 1       | 7.91        | 36.22       | 57.62       | 1.73        | 0.09        | 3.57                |
| GDINO    | Nano  | BF16      | 30      | 8.21        | 9.56        | 11.26       | 0.94        | 0.09        | 1.14                |
| GDINO    | Nano  | BF16      | 40      | 9.77        | 13.80       | 14.93       | 0.91        | 0.09        | 0.50                |
| GDINO    | Nano  | BF16      | 50      | 10.48       | 14.26       | 14.93       | 0.87        | 0.09        | 0.03                |
| GDINO    | Nano  | FP8       | 1       | 7.89        | 36.22       | 57.62       | 1.35        | 0.09        | 3.57                |
| GDINO    | Nano  | FP8       | 30      | 7.84        | 9.51        | 9.96        | 0.93        | 0.09        | 1.23                |
| GDINO    | Nano  | FP8       | 40      | 9.27        | 13.19       | 14.82       | 0.86        | 0.09        | 0.05                |
| GDINO    | Nano  | FP8       | 50      | 11.05       | 14.37       | 14.99       | 0.84        | 0.09        | 0.33                |
| GDINO    | Super | FP8       | 1       | 8.08        | 36.22       | 57.62       | 1.87        | 0.09        | 3.57                |
| GDINO    | Super | FP8       | 30      | 8.39        | 9.56        | 11.65       | 1.00        | 0.09        | 1.08                |
| GDINO    | Super | FP8       | 40      | 9.67        | 13.65       | 14.91       | 0.96        | 0.09        | 0.32                |
| GDINO    | Super | FP8       | 50      | 11.25       | 14.39       | 14.98       | 0.96        | 0.09        | 0.35                |
| GDINO    | Super | FP8       | 60      | 12.57       | 14.62       | 25.75       | 0.97        | 0.09        | 0.02                |
| GDINO    | Edge  | BF16      | 1       | 7.33        | 33.22       | 57.32       | 1.87        | 0.09        | 4.02                |
| GDINO    | Edge  | BF16      | 30      | 8.39        | 9.55        | 13.47       | 0.92        | 0.09        | 1.00                |
| GDINO    | Edge  | BF16      | 40      | 9.90        | 13.77       | 14.99       | 0.91        | 0.09        | 0.46                |
| GDINO    | Edge  | BF16      | 50      | 11.35       | 14.50       | 14.95       | 0.95        | 0.09        | 0.00                |

**OSL 100 (captioning)**

| CV Model | VLM   | Precision | Streams | E2E Avg (s) | E2E P90 (s) | E2E P99 (s) | VLM P90 (s) | VST P90 (s) | Verified/min/stream |
| -------- | ----- | --------- | ------- | ----------- | ----------- | ----------- | ----------- | ----------- | ------------------- |
| RT-DETR  | Nano  | BF16      | 1       | 2.69        | 4.61        | 4.96        | 1.84        | 0.09        | 7.55                |
| RT-DETR  | Nano  | FP8       | 1       | 2.58        | 4.57        | 4.96        | 1.83        | 0.09        | 7.52                |
| RT-DETR  | Nano  | FP8       | 20      | 6.90        | 12.97       | 24.15       | 7.16        | 0.09        | 7.45                |
| RT-DETR  | Super | FP8       | 1       | 3.54        | 4.74        | 6.69        | 1.90        | 0.09        | 6.62                |
| RT-DETR  | Edge  | BF16      | 1       | 3.12        | 4.66        | 4.97        | 1.86        | 0.09        | 6.65                |
| GDINO    | Nano  | BF16      | 1       | 9.37        | 26.71       | 56.34       | 1.88        | 0.09        | 5.28                |
| GDINO    | Nano  | BF16      | 30      | 13.37       | 24.29       | 29.63       | 1.85        | 0.09        | 1.19                |
| GDINO    | Nano  | BF16      | 40      | 12.56       | 14.94       | 37.60       | 1.75        | 0.09        | 0.52                |
| GDINO    | Nano  | BF16      | 50      | 13.04       | 17.37       | 28.74       | 1.66        | 0.09        | 0.09                |
| GDINO    | Nano  | BF16      | 60      | 14.90       | 27.50       | 29.75       | 0.99        | 0.09        | 0.01                |
| GDINO    | Nano  | FP8       | 1       | 9.35        | 27.17       | 56.46       | 1.87        | 0.09        | 5.32                |
| GDINO    | Nano  | FP8       | 30      | 12.39       | 20.69       | 29.41       | 1.78        | 0.09        | 1.14                |
| GDINO    | Nano  | FP8       | 40      | 12.11       | 14.70       | 28.80       | 1.56        | 0.09        | 0.52                |
| GDINO    | Nano  | FP8       | 50      | 12.65       | 14.71       | 28.06       | 1.00        | 0.09        | 0.34                |
| GDINO    | Nano  | FP8       | 60      | 14.65       | 27.00       | 29.70       | 0.94        | 0.09        | 0.00                |
| GDINO    | Super | FP8       | 1       | 10.33       | 27.17       | 56.46       | 1.89        | 0.09        | 5.32                |
| GDINO    | Super | FP8       | 30      | 16.28       | 27.82       | 50.81       | 1.90        | 0.09        | 1.07                |
| GDINO    | Super | FP8       | 50      | 14.42       | 24.67       | 29.88       | 1.85        | 0.09        | 0.38                |
| GDINO    | Edge  | BF16      | 1       | 9.81        | 27.17       | 56.46       | 1.88        | 0.09        | 5.32                |
| GDINO    | Edge  | BF16      | 30      | 12.03       | 16.08       | 29.19       | 1.63        | 0.09        | 1.13                |
| GDINO    | Edge  | BF16      | 40      | 12.57       | 14.78       | 29.80       | 1.58        | 0.09        | 0.13                |
| GDINO    | Edge  | BF16      | 50      | 13.51       | 21.43       | 29.14       | 1.76        | 0.09        | 0.01                |

#### L40S

**OSL 1 (alert verification)**

| CV Model | VLM  | Precision | Streams | E2E Avg (s) | E2E P90 (s) | E2E P99 (s) | VLM P90 (s) | VST P90 (s) | Verified/min/stream |
| -------- | ---- | --------- | ------- | ----------- | ----------- | ----------- | ----------- | ----------- | ------------------- |
| RT-DETR  | Nano | BF16      | 1       | 0.98        | 2.20        | 2.47        | 1.76        | 0.09        | 5.38                |
| RT-DETR  | Nano | BF16      | 20      | 2.49        | 4.45        | 4.95        | 1.83        | 0.09        | 5.33                |
| RT-DETR  | Nano | BF16      | 40      | 4.85        | 8.63        | 9.86        | 1.94        | 0.09        | 7.37                |
| RT-DETR  | Nano | FP8       | 1       | 0.93        | 2.13        | 2.46        | 1.72        | 0.09        | 5.45                |
| RT-DETR  | Nano | FP8       | 20      | 2.33        | 4.29        | 4.93        | 1.63        | 0.09        | 5.34                |
| RT-DETR  | Nano | FP8       | 40      | 4.54        | 6.76        | 9.68        | 1.68        | 0.09        | 7.36                |
| RT-DETR  | Edge | BF16      | 1       | 1.07        | 2.24        | 2.48        | 1.81        | 0.09        | 7.13                |
| RT-DETR  | Edge | BF16      | 20      | 2.66        | 4.56        | 4.96        | 1.95        | 0.09        | 5.82                |
| RT-DETR  | Edge | BF16      | 40      | 4.39        | 4.89        | 9.13        | 1.50        | 0.09        | 7.74                |
| GDINO    | Nano | BF16      | 1       | 7.78        | 35.42       | 57.54       | 1.84        | 0.09        | 3.55                |
| GDINO    | Nano | BF16      | 20      | 10.03       | 14.19       | 14.93       | 0.89        | 0.09        | 0.46                |
| GDINO    | Nano | BF16      | 30      | 13.18       | 20.14       | 29.50       | 0.82        | 0.09        | 0.11                |
| GDINO    | Nano | BF16      | 40      | 17.29       | 28.05       | 29.80       | 0.57        | 0.09        | 0.01                |
| GDINO    | Nano | FP8       | 1       | 7.92        | 36.11       | 57.61       | 1.80        | 0.09        | 3.58                |
| GDINO    | Edge | BF16      | 1       | 7.30        | 33.33       | 57.33       | 1.86        | 0.09        | 4.00                |
| GDINO    | Edge | BF16      | 20      | 9.99        | 14.13       | 14.94       | 0.76        | 0.09        | 0.45                |
| GDINO    | Edge | BF16      | 30      | 12.79       | 20.14       | 29.98       | 0.61        | 0.09        | 0.11                |
| GDINO    | Edge | BF16      | 40      | 16.23       | 27.82       | 85.20       | 0.68        | 0.09        | 0.02                |

**OSL 100 (captioning)**

| CV Model | VLM  | Precision | Streams | E2E Avg (s) | E2E P90 (s) | E2E P99 (s) | VLM P90 (s) | VST P90 (s) | Verified/min/stream |
| -------- | ---- | --------- | ------- | ----------- | ----------- | ----------- | ----------- | ----------- | ------------------- |
| RT-DETR  | Nano | BF16      | 1       | 4.80        | 8.92        | 9.89        | 2.85        | 0.09        | 7.52                |
| RT-DETR  | Nano | FP8       | 1       | 4.24        | 6.30        | 9.63        | 2.62        | 0.09        | 7.52                |
| RT-DETR  | Edge | BF16      | 1       | 2.99        | 4.65        | 4.97        | 1.85        | 0.09        | 7.52                |
| GDINO    | Nano | BF16      | 1       | 11.33       | 26.71       | 112.68      | 2.86        | 0.09        | 5.28                |
| GDINO    | Nano | BF16      | 20      | 14.80       | 25.58       | 29.80       | 2.79        | 0.09        | 0.44                |
| GDINO    | Nano | BF16      | 30      | 16.57       | 28.02       | 29.80       | 2.73        | 0.09        | 0.13                |
| GDINO    | Nano | BF16      | 40      | 20.72       | 28.25       | 29.82       | 2.65        | 0.09        | 0.01                |
| GDINO    | Nano | FP8       | 1       | 10.92       | 27.23       | 109.96      | 2.58        | 0.09        | 5.30                |
| GDINO    | Nano | FP8       | 20      | 13.16       | 14.96       | 28.38       | 1.96        | 0.09        | 0.35                |
| GDINO    | Nano | FP8       | 30      | 15.19       | 27.63       | 29.76       | 1.97        | 0.09        | 0.12                |
| GDINO    | Nano | FP8       | 40      | 19.06       | 28.38       | 29.84       | 2.09        | 0.09        | 0.03                |
| GDINO    | Edge | BF16      | 1       | 9.68        | 26.71       | 56.34       | 1.88        | 0.09        | 5.28                |
| GDINO    | Edge | BF16      | 20      | 12.12       | 14.50       | 40.80       | 1.20        | 0.09        | 0.41                |
| GDINO    | Edge | BF16      | 30      | 14.89       | 27.06       | 29.84       | 1.08        | 0.09        | 0.11                |

#### RTX PRO 4500

**OSL 1 (alert verification)**

| CV Model | VLM  | Precision | Streams | E2E Avg (s) | E2E P90 (s) | E2E P99 (s) | VLM P90 (s) | VST P90 (s) | Verified/min/stream |
| -------- | ---- | --------- | ------- | ----------- | ----------- | ----------- | ----------- | ----------- | ------------------- |
| RT-DETR  | Nano | BF16      | 1       | 1.19        | 2.26        | 2.48        | 1.83        | 0.09        | 4.48                |
| RT-DETR  | Nano | FP8       | 1       | 0.99        | 2.21        | 2.47        | 1.74        | 0.09        | 5.35                |
| RT-DETR  | Nano | FP8       | 20      | 2.71        | 4.59        | 4.97        | 1.86        | 0.09        | 5.67                |
| RT-DETR  | Nano | FP8       | 40      | 5.03        | 9.10        | 9.91        | 1.76        | 0.09        | 6.35                |
| RT-DETR  | Nano | NVFP4     | 1       | 0.91        | 2.11        | 2.46        | 1.58        | 0.09        | 5.38                |
| RT-DETR  | Nano | NVFP4     | 20      | 2.51        | 4.48        | 4.95        | 1.60        | 0.09        | 5.65                |
| RT-DETR  | Nano | NVFP4     | 40      | 4.89        | 8.75        | 9.88        | 1.64        | 0.09        | 6.50                |
| RT-DETR  | Edge | BF16      | 1       | 1.20        | 2.25        | 2.48        | 1.83        | 0.09        | 6.72                |
| RT-DETR  | Edge | BF16      | 20      | 5.84        | 11.70       | 21.44       | 9.63        | 0.09        | 6.80                |
| RT-DETR  | Edge | BF16      | 40      | 5.16        | 9.25        | 9.92        | 1.85        | 0.09        | 6.49                |
| GDINO    | Nano | BF16      | 1       | 8.24        | 36.22       | 57.62       | 1.87        | 0.09        | 3.57                |
| GDINO    | Nano | BF16      | 30      | 16.22       | 27.61       | 29.76       | 0.97        | 0.09        | 0.02                |
| GDINO    | Nano | FP8       | 1       | 8.03        | 36.11       | 57.61       | 1.86        | 0.09        | 3.58                |
| GDINO    | Nano | FP8       | 30      | 16.03       | 27.89       | 29.79       | 0.95        | 0.09        | 0.02                |
| GDINO    | Nano | NVFP4     | 1       | 7.92        | 36.11       | 57.61       | 1.80        | 0.09        | 3.58                |
| GDINO    | Edge | BF16      | 1       | 7.43        | 33.33       | 57.33       | 1.87        | 0.09        | 4.00                |
| GDINO    | Edge | BF16      | 30      | 17.57       | 28.00       | 29.80       | 1.30        | 0.09        | 0.02                |

**OSL 100 (captioning)**

| CV Model | VLM  | Precision | Streams | E2E Avg (s) | E2E P90 (s) | E2E P99 (s) | VLM P90 (s) | VST P90 (s) | Verified/min/stream |
| -------- | ---- | --------- | ------- | ----------- | ----------- | ----------- | ----------- | ----------- | ------------------- |
| RT-DETR  | Nano | BF16      | 1       | 4.31        | 7.93        | 9.79        | 2.71        | 0.09        | 7.52                |
| RT-DETR  | Nano | FP8       | 1       | 4.09        | 6.11        | 9.61        | 2.44        | 0.09        | 7.52                |
| RT-DETR  | Nano | NVFP4     | 1       | 3.37        | 4.85        | 9.10        | 1.97        | 0.09        | 7.52                |
| RT-DETR  | Edge | BF16      | 1       | 3.46        | 4.72        | 4.97        | 1.89        | 0.09        | 7.52                |
| GDINO    | Nano | BF16      | 1       | 11.64       | 53.16       | 112.68      | 2.88        | 0.09        | 5.28                |
| GDINO    | Nano | BF16      | 30      | 20.77       | 28.46       | 29.85       | 2.77        | 0.09        | 0.02                |
| GDINO    | Nano | BF16      | 40      | 26.50       | 28.50       | 29.85       | 2.20        | 0.09        | 0.00                |
| GDINO    | Nano | FP8       | 1       | 10.07       | 27.23       | 56.47       | 1.89        | 0.09        | 5.30                |
| GDINO    | Nano | FP8       | 30      | 18.47       | 28.44       | 29.84       | 1.92        | 0.09        | 0.03                |
| GDINO    | Nano | FP8       | 40      | 20.92       | 28.50       | 29.85       | 1.90        | 0.09        | 0.00                |
| GDINO    | Nano | NVFP4     | 1       | 9.71        | 27.17       | 56.46       | 1.88        | 0.09        | 5.32                |
| GDINO    | Nano | NVFP4     | 30      | 18.23       | 28.21       | 29.82       | 1.73        | 0.09        | 0.03                |
| GDINO    | Edge | BF16      | 1       | 10.06       | 26.71       | 57.40       | 1.89        | 0.09        | 5.28                |
| GDINO    | Edge | BF16      | 30      | 19.24       | 28.62       | 46.80       | 1.69        | 0.09        | 0.02                |
| GDINO    | Edge | BF16      | 40      | 26.03       | 28.50       | 29.85       | 1.80        | 0.09        | 0.00                |

#### RTX PRO 6000

**OSL 1 (alert verification)**

| CV Model | VLM   | Precision | Streams | E2E Avg (s) | E2E P90 (s) | E2E P99 (s) | VLM P90 (s) | VST P90 (s) | Verified/min/stream |
| -------- | ----- | --------- | ------- | ----------- | ----------- | ----------- | ----------- | ----------- | ------------------- |
| RT-DETR  | Nano  | BF16      | 1       | 0.58        | 0.99        | 2.33        | 0.86        | 0.09        | 5.38                |
| RT-DETR  | Nano  | BF16      | 20      | 0.76        | 1.83        | 2.43        | 1.30        | 0.09        | 5.31                |
| RT-DETR  | Nano  | BF16      | 40      | 2.32        | 4.25        | 4.92        | 1.51        | 0.09        | 5.18                |
| RT-DETR  | Nano  | FP8       | 1       | 0.70        | 0.98        | 2.32        | 0.90        | 0.09        | 5.38                |
| RT-DETR  | Nano  | FP8       | 20      | 0.67        | 1.56        | 2.41        | 0.85        | 0.09        | 5.30                |
| RT-DETR  | Nano  | FP8       | 40      | 2.18        | 3.94        | 4.89        | 0.99        | 0.09        | 5.16                |
| RT-DETR  | Nano  | NVFP4     | 1       | 0.71        | 0.99        | 2.33        | 0.90        | 0.09        | 5.35                |
| RT-DETR  | Nano  | NVFP4     | 20      | 0.67        | 1.26        | 2.38        | 0.83        | 0.09        | 5.30                |
| RT-DETR  | Nano  | NVFP4     | 40      | 2.16        | 3.86        | 4.89        | 0.96        | 0.09        | 5.28                |
| RT-DETR  | Super | FP8       | 1       | 0.78        | 0.99        | 2.34        | 0.91        | 0.09        | 5.35                |
| RT-DETR  | Super | FP8       | 20      | 1.55        | 2.37        | 4.36        | 2.45        | 0.09        | 5.31                |
| RT-DETR  | Super | NVFP4     | 1       | 0.67        | 0.98        | 2.32        | 0.90        | 0.09        | 5.35                |
| RT-DETR  | Super | NVFP4     | 20      | 0.98        | 2.12        | 2.46        | 1.70        | 0.09        | 5.31                |
| RT-DETR  | Super | NVFP4     | 40      | 3.11        | 4.84        | 9.14        | 2.95        | 0.09        | 5.51                |
| RT-DETR  | Edge  | BF16      | 1       | 0.88        | 2.10        | 2.46        | 0.93        | 0.09        | 7.47                |
| RT-DETR  | Edge  | BF16      | 20      | 1.00        | 2.18        | 2.47        | 1.70        | 0.09        | 5.39                |
| RT-DETR  | Edge  | BF16      | 40      | 2.63        | 4.54        | 4.96        | 1.91        | 0.09        | 5.94                |
| GDINO    | Nano  | BF16      | 1       | 7.52        | 36.11       | 57.61       | 0.93        | 0.09        | 3.58                |
| GDINO    | Nano  | BF16      | 50      | 12.54       | 14.96       | 31.35       | 0.46        | 0.09        | 0.13                |
| GDINO    | Nano  | BF16      | 60      | 15.91       | 27.20       | 89.40       | 0.49        | 0.09        | 0.01                |
| GDINO    | Nano  | FP8       | 1       | 7.68        | 36.22       | 57.62       | 0.94        | 0.09        | 3.57                |
| GDINO    | Nano  | FP8       | 60      | 15.27       | 27.21       | 90.20       | 0.48        | 0.09        | 0.04                |
| GDINO    | Nano  | NVFP4     | 1       | 7.65        | 36.11       | 57.61       | 0.94        | 0.09        | 3.58                |
| GDINO    | Nano  | NVFP4     | 50      | 12.87       | 14.69       | 47.10       | 0.46        | 0.09        | 0.16                |
| GDINO    | Nano  | NVFP4     | 60      | 14.15       | 26.25       | 29.82       | 0.46        | 0.09        | 0.05                |
| GDINO    | Super | FP8       | 1       | 7.98        | 36.22       | 57.62       | 1.83        | 0.09        | 3.57                |
| GDINO    | Super | NVFP4     | 1       | 7.81        | 36.11       | 57.61       | 0.95        | 0.09        | 3.58                |
| GDINO    | Edge  | BF16      | 1       | 7.01        | 33.22       | 57.32       | 0.94        | 0.09        | 4.02                |
| GDINO    | Edge  | BF16      | 50      | 12.60       | 14.82       | 29.32       | 0.61        | 0.09        | 0.14                |

**OSL 100 (captioning)**

| CV Model | VLM   | Precision | Streams | E2E Avg (s) | E2E P90 (s) | E2E P99 (s) | VLM P90 (s) | VST P90 (s) | Verified/min/stream |
| -------- | ----- | --------- | ------- | ----------- | ----------- | ----------- | ----------- | ----------- | ------------------- |
| RT-DETR  | Nano  | BF16      | 1       | 2.57        | 4.57        | 4.96        | 1.81        | 0.09        | 7.52                |
| RT-DETR  | Nano  | FP8       | 1       | 2.65        | 4.58        | 4.97        | 1.80        | 0.09        | 7.52                |
| RT-DETR  | Nano  | FP8       | 20      | 6.17        | 11.65       | 19.33       | 6.92        | 0.09        | 7.47                |
| RT-DETR  | Nano  | NVFP4     | 1       | 2.22        | 3.97        | 4.90        | 1.18        | 0.09        | 7.52                |
| RT-DETR  | Nano  | NVFP4     | 20      | 3.86        | 7.51        | 9.77        | 2.95        | 0.09        | 7.47                |
| RT-DETR  | Super | FP8       | 1       | 3.97        | 6.30        | 9.63        | 2.36        | 0.09        | 7.52                |
| RT-DETR  | Super | NVFP4     | 1       | 3.44        | 4.77        | 7.18        | 1.92        | 0.09        | 7.52                |
| RT-DETR  | Edge  | BF16      | 1       | 2.44        | 4.53        | 4.95        | 1.79        | 0.09        | 7.52                |
| RT-DETR  | Edge  | BF16      | 20      | 8.45        | 13.92       | 25.87       | 8.50        | 0.09        | 7.46                |
| GDINO    | Nano  | BF16      | 1       | 9.36        | 27.17       | 56.46       | 1.84        | 0.09        | 5.32                |
| GDINO    | Nano  | BF16      | 60      | 15.63       | 27.36       | 29.97       | 0.96        | 0.09        | 0.03                |
| GDINO    | Nano  | FP8       | 1       | 9.07        | 27.17       | 56.46       | 1.67        | 0.09        | 5.32                |
| GDINO    | Nano  | FP8       | 50      | 13.98       | 26.06       | 29.93       | 0.96        | 0.09        | 0.16                |
| GDINO    | Nano  | NVFP4     | 1       | 8.96        | 27.23       | 56.47       | 1.00        | 0.09        | 5.30                |
| GDINO    | Super | FP8       | 1       | 11.00       | 28.84       | 110.49      | 2.74        | 0.09        | 5.28                |
| GDINO    | Super | NVFP4     | 1       | 9.81        | 27.17       | 56.46       | 1.89        | 0.09        | 5.32                |
| GDINO    | Super | NVFP4     | 50      | 15.14       | 27.55       | 37.20       | 1.84        | 0.09        | 0.15                |
| GDINO    | Super | NVFP4     | 60      | 16.35       | 27.99       | 29.97       | 1.79        | 0.09        | 0.07                |
| GDINO    | Edge  | BF16      | 1       | 9.29        | 27.23       | 56.47       | 1.83        | 0.09        | 5.30                |

#### DGX Spark

**OSL 1 (alert verification)**

| CV Model | VLM   | Precision | Streams | E2E Avg (s) | E2E P90 (s) | E2E P99 (s) | VLM P90 (s) | VST P90 (s) | Verified/min/stream |
| -------- | ----- | --------- | ------- | ----------- | ----------- | ----------- | ----------- | ----------- | ------------------- |
| RT-DETR  | Nano  | BF16      | 1       | 1.54        | 2.46        | 4.70        | 2.67        | 0.09        | 5.38                |
| RT-DETR  | Nano  | FP8       | 1       | 1.26        | 2.22        | 2.48        | 1.78        | 0.09        | 5.35                |
| RT-DETR  | Nano  | NVFP4     | 1       | 1.14        | 2.22        | 2.47        | 1.78        | 0.09        | 5.38                |
| RT-DETR  | Super | BF16      | 1       | 3.34        | 4.75        | 4.97        | 1.90        | 0.09        | 7.68                |
| RT-DETR  | Super | FP8       | 1       | 3.80        | 7.80        | 9.78        | 7.35        | 0.09        | 6.02                |
| RT-DETR  | Super | NVFP4     | 1       | 2.28        | 4.49        | 4.95        | 3.79        | 0.09        | 5.45                |
| RT-DETR  | Edge  | BF16      | 1       | 1.66        | 2.53        | 4.75        | 2.29        | 0.09        | 7.73                |
| GDINO    | Nano  | BF16      | 1       | 9.56        | 36.89       | 57.69       | 2.86        | 0.09        | 3.47                |
| GDINO    | Nano  | FP8       | 1       | 9.18        | 36.78       | 57.68       | 2.84        | 0.09        | 3.48                |
| GDINO    | Nano  | NVFP4     | 1       | 8.95        | 36.89       | 57.69       | 2.74        | 0.09        | 3.47                |
| GDINO    | Super | BF16      | 1       | 9.97        | 26.81       | 57.45       | 1.90        | 0.09        | 5.35                |
| GDINO    | Super | FP8       | 1       | 10.64       | 23.04       | 49.66       | 23.47       | 0.09        | 6.32                |
| GDINO    | Super | NVFP4     | 1       | 9.72        | 34.32       | 58.51       | 4.69        | 0.09        | 3.73                |
| GDINO    | Edge  | BF16      | 1       | 8.72        | 34.56       | 57.46       | 2.87        | 0.09        | 3.82                |

**OSL 100 (captioning)**

| CV Model | VLM   | Precision | Streams | E2E Avg (s) | E2E P90 (s) | E2E P99 (s) | VLM P90 (s) | VST P90 (s) | Verified/min/stream |
| -------- | ----- | --------- | ------- | ----------- | ----------- | ----------- | ----------- | ----------- | ------------------- |
| RT-DETR  | Nano  | BF16      | 1       | 1.67        | 2.49        | 4.74        | 2.73        | 0.09        | 5.38                |
| RT-DETR  | Nano  | FP8       | 1       | 1.36        | 2.28        | 3.38        | 1.79        | 0.09        | 5.40                |
| RT-DETR  | Nano  | NVFP4     | 1       | 1.18        | 2.21        | 2.47        | 1.77        | 0.10        | 5.38                |
| RT-DETR  | Super | BF16      | 1       | 3.35        | 4.75        | 4.98        | 1.90        | 0.10        | 7.58                |
| RT-DETR  | Super | FP8       | 1       | 4.45        | 9.05        | 9.90        | 8.79        | 0.09        | 5.97                |
| RT-DETR  | Super | NVFP4     | 1       | 2.49        | 4.53        | 5.71        | 4.32        | 0.09        | 5.72                |
| RT-DETR  | Edge  | BF16      | 1       | 1.80        | 3.28        | 4.83        | 2.65        | 0.09        | 7.33                |
| GDINO    | Nano  | BF16      | 1       | 9.86        | 37.22       | 57.72       | 2.91        | 0.09        | 3.42                |
| GDINO    | Nano  | FP8       | 1       | 9.38        | 37.11       | 57.71       | 2.84        | 0.09        | 3.43                |
| GDINO    | Nano  | NVFP4     | 1       | 9.07        | 37.11       | 57.71       | 2.75        | 0.09        | 3.43                |
| GDINO    | Super | BF16      | 1       | 9.84        | 26.31       | 57.37       | 1.90        | 0.09        | 5.32                |
| GDINO    | Super | FP8       | 1       | 11.71       | 25.25       | 47.49       | 24.36       | 0.09        | 6.95                |
| GDINO    | Super | NVFP4     | 1       | 9.60        | 32.88       | 57.29       | 4.67        | 0.09        | 3.77                |
| GDINO    | Edge  | BF16      | 1       | 8.49        | 33.46       | 57.35       | 2.86        | 0.09        | 3.83                |

#### AGX Thor

**OSL 1 (alert verification)**

| CV Model | VLM   | Precision | Streams | E2E Avg (s) | E2E P90 (s) | E2E P99 (s) | VLM P90 (s) | VST P90 (s) | Verified/min/stream |
| -------- | ----- | --------- | ------- | ----------- | ----------- | ----------- | ----------- | ----------- | ------------------- |
| RT-DETR  | Nano  | BF16      | 1       | 1.96        | 3.06        | 4.81        | 2.75        | 0.09        | 2.20                |
| RT-DETR  | Nano  | BF16      | 10      | 9.11        | 13.92       | 23.75       | 2.93        | 0.09        | 0.42                |
| RT-DETR  | Nano  | FP8       | 1       | 1.75        | 2.47        | 4.70        | 2.40        | 0.09        | 0.20                |
| RT-DETR  | Nano  | FP8       | 10      | 7.21        | 9.91        | 14.55       | 2.50        | 0.09        | 4.67                |
| RT-DETR  | Nano  | NVFP4     | 1       | 1.42        | 2.46        | 4.71        | 1.96        | 0.09        | 5.15                |
| RT-DETR  | Nano  | NVFP4     | 10      | 6.84        | 9.72        | 14.13       | 1.98        | 0.09        | 6.29                |
| RT-DETR  | Super | NVFP4     | 1       | 2.60        | 4.56        | 5.86        | 4.49        | 0.09        | 5.52                |
| RT-DETR  | Super | NVFP4     | 10      | 11.14       | 20.70       | 29.21       | 4.34        | 0.09        | 1.97                |
| RT-DETR  | Edge  | BF16      | 1       | 2.69        | 5.71        | 9.57        | 4.55        | 0.09        | 2.72                |
| RT-DETR  | Edge  | BF16      | 10      | 8.11        | 12.77       | 22.31       | 2.84        | 0.09        | 4.27                |
| GDINO    | Nano  | BF16      | 1       | 6.89        | 5.50        | 61.60       | 2.73        | 0.09        | 4.87                |
| GDINO    | Nano  | FP8       | 1       | 7.30        | 4.99        | 55.90       | 2.59        | 0.09        | 4.78                |
| GDINO    | Nano  | NVFP4     | 1       | 7.97        | 19.07       | 58.84       | 2.57        | 0.09        | 4.68                |
| GDINO    | Super | NVFP4     | 1       | 7.47        | 9.07        | 93.45       | 4.08        | 0.09        | 5.90                |
| GDINO    | Edge  | BF16      | 1       | 7.01        | 8.75        | 71.40       | 3.00        | 0.09        | 6.75                |

**OSL 100 (captioning)**

| CV Model | VLM   | Precision | Streams | E2E Avg (s) | E2E P90 (s) | E2E P99 (s) | VLM P90 (s) | VST P90 (s) | Verified/min/stream |
| -------- | ----- | --------- | ------- | ----------- | ----------- | ----------- | ----------- | ----------- | ------------------- |
| RT-DETR  | Nano  | BF16      | 1       | 2.15        | 4.36        | 4.94        | 2.77        | 0.09        | 5.15                |
| RT-DETR  | Nano  | BF16      | 10      | 9.62        | 14.47       | 27.94       | 3.58        | 0.09        | 1.38                |
| RT-DETR  | Nano  | FP8       | 1       | 1.76        | 2.65        | 4.76        | 2.78        | 0.09        | 5.02                |
| RT-DETR  | Nano  | FP8       | 10      | 8.31        | 13.10       | 24.26       | 2.82        | 0.09        | 1.34                |
| RT-DETR  | Nano  | NVFP4     | 1       | 1.50        | 2.47        | 4.72        | 1.97        | 0.09        | 5.12                |
| RT-DETR  | Nano  | NVFP4     | 10      | 7.55        | 10.57       | 14.63       | 2.50        | 0.09        | 2.64                |
| RT-DETR  | Super | NVFP4     | 1       | 3.14        | 4.80        | 9.07        | 4.67        | 0.09        | 5.28                |
| RT-DETR  | Super | NVFP4     | 10      | 13.73       | 27.43       | 53.66       | 4.72        | 0.09        | 0.49                |
| RT-DETR  | Edge  | BF16      | 1       | 2.64        | 4.97        | 9.47        | 4.69        | 0.09        | 6.30                |
| RT-DETR  | Edge  | BF16      | 10      | 9.59        | 14.68       | 28.10       | 2.92        | 0.09        | 4.83                |
| GDINO    | Nano  | BF16      | 1       | 6.64        | 5.57        | 59.52       | 2.75        | 0.09        | 5.40                |
| GDINO    | Nano  | FP8       | 1       | 7.48        | 7.42        | 57.13       | 2.71        | 0.09        | 5.02                |
| GDINO    | Nano  | NVFP4     | 1       | 7.82        | 4.99        | 57.44       | 2.61        | 0.09        | 4.93                |
| GDINO    | Super | NVFP4     | 1       | 6.24        | 8.33        | 55.68       | 3.85        | 0.09        | 5.73                |
| GDINO    | Edge  | BF16      | 1       | 7.18        | 9.29        | 85.37       | 3.02        | 0.09        | 6.73                |

> **Note**
>
> Configurations whose E2E average latency exceeds 30 s are not shown, because the
> stream count is beyond what the platform can sustain for that configuration.

> **Note**
>
> All benchmarks use input streams at 1080p, H.264 input.
>
> For dGPU platform tests (NVIDIA B200, DGX H100 SXM, L40S, RTX PRO 4500, and
> RTX PRO 6000), two GPUs were used for the Nano and Edge models: one GPU dedicated
> to RTVI-CV and one GPU dedicated to RTVI-VLM. Super-model runs used three GPUs. On L40S and RTX PRO 4500, the LLM used by
> the pipeline is served remotely rather than on the local GPUs.
>
> Results are shown for OSL 1 (alert verification) and OSL 100 (captioning). GDINO end-to-end latency is
> dominated by the upstream CV and analytics stage, so GDINO E2E P90 and P99 are
> much higher than RT-DETR at the same stream count. Verified/min/stream is the
> number of verified alerts per minute for each active stream.

> **Note**
>
> As stream counts increase, the average CV frame rate per stream may decrease, which
> causes a drop in Verified/min/stream.