Skip to navigation
AppendixPerformanceAgent Workflows

Video Summarization Performance

View as Markdown

Overview

The Video Summarization microservice processes uploaded video files through RTVI-VLM for per-chunk vision inference and a summarization LLM (CA-RAG) for event aggregation. The tables below report mean end-to-end (E2E) latency for file summarization and maximum concurrent request capacity (burst) across video lengths. Use these numbers to size GPU count and topology for batch summarization workloads on H100, RTX Pro 6000 SE, and L40S platforms.

Test Configuration

ParameterValue
VSS release3.2
VLMCosmos3 Nano Reasoner
Model precisionFP8
Summarization LLMNemotron 3 Nano (default Video Summarization stack)
Chunk duration10 seconds
Frames per chunk20
Vision input tokens9k
Output max tokens100
Test scenarioWarehouse safety monitoring
Test videos1, 10, 30, 60, 120, and 720 minute MP4
GPU topologies1 (1 GPU), 2x2 (2 VLM + 2 LLM), 4x4 (4 VLM + 4 LLM)
Platforms testedH100, RTX Pro 6000 SE, L40S

E2E Latency

Topology 4×4 (4 VLM + 4 LLM GPUs)

Video length1 min10 min30 min60 min120 min
E2E Avg (s)5.1620.152.964.7119

720-minute E2E Avg: 615 s

Topology 2×2 (2 VLM + 2 LLM GPUs)

Video length1 min10 min30 min60 min120 min
E2E Avg (s)5.7023.359.7128205

720-minute E2E Avg: 1151 s

Topology 1 (1 GPU — VLM and LLM colocated)

Video length1 min10 min30 min60 min120 min
E2E Avg (s)12.150.7117265438

Maximum Concurrency

Maximum concurrent summarize requests at each target mean E2E latency. Test videos are 1, 5, and 10 minute MP4.

Topology 4×4 (4 VLM + 4 LLM GPUs)

Video length1 min5 min10 min
Target avg latency (s)60300600
Max concurrency73114125

Topology 2×2 (2 VLM + 2 LLM GPUs)

Video length1 min5 min10 min
Target avg latency (s)60300600
Max concurrency426572

All measurements use Cosmos3 Nano Reasoner FP8 with Nemotron 3 Nano, 20 frames per 10-second chunk, 9k vision tokens, and max_tokens=100.