Agent Profiles

The VSS Agent connects to a matching backend blueprint. Each profile configures the agent for a specific use case with tailored prompts, tools, and report templates.

Blueprint Profiles

Three production blueprint profiles are available. The agent configuration connects to the corresponding backend analytics pipeline.

Warehouse Blueprint

Industrial facility monitoring for safety incidents.

• Config: deployments/warehouse/vss-agent/configs/config.yml

• Warehouse Blueprint Documentation

Smart Cities Blueprint

Traffic and urban monitoring at intersections.

• Config: deployments/smartcities/vss-agent/configs/config.yml

• Smart Cities Blueprint Documentation

Developer Profiles

For development and testing without a full blueprint deployment, three developer profiles are available. These use OOBE (Out-of-Box Experience) mode for standalone operation.

dev-profile-base

Basic video upload and analysis without VA-MCP or incident database.

• Config: deployments/developer-workflow/dev-profile-base/vss-agent/configs/config.yml

• Use Case: Upload videos, generate reports, answer questions about video content

Capabilities:

• Upload videos via /api/v1/videos endpoint

• Generate video analysis reports with timestamped observations

• Answer follow-up questions about video content or report content

• Retrieve snapshots and video clips

Tools Available:

• video_understanding: Analyze video content using Cosmos VLM

• vst_video_clip: Get video playback URLs

• vst_snapshot: Get snapshot images at specific timestamps

• vst_stream_list: List available video streams

dev-profile-lvs

Long video analysis with interactive Human-in-the-Loop (HITL) prompts.

• Config: deployments/developer-workflow/dev-profile-lvs/vss-agent/configs/config.yml

• Use Case: Analyze long videos with configurable scenario, events, and objects

Capabilities:

All dev-profile-base capabilities, plus:

• Long Video Summarization (LVS) for videos longer than 1 minute

• Interactive HITL prompts for scenario, events, and objects of interest

Additional Tools:

• lvs_video_understanding: Analyze long videos with configurable parameters

• vst_timeline: Get video timeline information for duration-based tool selection

HITL Prompts:

When using LVS, the agent prompts for:

1. Scenario: What type of monitoring (e.g., “warehouse monitoring”, “traffic monitoring”)

2. Events: Comma-separated list of events to detect (e.g., “accident, forklift stuck, person entering restricted area”)

3. Objects (optional): Objects to focus on (e.g., “forklifts, pallets, workers”)

dev-profile-search

Video embedding search with semantic queries.

• Config: deployments/developer-workflow/dev-profile-search/vss-agent/configs/config.yml

• Use Case: Semantic search across video content using Cosmos Embed

Capabilities:

All dev-profile-base capabilities, plus:

• Semantic video search using Cosmos Embed embeddings

• Real-time video ingest with embedding generation

• Search API endpoint at /api/search

Additional Tools:

• search: Semantic search across indexed video content

• embed_search: Low-level embedding search against Elasticsearch

Requirements:

• Cosmos Embed NIM endpoint for embedding generation

• Elasticsearch for storing and querying embeddings

• RTVI service for real-time video ingest

Developer Profile Comparison

Feature	dev-profile-base	dev-profile-lvs	dev-profile-search
Video Upload	✓	✓	✓
Video Analysis (VLM)	✓	✓	✓
Report Generation	✓	✓	✓
Long Video Support	✗	✓ (LVS)	✗
Semantic Search	✗	✗	✓
Elasticsearch Required	✗	✗	✓

Services

When deployed, the following services are available:

Service	Default URL	Description
VSS Agent	http://localhost:8000	Agent API
VA-MCP	http://localhost:9901	Video Analytics MCP

For deployment instructions, see the respective blueprint documentation linked above.