Agent Skills#

Agent Skills are reusable, self-contained capabilities that coding agents (such as Claude Code and Codex) can load to deploy and operate the VSS Blueprint. Each Skill follows the agentskills.io specification and packages prompts, reference data, and helper scripts so the coding agent can deploy a VSS profile, manage cameras and streams, query incidents, run searches, summarize videos, and generate reports. All from natural-language requests.

Note

Agent Skills are a developer-side tool for working with the VSS Blueprint. They are distinct from the in-product VSS Agent, which runs inside a deployed VSS workflow and orchestrates VSS microservices.

Skill Anatomy#

Each VSS Skill is a folder under the skills/ directory of the VSS Blueprint repository. A VSS Skill folder contains the following:

  • A SKILL.md file with YAML frontmatter declaring the Skill’s name, description, version, and license. The description tells the coding agent when to invoke that VSS Skill. For example, when the user asks to deploy a VSS profile, add a camera, query alerts, or summarize a video.

  • The body of SKILL.md documents the VSS commands, API calls, and patterns the Skill uses. Typically curl calls against the VSS REST API, the VA-MCP server, or VIOS.

  • Optional reference files such as VSS deployment notes, alert-type prompt templates, or example payloads for the workflow the Skill targets.

  • Optional evaluation specs under eval/ that are exercised by the VSS Skills Eval CI workflow on every pull request that touches the skills/ directory of the VSS Blueprint repository.

When a coding agent loads its skills directory, it reads each SKILL.md frontmatter and adds the VSS Skill to its tool list. The agent then matches user requests against the Skills’ descriptions and invokes the appropriate VSS Skill for each task. VSS Skills can be added or updated while the agent is running. When a new SKILL.md appears in the skills directory, the agent picks it up without a restart.

Usage Patterns#

VSS Agent Skills are used in two complementary ways:

  • Deployment. A coding agent such as Claude Code or Codex loads the Skills, takes a natural-language request like “Deploy VSS for video search”, and invokes vss-deploy-profile to select the profile, run pre-flight checks, and bring up the Docker Compose stack. The developer does not need to remember exact flags, profile names, or compose commands.

  • Runtime. Once VSS is running, an always-on coding agent such as NemoClaw loads the same Skills and acts as the operational interface to the deployment. The user can manage cameras and streams, submit and verify alerts, run searches, summarize videos, and generate reports through natural-language chat.

Available Skills#

The official VSS Agent Skills are published in the skills/ folder of the VSS Blueprint repository on GitHub, and mirrored to the public NVIDIA Skills catalog at github.com/nvidia/skills.

Skill

Description

vss-ask-video

Answer text questions about video content by forwarding the request to the VSS agent’s video_understanding tool for frame-level VLM analysis.

vss-deploy-dense-captioning

Deploy and call the RT-VLM dense captioning microservice on stored video files and live RTSP streams. Captions, alerts, stream management, or OpenAI-compatible completions.

vss-deploy-detection-tracking-2d

Deploy, operate, debug, or tear down the RTVI-CV (Real Time Video Intelligence CV) perception microservice locally, and call its REST API to manage streams, health probes, and metrics. Supports the warehouse-2d, warehouse-3d, smartcity-rtdetr, and smartcity-gdino use cases.

vss-deploy-detection-tracking-3d

Deploy and operate the RTVI-CV-3D stack (MV3DT / Multi-View 3D Tracking) — per-camera DeepStream perception plus BEV Fusion over multiple calibrated cameras — on sample, custom-video, or RTSP inputs. Auto-chains to vss-generate-video-calibration when calibration data is missing.

vss-deploy-profile

Configure, deploy, debug, verify, or tear down any VSS profile using a Docker Compose-centric workflow.

vss-deploy-video-embedding

Deploy and operate the RT-Embed video-embedding microservice — /v1 REST API for file, text, and video embeddings and live RTSP, plus Redis, Kafka, and OpenTelemetry integration.

vss-generate-video-calibration

Calibrate a multi-camera dataset with AutoMagicCalib (AMC) from local MP4s, live RTSP streams, or the bundled sample dataset.

vss-generate-video-report

Produce video analysis reports by querying the VSS agent’s /generate endpoint and formatting the reply as a standard markdown report.

vss-generate-video-report-rag

Generate video summary reports using the video_search_frag extension with Video Summarization, Enterprise RAG context, and HITL parameter collection.

vss-manage-alerts

Add, manage, and monitor alerts on streamed video in either CV verification mode or VLM real-time mode.

vss-manage-video-io-storage

Manage video and stream operations, recording timelines, clip extraction, and snapshots using the Video IO and Storage (VIOS) microservices.

vss-query-analytics

Query video analytics data and metrics from Elasticsearch via the VA-MCP server.

vss-search-archive

Search video archives using natural language with fusion search over Cosmos Embed1 embeddings and computer vision attribute matching.

vss-setup-behavior-analytics

Deploy the vss-behavior-analytics service standalone, with custom configs or calibration, without the full warehouse stack.

vss-setup-video-analytics-api

Deploy the vss-video-analytics-api REST service standalone against custom Elasticsearch and Kafka infrastructure.

vss-summarize-video

Summarize a video by routing short clips to the VLM NIM and longer videos to the Video Summarization microservice.

Supported Agent Harnesses#

VSS Agent Skills can be loaded by any agent harness that follows the agentskills.io specification. The following agent harnesses are validated:

For more information on agent harnesses with VSS, see Agent Harnesses. Each agent harness expects Skills under a standard directory:

Host

Skills directory

Claude Code

~/.claude/skills/<name>/

Codex

~/.codex/skills/<name>/

Other agentskills.io-compatible hosts

~/.agents/skills/<name>/

Installing Skills#

Skills are installed by asking the coding agent itself to read the Skills catalog and register each Skill into the host’s skills directory. Symlinks are used so that a git pull of the Blueprint repository keeps every Skill up to date automatically.

Step 1: Clone the VSS Blueprint repository#

git clone https://github.com/NVIDIA-AI-Blueprints/video-search-and-summarization.git
cd video-search-and-summarization

Step 2: Ask the coding agent to install the Skills#

Open the repository in your coding agent and paste the following prompt:

Read skills/README.md and every SKILL.md file under skills/. For each skill in the
catalog, install it for this host so I can invoke it from a shell or chat session.
Use the host's standard skills directory. Symlink each skill folder rather than
copying it so a git pull here keeps every install up to date. Skip skills that are
already installed and pointing at this checkout. When you're done, list the skills
you registered and which directory you used.

The agent reads each SKILL.md frontmatter, creates the symlinks, and confirms what is installed. The Skills become invokable in the next agent session.

To install only a single Skill, ask the agent:

Install only skills/<name>/ for this host the same way.

Using Skills#

With Skills installed, prompt the coding agent in natural language to deploy a profile or operate the deployment. The agent invokes the matching Skill from the catalog. For example, vss-deploy-profile stands up a stack, and vss-manage-video-io-storage or vss-summarize-video manage cameras and summaries. See Usage Patterns for deploy-time versus runtime use.

One request can chain multiple Skills when the task needs several steps or a missing prerequisite.

In your coding agent, use prompts such as:

Deploy VSS for video summarization.

Add an RTSP camera at rtsp://10.0.0.5:554/stream1.

Updating Skills#

After a git pull of the Blueprint repository, the symlinks already point at the updated content. No further action is needed unless new Skills were added or existing ones were renamed. To pick up newly added Skills, ask the agent:

Re-read skills/README.md and add any new skills missing from this host's skills
directory.

To uninstall every Skill, ask the agent:

Remove every VSS skill symlink you previously created under this host's skills
directory.