Release Notes

These Release Notes describe the key features, software enhancements and improvements, and known issues for the VSS release product package.

VSS 3.3.0

Key Features and Enhancements

• Released vss-build-vision-ai, a unified Agent Skill for composing and deploying custom vision stacks. It deploys the base, search, lvs, and alerts developer profiles plus the warehouse industry profile, and can extend a running deployment with a delta overlay.

  vss-build-vision-ai replaces the removed vss-deploy-profile Agent Skill and takes no -p / -m flags — describe the profile in the request instead:

  vss-deploy-profile	vss-build-vision-ai
  -p base / -p search / -p lvs	the matching stock developer workflow
  -p alerts -m verification / -p alerts -m real-time	the stock Alerts workflow in 2d_cv / 2d_vlm mode
  -p warehouse	the warehouse industry profile (variant selected during intake)

  The /deploy pre-GA alias now resolves to /vss-build-vision-ai.

  vss-build-vision-ai also composes lean stacks from the capabilities you name. Unless the request names the in-stack VSS Agent, base, alerts, lvs, and search builds omit the Agent and Phoenix, and keep the Agent’s LLM only when a selected service or the harness needs it.

• Reorganized the Agent Skills into deployment/, operations/, tools/, and benchmarking/ folders, with vss-build-vision-ai at the top level. Symlinks from a 3.2.x install no longer resolve for moved Skills; see Updating Skills.

• Added the vss CLI as the supported way to operate a running deployment from a coding agent or the host. Configure it once with vss configure --base-url. Command groups include vios, vlm, summarize, search, memory, and analytics. Skills drive the CLI rather than constructing REST endpoints.

• Added VSS CLI agent memory. Elasticsearch stores structured jobs and records. Enable it with vss configure memory, then read records with vss memory get, query, and events. OpenClaw Markdown notes are an optional recall cache.

• Added bounded memory introspection with vss configure memory introspection and vss memory introspect. Introspection answers from stored records first and requests at most three grounded VLM follow-ups. vss-ask-video checks hot conversation context, Markdown memory, structured memory, and introspection before it runs vss vlm run on an exact window.

• Moved VSS documentation releases to GitHub Actions and Fern. Documentation is authored alongside the source in this repository and built and published with Fern.

• Added Adaptive EVS video token pruning to the Real-Time VLM (RTVI-VLM) microservice. Building on fixed-rate Efficient Video Sampling (EVS), Adaptive EVS adds content-dependent similarity pruning, per-stream video sessions, and token-budgeted clip packing to reduce redundant visual input and per-chunk inference time on long videos and live streams. Enable it with VIA_EVS_SESSION=true and VLM_VIDEO_PRUNING_RATE=0.5, and optionally tune how aggressively similar frames are pruned with VLLM_EVS_SIMILARITY_THRESHOLD (default 0.4). It is supported for local, vLLM-compatible Qwen3-VL-based models. Refer to Efficient Video Sampling (EVS and Adaptive EVS) in the Real-Time VLM documentation.

Agent Workflows

Known Issues and Limitations

• The agent could return a video URL with the OpenShell origin http://host.openshell.internal:7777; replace this with your own browser-facing origin to open the URL.
• Alerts developer profile (real-time mode): The default sample prompt may cause false positives on some streams. When using a video other than the default sample stream, update and make the alert prompt as specific as possible about the conditions to detect.
• Alerts developer profile (real-time mode) input support: Only RTSP streams are supported as input. Attempting to upload MP4 files directly to VIOS causes RTVI VLM errors.
• RTSP searches return no results for queries without visual attributes: In the Search profile, in-stack VSS Agent query decomposition applies a 2025-01-01 time window to queries that describe an action or event but no visual attributes such as clothing or color, for example show person climbing ladder. The window excludes the stream’s recordings, so the search returns no results even when matching footage exists. Queries that include visual descriptors and searches over uploaded videos are unaffected, as are the vss CLI and the vss-search-archive skill.
• Attribute-only searches scoped to an uploaded video return no results: In the Search profile, the VSS Agent returns no results when a query describes visual attributes but no action and names an uploaded video as the video source, such as show person with green jacket in video source warehouse_sample. Queries without a video source, fusion searches that combine an action with visual attributes, and searches over RTSP streams are unaffected, as are the vss CLI and the vss-search-archive skill. Workaround: omit the video source from the query.
• NemoClaw harness bring-up fails on leftovers from an earlier deployment: A NemoClaw build can fail after the stack has already been deployed when a gateway, dashboard relay, or sandbox from an earlier deployment is still running — holding 8080, 18789, or 18790, or leaving /tmp/nemoclaw-dashboard-*.log files the relay cannot write. A relay that still answers its health check can also make the notebook report success over a relay that never bound. Workaround: before deploying, confirm that 8080, 18789, and 18790 are free, that no relay or sandbox from an earlier run is alive, and that both /tmp/nemoclaw-dashboard-*.log paths are writable. Unprivileged, lsof and ss may show the listening socket without naming the process that owns it, so treat a held port and an unidentified owner as separate findings: the port blocks the build either way, and an unnamed holder means the owner is unknown, not that it belongs to someone else. Another user’s leftover harness is one possible cause on a shared host, and the one you cannot clear yourself — neither pkill nor nemoclaw destroy reaches their processes, so that user has to tear their own build down and delete the log files. The preflight fix will be available in the next release.
• Live RTSP caption gaps under overload: When RT-VLM inference cannot keep up with incoming video chunks, including at high stream concurrency, the bounded decoder-to-VLM queues drop decoded chunks to limit queued-frame memory growth. Affected chunks may produce empty captions while the live HTTP/SSE request remains active; logs report Live decoder backlog exceeded the bounded decoder-to-VLM transport. Separately, the GPU-memory admission guard may reject new live-stream requests with HTTP 503 ServerBusy when free GPU memory falls below its configured threshold. Reduce concurrent streams or the inference workload to match the platform’s sustainable throughput; increasing queue capacity does not resolve sustained overload.
• Live RTSP backend OOM on Jetson AGX Orin: Prolonged concurrent live captioning with Cosmos Reason 3 Edge has also encountered a backend out-of-memory termination during burst requests, followed by HTTP 503 ServiceUnavailable. Bounded decoder queues do not cap total service memory or guarantee protection against backend OOM. Reduce concurrency and restart the service if the backend has terminated.
• Report generation can intermittently fail after deployment with the Brev Launchable notebook: After deploying VSS 3.3.0 by running deploy_vss_launchable.ipynb, submitting report-generation queries in the VSS Chat UI can intermittently display Network Error instead of completing the report. Regenerating the report with the same query should resolve the issue.

VSS 3.2.1

VSS 3.2.1 is a bug fix release for the VSS 3.2.0 release.

Key Features and Enhancements

• Updated the default VLM model for the Real-Time Video Language Model (RT-VLM) microservice and the agent workflows to Cosmos Reason 3 Nano.
• Updated the Smart City Industry profile to use 3.2.0 microservices.
• Updated CPU-based multimedia support.

Note: Users should use the VSS 3.2.1 GitHub release/branch and the container tags referenced by the 3.2.1 deployment assets. The 3.2.1 manifests may intentionally reference a mix of new 3.2.1 tags and unchanged 3.2 tags; this is expected. Only containers requiring a remediation or feature delta were republished.

Older affected 2.x, 3.0, and 3.1 container images are deprecated and should not be used for new deployments. These older images will be removed from NGC on September 30, 2026.

VSS 3.2.0

These are the VSS 3.2.0 Release Notes. This is the first 3.X General Availability (GA) release. It hardens the modular architecture and documents the developer profiles.

Key Features and Enhancements

• Source code released for all microservices and agent workflows on GitHub.

• Agent Skills (Agent Skills) (Early Access) are now available for agent-driven deployment and operations with support for multiple agent harnesses. Skills are provided for:

  • Profile deployment.
  • Agent workflow use cases such as video summarization, search, and report generation.
  • Deployment and management of RTVI, Alerts, and Behavior Analytics microservices.
  • Camera and stream management.

• Added Helm chart support for all of the agent workflows.

• Added optional audio-in-video support on the base agent profile: remote Nemotron 3 Nano Omni VLM with audio-enabled prompts and configurations.

• Added a Real-time Computer Vision 3D (RT-CV-3D) microservice with support for MV3DT-based 3D multi-camera detection and tracking, and integrated it into a new Warehouse Blueprint application.

• Added the Automatic Calibration microservice for automated camera position and orientation calibration.

• NemoClaw + VSS (Early Access): The deploy_nemoclaw_vss.ipynb notebook for deploying and operating VSS from the NemoClaw / OpenClaw chat UI, bundled with the VSS OpenClaw plugin, VSS Agent Skills, a VSS policy preset, and the host-side VSS Orchestrator MCP server.

Agent Workflows

System Components

Known Issues and Limitations

• The OpenClaw chat UI can display a duplicate assistant response for a single user query. The duplicate is a rendering issue only; the underlying agent processes the query once and the tool invocations and results are unaffected. Refresh the chat or ignore the duplicated bubble.

• The OpenClaw UI can stop rendering agent responses after running for some time and appear stale, for example while periodic VSS deployment status polling continues in the background. Workaround: refresh the UI to see the latest responses.

• Docker Engine 29.5.0 and later can fail to pull some NGC-hosted image tags after the image layers download, with an error from registry: Incorrect Repository Format message. Use a supported Docker Engine version earlier than 29.5.0. If you must use Docker Engine 29.5.0 or later, add or merge the following daemon-side override in /etc/docker/daemon.json and restart Docker:

  {
    "features": {
      "containerd-snapshotter": false
    }
  }

  This disables the containerd snapshotter image store path for the Docker daemon and uses the legacy Docker graphdriver image store. Preserve any existing daemon settings, such as the required exec-opts cgroup driver configuration.

• Agent with audio-enabled configuration for Nemotron 3 Nano Omni model is known to encounter multiple retry issues with videos that are missing audio or non-compatible audio streams.

• After deploying with the Brev Launchable, Brev can occasionally report services unhealthy even though Jupyter Notebook, VSS deployment, and VSS skills continue to function. This appears to be an intermittent Brev status issue and is harmless for VSS operations.

• Phoenix must be accessed through HAProxy on port 7777 (https://7777-<id>.brevlab.com/phoenix), not directly on port 6006. Phoenix is configured for reverse-proxy access with PHOENIX_HOST_ROOT_PATH=/phoenix; HAProxy strips that prefix before forwarding, while direct access on 6006 leaves paths such as /phoenix/graphql unchanged and returns 405 Method Not Allowed, which breaks the UI. Port 6006 does not need a Brev secure link for Phoenix.

• (Search profile, Brev 2-GPU environments): When using local VLM, the Search critic is automatically disabled with a warning to avoid invalid local VLM GPU assignment. As a result, critic-based verification is unavailable unless you use a remote VLM or a host with more than 2 GPUs.

  [WARN] Brev environment has 2 GPU(s). Disabling Search critic to avoid starting the local VLM on GPU 2.

VSS 3.1.0

These are the VSS 3.1.0 Release Notes. This is an early access release including a refactored architecture and new features. There are some features in an alpha state and should not be used in production.

Key Features and Enhancements

• Updated the Search Agent Workflow to introduce attribute search, multi-embedding fusion search, and a critic agent to review search results.
• Updated the Real-Time Computer Vision (RT-CV) microservice to support embedding generation for detected objects. This release supports two embedding models: RADIO-CLIP and SigLIP2.
• Updated Brev Launchable deployment to support the 3.X architecture and deploy all of the agent workflows.
• Added support for AGX Thor and DGX Spark with hybrid deployment (remote LLM) of the Base and Alerts profiles.
• Added additional deployment options for undefined hardware profiles.

VSS 3.0.0

These are the VSS 3.0.0 Release Notes. This is an early access release including a refactored architecture and new features. There are some features in an alpha state and should not be used in production.

Key Features and Enhancements

• Updated the out-of-the-box-experience which includes launching a minimal vision agent and allowing developers to add on agent workflows using a combination of microservices. Agent workflows available in this release are:

  • Report generation and Q&A: The agent can generate templated reports and answer questions using the VLM. This is part of the base agent profile in Quickstart.
  • Video summarization: The agent can generate long video summaries with time-stamped highlights.
  • Alert verification: Augment existing CV pipelines with VLMs to verify events and extract additional insights.
  • Real-Time VLM alerts: Generate tail-end alerts using VLM.
  • Search: Open vocabulary search for actions and events. This is an alpha feature.

• Introduced 2 industry-specific, large scale, blueprint examples for smart cities and warehouses.

• Modularized the VSS architecture, introducing new microservices and APIs.

• Introduced a top-level agent, capable of planning and executing vision-based workflows leveraging the new microservices.

• Introduced Real-Time Video Intelligence (RTVI) microservices for accelerated feature extraction from stored and streamed video. Three microservices are included in this release:

  • Real-Time VLM (RT-VLM): Generates captions and alerts for live streams using Vision Language Models.
  • Real-Time Embedding (RT-Embedding): Generates embeddings for live streams and video files.
  • Real-Time Computer Vision (RT-CV): Detects and tracks objects in live streams and video files.

• Refactored video summarization workflow into the Video Summarization microservice.

• Introduced Video IO and Storage (VIOS) microservices, to manage video (stored and streamed), recording, and playback.

• Introduced Behavior Analytics microservice, to set up heuristics for event creation based on computer vision metadata.

• Introduced calibration microservices, to calibrate the camera position and orientation for 3D and multi-view applications.

• Integrated a new API Gateway / MCP (Model Context Protocol) server to route requests to the appropriate microservices.