> This page is for version Dev. > For other versions, use one of these documentation indexes: > - 3.3.0 (default): https://docs.nvidia.com/vss/3.3.0/llms.txt > - Dev: https://docs.nvidia.com/vss/dev/llms.txt > For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.nvidia.com/vss/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/vss/_mcp/server. # FAQ This section provides categorized frequently asked questions. ## Blueprint & Agent Deployment ### Are VIOS and VST the same service? VIOS (Video IO & Storage) is also referred to as VST (Video Storage Toolkit). The Warehouse documentation uses VIOS for the service and UI; URLs such as `/vst/` and configuration variables such as `VST_USE_SDRC` retain the VST name. ### How do I resolve a failure when deploying the blueprints and agents? We recommend the following steps to resolve deployment failures - * Make sure a previous deployment isn't already running using `docker ps`. If a previous deployment exists, then stop the deployment and run the app again. * A failure can happen if the previous deployment was not stopped and a re-deployment was performed. * Make sure the correct values are set in `$VSS_APPS_DIR/industry-profiles/warehouse-operations/.env` and `overrides.env`. * Make sure the deployment user can read and write the directory configured by `VSS_DATA_DIR`. * If you intentionally need a clean redeployment, follow [Teardown the Deployment](/vision-agent/agent-workflows/industry-specific-examples/warehouse-operations-blueprint/quickstart#teardown-the-deployment) to remove the deployment volumes and data. * Use `docker compose ps` to identify unhealthy services, then inspect a service with `docker compose logs `. We recommend starting with the DeepStream and Kafka services. * A port being occupied is a common cause for deployment failure. ### Why are my configuration changes not reflected after redeployment? The Blueprint Configurator reapplies only the settings it manages during deployment. For a managed setting, change its source value in the Warehouse `.env`, `overrides.env`, or `blueprint_config.yml` file. For an unmanaged setting, edit the target configuration file directly after cleanup. For a clean redeployment, [stop the deployment](/vision-agent/agent-workflows/industry-specific-examples/warehouse-operations-blueprint/quickstart#teardown-the-deployment), run cleanup from `$VSS_APPS_DIR`, and only then edit the configuration and redeploy: ```bash bash scripts/cleanup_all_datalog.sh \ -e industry-profiles/warehouse-operations/overrides.env ``` > **Warning** > > Cleanup restores files from their oldest configurator backups before deleting the backups. Changes made before cleanup can be lost. Cleanup also deletes Warehouse runtime data and generated artifacts. ### Why does vss-elasticsearch-init fail during deployment (race condition with Elasticsearch)? **Observation:** When deploying the blueprint with `docker compose up`, the `elasticsearch-init-container` service (`vss-elasticsearch-init` container) starts and tries to connect to Elasticsearch before it is fully ready. The init container uses all retry attempts (default 20) and exits with code 1, while Elasticsearch becomes healthy soon after. Example log output from the `vss-elasticsearch-init` container: ```text Attempting to connect to the Elasticsearch server for ILM policy creation. Unable to connect to ES. Trying to reconnect - (attempt 1/20) ... Unable to connect to ES. Trying to reconnect - (attempt 20/20) Max attempts to connect to ES reached. Exiting Script. ``` Container status after failure: ```text vss-elasticsearch-init Exited (1) elasticsearch Up (healthy) ``` This can also be caused by network issues on the host. Try increasing `ELASTICSEARCH_CONNECTION_MAX_ATTEMPTS` in `$VSS_APPS_DIR/industry-profiles/warehouse-operations/.env`, then redeploy. ### Why does Kibana show request timeouts or ELK services fail startup health checks? If Kibana shows `Request timeout: "Request timed out"`, inspect the relevant service logs before changing Kibana settings. A common root cause is Elasticsearch disk pressure. In Kibana Discover, this can appear as `Unable to retrieve search results` with `Request timeout: "Request timed out"`. Check Elasticsearch allocation and disk watermarks before tuning Kibana timeouts. See the Elasticsearch disk watermark FAQ below. If Docker marks Elasticsearch, Kibana, or Logstash as `unhealthy` during startup, that is a separate Docker health check timeout. Check the service logs first; if the service is only starting slowly, increase the affected service's `healthcheck.retries` value in `video-search-and-summarization/deploy/docker/services/infra/compose.yml` (or `deploy/docker/services/infra/compose.yml` from the deployment repository root) and redeploy. ### Why is CPU usage high? For Docker Compose, use `docker stats` to identify which services are consuming the most CPU. For Helm deployments, use `kubectl top pods -n `; this command requires the Kubernetes Metrics API. Then review the configuration and logs for the affected services or pods. For example, Kafka-based deployments run four parallel Logstash analytics pipelines (`mdx-kafka`, `mdx-kafka-1`, `mdx-kafka-2`, and `mdx-kafka-3`), in addition to the separate `mdx-lvs` pipeline. This parallelism can increase CPU usage on hosts with fewer CPU cores. For Docker Compose deployments, reduce the analytics pipeline count in `$VSS_APPS_DIR/services/infra/elk/logstash/configs/pipelines-kafka.yml`. Comment out extra `pipeline.id` entries and their corresponding `path.config` lines, keeping at least `mdx-kafka`, then restart Logstash. Fewer pipelines can reduce CPU overhead but also reduce ingestion throughput. Leave `mdx-lvs` unchanged. Redis deployments already use one analytics pipeline. Another example is high CPU usage from PyTorch thread oversubscription when running Sparse4D. See **PyTorch thread oversubscription (Sparse4D)** under [Performance Notes](/system-components/vision-microservices/object-detection-and-tracking#performance-notes) for tuning guidance. ### Why does Elasticsearch stop accepting writes or mark indices read-only when disk usage is high? Elasticsearch uses disk watermarks to protect the ELK stack from running out of disk space. One common symptom is Kibana Discover failing with `Unable to retrieve search results` and `Request timeout: "Request timed out"`. The default watermarks are: * **Low watermark (default `85%` disk used):** Elasticsearch avoids allocating new shards to the node. * **High watermark (default `90%` disk used):** Elasticsearch starts relocating shards away from the node when possible. * **Flood stage (default `95%` disk used):** Elasticsearch sets affected indices to `read_only_allow_delete` so writes stop before the disk fills completely. Check disk allocation and the active watermark settings: ```bash curl --location "http://:9200/_cat/allocation?v" curl --location "http://:9200/_cluster/settings?include_defaults=true&filter_path=**.disk.watermark*" ``` If flood stage is reached, fix the disk pressure first. Common options are: * Remove unnecessary local data or logs from the Elasticsearch host. * Remove unnecessary containers which are not being used. * Add disk capacity or add Elasticsearch capacity so shards have somewhere safe to move. After disk usage drops below the high watermark, Elasticsearch normally removes the block automatically. If an index remains read-only after disk pressure is fixed, clear it manually: ```bash curl --location --request PUT "http://:9200/*/_settings?expand_wildcards=all" \ --header "Content-Type: application/json" \ --data "{\"index.blocks.read_only_allow_delete\": null}" ``` ### Why am I not seeing data on the Kibana/2D Blueprint UI? We recommend the following steps to resolve the issue - * If you are using nvstreamer, verify that the video is present in the mount or that the files were uploaded correctly. * Check if the VST is able to play all the streams. Alternatively, you could also check if the streams are playable using ffplay, VLC or testRTSPClient. * Check the logs of DeepStream to ensure the FPS is showing up correctly for the input streams. * Inspect Kibana to see if the data is being sent. If it's not, check the Logstash logs for any errors. If no errors are found, write a Kafka consumer script to verify if raw data is being sent by DeepStream. * If the UI is loading partially, ensure that you have added all the pre-requisite files, such as the calibration file, images, and image metadata json file. Ensure that the image's name matches with the value present in image metadata json file. * Finally, make sure that the sensor IDs listed in the calibration file matches with the ones mentioned in DeepStream configuration. Also ensure that the sensorIDs of the streams that get added directly via VST matches with the ones mentioned in calibration file. ### My videos produce incorrect timestamps for messages or are processed over an extended duration * For DeepStream to properly work with NVStreamer the input videos need to be encoded to not contain any b frames. To check if the b-frames are present in a video, use either of the following commands: ```bash mediainfo | grep bframes ffprobe -show_frames | grep -E 'pict_type=B' ``` * You can encode your videos to this format by running ```bash ffmpeg -i input_video.mp4 -vcodec libx264 -crf 25 -x264opts "bframes=0:keyint=30" encoded_output_video.mp4 ``` Update the value of the `keyint` with the fps of your video. > **Note** > > The videos can be encoded while uploading via NVStreamer UI. ### I get the message "unauthorized: authentication required" when downloading containers from NGC Follow [How to get an NVIDIA NGC API key](#how-to-get-an-nvidia-ngc-api-key), then sign in to the container registry: ```bash docker login nvcr.io --username '$oauthtoken' ``` Enter your NGC API key when prompted for the password, then retry the download. ### Troubleshooting NVStreamer/VST Videos in Docker-Compose * Verify that the NVStreamer/VST is serving the videos by opening the RTSP streams in either `vlc` or `ffplay`. If using VLC click on “Media” and “Open Network Stream” and enter the following RTSP URL - `rtsp://