> This page is for version Dev. > For other versions, use one of these documentation indexes: > - 3.3.0 (default): https://docs.nvidia.com/vss/3.3.0/llms.txt > - Dev: https://docs.nvidia.com/vss/dev/llms.txt > For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.nvidia.com/vss/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/vss/_mcp/server. # Getting Started This guide explains how to deploy and access the **Auto Calibration microservice and user interface**. The detailed Auto Calibration workflow, input requirements, rectification, alignment, results, and troubleshooting content is documented in the [Auto Calibration Workflow Steps](/system-components/calibration/auto-calibration/workflow-steps), [Custom Dataset](/system-components/calibration/auto-calibration/custom-dataset), and [Troubleshooting](/system-components/calibration/auto-calibration/troubleshooting) pages. ## Prerequisites Before using the VSS Auto Calibration service and UI, ensure you have: * x86_64 system * Ubuntu 24.04 * NVIDIA GPU with hardware encoder (NVENC) * NVIDIA Driver 590 or newer * Docker configured to run without `sudo` * NVIDIA Container Toolkit * An NGC API key with access to the required VSS images Required calibration inputs and detailed method-specific requirements are documented in [Custom Dataset](/system-components/calibration/auto-calibration/custom-dataset). ## Deployment Steps (Docker Compose) These steps deploy the Auto Calibration service using the VSS repository. They are separate from running Auto Calibration as a Warehouse Blueprint profile. To use a coding agent instead, see the [Agent Skills walkthrough](#agent-skills-walkthrough). > **Note** > > **Running Auto Calibration as a Warehouse Blueprint profile** > > Auto Calibration can also be deployed as part of the Warehouse Blueprint with `MODE=auto-calibration` and `BP_PROFILE=bp_wh_auto_calib`. That profile starts the same `vss-auto-calibration` service and UI from the blueprint's Compose stack, alongside the VIOS/VST services, rather than through the steps below. The blueprint profile deploys no `sdr-controller`, so the SDRC routing keys `VST_USE_SDRC`, `STREAM_PROCESSOR_MODULE_ENDPOINT`, and `VST_NGINX_MODE` must be commented out so the direct VST defaults in `services/vios/vst.env` apply. `blueprint-deploy.sh` comments them for you in the generated env file; with the direct `docker compose` command you must comment them yourself in `industry-profiles/warehouse-operations/overrides.env`. For the exact settings, see [Configure Environment settings](/vision-agent/agent-workflows/industry-specific-examples/warehouse-operations-blueprint/quickstart#configure-environment-settings) in the Warehouse Quickstart. ### 1. Authenticate with NGC ```bash export NGC_CLI_API_KEY= export NGC_CLI_ORG='nvidia' printf '%s' "${NGC_CLI_API_KEY}" | docker login nvcr.io -u '$oauthtoken' --password-stdin ``` Generate an API key from [NGC Setup](https://org.ngc.nvidia.com/setup) if needed. Do not print or commit the key. ### 2. Clone the VSS repository ```bash git clone https://github.com/NVIDIA-AI-Blueprints/video-search-and-summarization.git cd video-search-and-summarization git checkout "develop" ``` ### 3. Open the VSS Auto Calibration deployment directory ```bash cd deploy/docker/services/auto-calibration ``` The directory contains: ```text compose.yml ms/compose.yml ui/compose.yml ``` ### 4. Configure the VSS environment Create `.env` in the current directory: ```dotenv VSS_AUTO_CALIBRATION_HOST_PORT=8010 VSS_AUTO_CALIBRATION_UI_HOST_PORT=5000 VSS_APPS_DIR=/path/to/video-search-and-summarization/deploy/docker VSS_DATA_DIR=/path/to/your/data_dir HOST_IP= # Optional, uncomment and update this to support RTSP Stream input # VIOS_BASE_URL=http://:30888 ``` Replace the paths and host address with your values. `HOST_IP` must be reachable from the browser. This Compose deployment does not start VIOS. To use RTSP capture, deploy VIOS separately and set `VIOS_BASE_URL` to an endpoint reachable from the Auto Calibration container. Otherwise, use file upload. ### 5. Optional VGGT model setup The Auto Calibration service can run the AMC method without VGGT support. To enable VGGT calibration, download the licensed [VGGT model](https://huggingface.co/facebook/VGGT-1B-Commercial) and place it in `${VSS_DATA_DIR}/auto-calib/vggt/`: ```bash mkdir -p "${VSS_DATA_DIR}/auto-calib/vggt" ``` For the default case, use `vggt_1B_commercial.pt`. To use any other model name, update `VGGT_MODEL_PATH` in `.env` to the new model's full path inside the container: `/tmp/vggt_model/`. You must accept the model license before downloading it. ### 6. Prepare the projects directory ```bash mkdir -p "${VSS_APPS_DIR}/services/auto-calibration/projects" if ! sudo -n true 2>/dev/null; then echo "sudo requires authentication; enter your password when prompted." sudo -v || { echo "sudo authentication failed. Verify sudo access, then retry." >&2; exit 1; } fi sudo chown -R 1000:1000 "${VSS_DATA_DIR}/auto-calib" sudo chown -R 1000:1000 "${VSS_APPS_DIR}/services/auto-calibration/projects" ``` The Auto Calibration service stores project data in the projects directory and the containers use UID/GID `1000`. ### 7. Start the VSS Auto Calibration service and UI ```bash docker compose \ --profile "vss-auto-calibration" \ --profile "vss-auto-calibration-ui" \ up -d ``` Wait for the service to be ready. The microservice will be available at: ```text http://: ``` The UI is available at: ```text http://: ``` ### 8. Stop the service ```bash docker compose \ --profile "vss-auto-calibration" \ --profile "vss-auto-calibration-ui" \ down ``` ## Agent Skills Walkthrough Use the `vss-generate-video-calibration` skill for the workflows below. Each includes example prompts and what the agent does in response. Follow [Installing Skills](/vision-agent/agent-skills#installing-skills) first; see [Agent Harnesses](/vision-agent/agent-harnesses) for how supported agents load them. The agent may ask for deployment paths, host and ports, video paths or RTSP URLs, a layout image, and the measured layout scale. Settings files, existing alignment files, ground truth, and focal-length overrides are optional. Missing scale or alignment can be supplied through the UI. Enter NGC and Hugging Face credentials securely in the terminal, never in chat. For normal calibration, the agent confirms the detector and plan; explicit tuning compares supported detectors automatically. ### Deploy Auto Calibration **Example prompts:** * `Start the Auto Calibration service.` * `Set up AMC with VGGT so I can calibrate videos.` **What the agent does:** 1. Checks platform requirements, Docker/GPU access, repository and data paths, host address, and service ports. It reuses a healthy stack or guides setup when needed. 2. Verifies NGC image access and configures the deployment environment and writable projects directory, asking for any required host-permission approval. 3. When VGGT is requested, checks the Hugging Face CLI and model availability, guides license acceptance and secure authentication, and stages the model before starting calibration. 4. Starts the backend and UI, verifies service readiness, and returns their URLs. A deployment-only request does not create projects or run calibration; a combined setup-and-calibration request continues to the requested input workflow. ### Run the bundled sample dataset **Example prompts:** * `Calibrate the sample dataset.` * `Verify the AMC install with the bundled sample dataset.` **What the agent does:** 1. Checks the host and deploys AMC if it is not already running. 2. Obtains the bundled sample and uploads its ordered videos, layout, alignment, and ground truth. 3. Stages the sample's already-linear media and verifies the project. It runs VGGT first when ready, then AMC with the sample's default `resnet` detector, polling each independently and post-processing each completed multi-camera method. 4. Reports progress, evaluation metrics, and any failures, then returns the UI URL and available output paths for review. ### Calibrate local video files **Example prompts:** * `Calibrate local video files at .` * `Calibrate videos at using the transformer detector.` * `Calibrate videos at and run VGGT too.` **What the agent does:** 1. Checks the host and reuses or deploys AMC, then discovers synchronized MP4s in overlapping-FOV order and collects the layout, scale, alignment, and optional settings, ground truth, and focal lengths. 2. Uploads the inputs and guides any missing UI actions: layout in Step 2, measured scale in Step 3, completed media preparation in Step 4, and interactive alignment in Step 5. Already-linear videos are staged directly; distorted videos require reviewed rectification before alignment and calibration. 3. Resolves the detector (`resnet` or `transformer`) and optional settings, confirms the single-run plan, and verifies the project. 4. Runs VGGT first when available for multi-camera input, then AMC, post-processing each completed method before another calibration starts. Single-camera AMC uses its single-view output path. It reports progress and errors using each method's state and logs. 5. Returns results and output paths, then offers bounded tuning after a completed result or a potentially tunable failure. Missing inputs or infrastructure problems must be resolved before tuning. ### Tune a custom dataset **Example prompts:** * `Calibrate and tune the custom dataset at .` * `Tune the calibration for project .` **What the agent does:** 1. Reuses a healthy AMC service and UI or sets them up first, prepares VGGT for eligible multi-camera input, and collects the dataset inputs. If layout scale or alignment is missing, it guides you through the appropriate UI steps, including media preparation before interactive alignment. 2. Starts from the current service defaults, compares supported detectors automatically, and runs no more than five AMC configuration attempts without further approval. A settings file is optional. Videos, camera order, scale, alignment, and media preparation remain unchanged across attempts. 3. Runs VGGT and AMC sequentially in each eligible multi-camera attempt, post-processes each completed method, and preserves their results separately. VGGT does not consume an AMC attempt. Logs and metrics guide parameter adjustments while the best completed AMC result is retained. 4. Reports every attempt and relevant trade-offs. Without ground truth, the recommendation is heuristic and requires visual review. A completed normal project can be reused as attempt 1 only when its inputs and effective baseline configuration are verified. The agent explains the runtime before starting: up to five AMC attempts with VGGT and post-processing can take several hours. Settings are service-wide, so tuning requires exclusive backend use without competing configuration changes or workloads. Projects are retained for review; cleanup requires approval after their artifacts are preserved outside the server project directory. ### Calibrate live RTSP streams **Example prompts:** * `Calibrate RTSP streams , , ...` * `Calibrate these live RTSP streams using the transformer detector.` **What the agent does:** 1. Checks the host and AMC service, confirms VIOS availability, and collects source RTSP URLs, capture duration, camera order, layout, scale, and optional calibration files. 2. Captures all cameras together in one synchronized session, then waits for automatic ingest to finish. It does not mix file uploads with active RTSP capture or record camera subsets separately. 3. Follows the same media preparation, alignment, detector confirmation, calibration, and post-processing steps as [local video calibration](#calibrate-local-video-files). 4. Reports progress, results, and an optional tuning offer. If tuning is requested, it uses the same captured clips across attempts rather than recapturing each time. ### Review results and exports **Example prompts:** * `Show me the calibration results for project warehouse_calibration.` * `Export calibration.json and the MV3DT ZIP for this project.` **What the agent does:** 1. Checks AMC, VGGT, and post-processing states separately and retrieves available logs, camera parameters, visualizations, and evaluation metrics. GT-based metrics are available only when ground truth was supplied. 2. Returns the project UI URL and available artifact paths, identifies missing results, and compares completed methods without treating one method's failure as failure of both. 3. Explains that VGGT visualizations are separate from AMC overlays. A VGGT-only result may lack trajectory overlays because those require AMC tracklet artifacts; available virtual-object visualizations, camera parameters, and exports can still be reviewed. 4. Exports the requested method's calibration JSON or MV3DT ZIP when available. These describe rectified camera geometry; downstream inputs must use matching rectified media or distortion remapping. ### Stop the service **Example prompts:** * `Stop the Auto Calibration service.` * `Tear down AMC but keep my calibration projects.` **What the agent does:** 1. Identifies the deployed AMC backend and UI and confirms any interruption of active calibration work before stopping them. 2. Preserves project data, downloaded models, and other host-mounted state unless removal was explicitly requested. 3. Requests confirmation before deleting any project or model data; stopping the service alone does not authorize cleanup. ## Next Steps * Follow the [Auto Calibration Workflow Steps](/system-components/calibration/auto-calibration/workflow-steps) for the complete 7-step UI workflow and screenshots. * Follow [Custom Dataset](/system-components/calibration/auto-calibration/custom-dataset) for input videos, alignment data, lens distortion, and ground-truth requirements. * Follow [Troubleshooting](/system-components/calibration/auto-calibration/troubleshooting) for common deployment and calibration failures.