> This page is for version 3.3.0 (default).
> For other versions, use one of these documentation indexes:
> - 3.3.0 (default): https://docs.nvidia.com/vss/3.3.0/llms.txt
> - Dev: https://docs.nvidia.com/vss/dev/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/vss/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/vss/_mcp/server.

# Model customization overview

The Video Search and Summarization (VSS) blueprint uses **embedding models** in two places: **object-level** embeddings from the Real-Time Computer Vision (RT-CV) microservice for detection crops and text queries, and **video-level** embeddings from the Real-Time Embedding microservice for clip- and stream-based semantic search. Recent TAO Toolkit releases support fine-tuning the same foundation models that these services use, so you can specialize embeddings for your domain and swap the artifacts into your deployment.

Industry blueprint deployments (for example [Warehouse Operations](/vision-agent/agent-workflows/industry-specific-examples/warehouse-operations-blueprint)) also ship **computer vision** perception models (Sparse4D, RT-DETR) that you can fine-tune with TAO and integrate into the Perception microservice. See [Sparse4D](/system-components/models/sparse4d-3d-multi-camera) and [RT-DETR](/system-components/models/rt-detr-2d-warehouse-detection).

This section maps **which components accept customized weights**, **where to fine-tune** (documentation and NGC model cards), and **how to integrate** fine-tuned models. For low-level service settings (environment variables, DeepStream config keys, APIs), see the linked microservice guides.

> **Note**
>
> Changing embedding dimension or similarity geometry usually requires **re-indexing** stored vectors (for example Elasticsearch indices used by search workflows) and revisiting **similarity thresholds** in analytics. Plan a full pipeline validation after swapping models.

## Summary

| VSS component                                                                                                  | Embedding role                                                                      | Supported model families                                                                   | Primary integration surface                                                                                                                                                                                                       |
| -------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [Object Detection and Tracking](/system-components/vision-microservices/object-detection-and-tracking) (RT-CV) | Object crop embeddings and optional text embeddings (ReID, text-to-image alignment) | [SigLIP 2](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/tao/models/siglip_v2)          | DeepStream INI: vision encoder + text embedder (ONNX / TensorRT); see [ReID and Embeddings](/system-components/vision-microservices/object-detection-and-tracking#reid-and-embeddings-rest-api-and-config-reference) on that page |
| [Real-Time Embedding](/system-components/vision-microservices/video-embedding-generation) (RT-Embedding)       | Video and text embeddings for semantic search and Kafka-published clip features     | [Cosmos-Embed1](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/tao/models/cosmos-embed1) | Container environment: `MODEL_PATH`, optional `MODEL_IMPLEMENTATION_PATH` / Triton repo scripts; see [Customizations](/system-components/vision-microservices/video-embedding-generation#customization) on that page              |

## Per-model guides

* [SigLIP 2 object embeddings](/system-components/models/siglip-2-object-embeddings) — **SigLIP 2** for RT-CV
* [Cosmos-Embed1](/system-components/models/cosmos-embed1-video-embedding) — Cosmos-Embed1 for RT-Embedding
* [Sparse4D](/system-components/models/sparse4d-3d-multi-camera) — TAO Sparse4D for multi-camera 3D detection and tracking (warehouse blueprint)
* [RT-DETR](/system-components/models/rt-detr-2d-warehouse-detection) — TAO RT-DETR for 2D warehouse detection (warehouse blueprint)

## Agent and search workflows

Developer profiles and agent tools that call **embedding endpoints** or query **Elasticsearch** (see [Agent Profiles](/vision-agent/vss-agents/agent-profiles)) assume compatible embedding spaces with the indexed documents. After swapping RT-CV or RT-Embedding models, confirm:

* API payloads still match expected vector sizes.
* Search and fusion logic (for example multi-embedding or attribute search) still uses comparable similarity scales.

For LLM and VLM configuration (not embedding fine-tuning), see [Configure the LLM](/vision-agent/vss-agents/agent-customization/customize-models/configure-the-llm) and [Configure the VLM](/vision-agent/vss-agents/agent-customization/customize-models/configure-the-vlm).

## Related documentation

* [Object Detection and Tracking](/system-components/vision-microservices/object-detection-and-tracking) — RT-CV pipelines, ReID, ONNX/TRT embedding configuration
* [Real-Time Embedding](/system-components/vision-microservices/video-embedding-generation) — RT-Embedding deployment, `MODEL_PATH`, Kafka output
* [Search](/vision-agent/agent-workflows/search) — Search workflow architecture (embeddings → message bus → Elasticsearch)
* [NVIDIA TAO Toolkit](https://docs.nvidia.com/tao/tao-toolkit/latest/index.html) — Training, export, and deployment guides