Sim2Real Data Transfer

This page is for version 3.3.0 (default). For other versions, use one of these documentation indexes: - 3.3.0 (default): https://docs.nvidia.com/vss/3.3.0/llms.txt - Dev: https://docs.nvidia.com/vss/dev/llms.txt - Archives: https://docs.nvidia.com/vss/archives/llms.txt

Sim2Real data transfer bridges the domain gap between synthetic multi-view videos generated in NVIDIA Omniverse and real-world warehouse imagery. By converting rendered multi-view videos into realistic variants, developers can validate perception models earlier, improve robustness, and reduce the cost of collecting or rerendering large real-world datasets.

Developers can plug NVIDIA Cosmos Transfer 1 into their simulation and synthetic data generation workflows to transform synthetic multi-view warehouse videos into photorealistic outputs while preserving scene structure. Apply consistent prompts and controls (for example, depth and edge) across camera views to maintain spatial and temporal coherence. This reduces the sim-to-real domain gap, improves downstream detection and tracking performance, and avoids expensive rerendering when exploring lighting, texture, or ambient variations. It also scales data diversity (fog, low light, texture changes) without altering the underlying scene layout. For a step-by-step recipe that processes multi-view warehouse videos with Cosmos Transfer 1, refer to the Cosmos Cookbook guide: Cosmos Transfer 1 Sim2Real for Multi-View Warehouse Detection and Tracking. The recipe demonstrates the end-to-end data augmentation workflow; for the transfer model itself, we recommend using the more advanced Cosmos Transfer 2.5, which has been validated for this warehouse workflow. To get started, see the Cosmos Transfer 2.5 inference guide. Note that the Cosmos Transfer 2.5 repository is no longer under active development and receives only limited maintenance updates; its successor is Cosmos 3, whose Generator models (Cosmos3-Nano and Cosmos3-Super) support video-to-video transfer with edge, blur, depth, and segmentation controls. See the Cosmos 3 Generator transfer examples.