> This page is for version 1.4.0 (Latest) (default). > For other versions, use one of these documentation indexes: > - 1.4.0 (Latest) (default): https://docs.nvidia.com/halos/outside-in-safety/1.4.0/llms.txt > For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.nvidia.com/halos/outside-in-safety/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/halos/outside-in-safety/_mcp/server. # SIL Architecture System architecture for SIL testing environment. ## System Diagram ![SIL System Architecture](/halos/outside-in-safety/_fern-files/_img/a05c6819020614e677d376fd1e3301d95f131e691b4f88a962383c892f6e5ad7.webp) ## Components ### 1. Isaac Sim Isaac Sim provides the **simulation environment** for SIL testing: 20×20m warehouse with loading dock, trailer, racking, and pallet storage. Materials and lighting match real-world conditions for camera feeds. **Forklift**: Spawned from the fleet file (`robots.yaml` `spawn:`) into a generated layer over the scene. Swivel-wheel kinematics — `back_wheel_drive` (velocity), `back_wheel_swivel` (steering). Safety indicator disc at `/World/forklift_b/body/body/safety_indicator` changes color by safety state (the mute decision, or the proximity level under `PSF_APP=pxc`). Physics handles steering, friction, weight distribution, and collision detection. **Digital humans**: IRA (Isaac Replicator Agent) for autonomous navigation. Path planning, obstacle avoidance, and animation blending. Agent count and behavior configurable in `default_config_ros.yaml`. **Cameras**: 3 cameras (overhead, side, loading dock). 1920×1080 RGB. Isaac Sim 6.1 self-hosts RTSP in-process — one server per camera on ports 8554/8555/8556. With `--enable-vst`, cameras auto-register via `cameras.yaml`. Cameras stream at **15 FPS** in both 2D and 3D mode. 3D additionally requires a unique timestamp per frame for Sparse4D temporal fusion, which the system (wall-clock) timestamps provide. **ROS2**: The control graph subscribes to `/forklift_b/cmd_vel` and the odometry graph publishes `/forklift_b/odom` (names from the fleet file); the indicator graph subscribes to `/safety/is_muted` from the Communication Layer, or to `/forklift_b/proximity/state` from the forklift-controller under `pxc`; `/clock` carries sim time. Uses jazzy, rmw_cyclonedds_cpp, domain ID 0. ### 2. VSS Warehouse Blueprint Perception pipeline consuming camera streams and generating safety events. **2D profile**: DeepStream with RT-DETR model for object detection and NvDCF for per-camera tracking. Publishes events to Kafka `mdx-events` topic. **3D profile**: DeepStream with Sparse4D model for 3D multi-view BEV detection and MTMC (Multi-Target Multi-Camera) tracking. Detections and tracks go to the Kafka `mdx-bev` topic; behavior analytics reads them and publishes the ROI, tripwire and proximity events to `mdx-events`, which is the seam the Safety Core reads in both profiles. See [3D Vision AI Profile](https://docs.nvidia.com/vss/vision-agent/agent-workflows/industry-specific-examples/warehouse-operations-blueprint/development-workflow/deployment-customization/3-d-vision-ai-profile) and [Sparse4D documentation](https://docs.nvidia.com/vss/vision-agent/agent-workflows/industry-specific-examples/warehouse-operations-blueprint/microservice-configuration/perception/3-d-multi-camera-detection-and-tracking-sparse-4-d) for details. The pipeline monitors ROI and Tripwire zones, publishing events to Kafka. Event format: ```json { "event": { "type": "EXIT", "id": "roi-id-1", "info": { "class": "roi" } }, "object": { "type": "Forklift", "id": "125", "confidence": 0.826, "speed": 15.409 }, "sensor": { "id": "Camera_01" }, "timestamp": "2026-01-26T20:27:20.366Z" } ``` Event types: | Event type | Class | Meaning | | ---------- | -------- | ---------------------------------- | | `ENTRY` | roi | Object enters ROI | | `EXIT` | roi | Object exits ROI | | `IN` | tripwire | Object crosses tripwire (entering) | | `OUT` | tripwire | Object crosses tripwire (exiting) | ### 3. Safety Core (PSF) Safety event processing: **Safety Event Integrator** (SEI): * Consumes Kafka events on `mdx-events` for ROI and tripwire alerts and `mdx-frames` for per-frame analysis, in both the 2D and the 3D profile * 2D (`nvpss.conf`, `deploymentMode = 2D`): multi-camera fusion with `timeWindowSize=6000` ms (the shipped SIL value); ATL events bypass fusion * 3D (`nvpss_bev.conf`, `deploymentMode = 3D`): fusion off — perception already fused the views, and events arrive under one BEV sensor, `bev-sensor-1`, declared as a group of the three cameras * Event validation * Forwards validated events to Decision Maker **Monitors**: the Safety AI Monitor (`PSF_LAUNCH_MODE`) and the Perception Monitor (`PSF_PCM_MODE`) both ship `skip`, and both `nvpss` files declare them not required (`required_safety_monitor = 0`, `required_perception_monitor = 0`, `required_mdx_instances = 1`). The container refuses to start when a key does not match its mode; see [Required monitors](/halos/outside-in-safety/deployment/custom-deployment#required-monitors). **Safety Decision Maker** (SDM): `PSF_APP` picks the decision maker; each one reads its event binding from `configs/_sdm.conf`. * `atl` (default) — Automated Trailer Loading: tracks forklift and human presence in the ROI and generates MUTE (0x02) or UNMUTE (0x07), sent as UDP packets (64 bytes) to the Communication Layer on port 12346. With two bays (`atl-dual-override.yaml`) zone 2 sends to `COMM_UDP_PORT_ZONE2` (12347). * `pxc` — Proximity: scores `Forklift` × `Person` separation (`EVENT_12/13/14`) and generates NORMAL (0x07), REDUCE (0x05) or STOP (0x02). * `both` — the two in one container (`--app atl,pxc`); proximity sends to `COMM_UDP_PORT_PXC` (12348). ### 4. Communication Layer **Protocol bridge** between Safety Core and Isaac Sim: * **UDP Receiver** (port 12346, plus 12347 / 12348 when zone 2 / proximity send on their own): Receives Safety Core commands * **OPC UA Server** (port 4840): Industrial protocol interface; zone 2 has its own nodes (`Zone2IsMuted`), and a proximity safe-release request is a write of `True` to `Safety.ProximitySafeReleaseRequest` * **ROS2 Bridge**: Publishes the mute to `/safety/is_muted` (Bool) and to `//safety/is_muted` for each robot in `COMM_ROBOT_IDS`, and proximity decisions to `/safety/proximity/pair` (JSON) and `/safety/proximity/mode`. Proximity never touches `/safety/is_muted`: its opcodes 0x02 and 0x07 mean the opposite of ATL's ROS2 configuration: * `ROS_DISTRO`: jazzy * `RMW_IMPLEMENTATION`: rmw_cyclonedds_cpp * `ROS_DOMAIN_ID`: 0 Logs commands to: `${MDX_DATA_DIR}/comm-layer/opc_server.log` ## Data Flow **2D mode**: ```text Isaac Sim Cameras (3 cams, 15 FPS) ↓ RTSP streams AI Perception (VSS Warehouse Blueprint) ├─ RT-DETR detection ├─ NvDCF tracking └─ ROI/Tripwire analytics ↓ Kafka events (mdx-events topic) Safety Core (PSF) ├─ SEI: Multi-camera fusion + event validation └─ SDM: ATL decision logic (MUTE/UNMUTE) ↓ UDP packet (64 bytes, port 12346) Communication Layer ├─ OPC UA update └─ ROS2 publish (/safety/is_muted) ↓ ROS2 topic Isaac Sim Action Graph └─ Update safety indicator (GREEN/ORANGE) ``` **3D mode**: ```text Isaac Sim Cameras (3 cams, 15 FPS) ↓ RTSP streams (unique timestamps) AI Perception (VSS Warehouse Blueprint) ├─ Sparse4D 3D multi-view BEV detection + tracking └─ BEV ROI/Tripwire/proximity analytics ↓ Kafka events (mdx-events topic, sensor id = bev-sensor-1) Safety Core (PSF, deploymentMode = 3D) ├─ SEI: Event validation + passthrough (fusion done at perception) └─ SDM: ATL (MUTE/UNMUTE) and/or proximity (NORMAL/REDUCE/STOP), per PSF_APP ↓ UDP packets (ATL port 12346; proximity 12348 under both) Communication Layer ├─ OPC UA update └─ ROS2 publish (/safety/is_muted; /safety/proximity/pair) ↓ ROS2 topics Isaac Sim Action Graph + forklift-controller ├─ Update safety indicator (mute, or proximity level) └─ Slow / stop the forklift on REDUCE / STOP (pxc, both) ``` ## Differences from Production SIL runs on x86 with Docker Compose. Key differences from production (IGX Thor): | Component | SIL (x86) | Production (IGX Thor) | | ------------------- | ------------------------------ | ------------------------------------- | | Platform | x86 workstation | IGX Thor (ARM) | | Cameras | Isaac Sim (simulated) | Real IP cameras | | Environment | Virtual warehouse | Physical facility | | Decision Maker OS | Linux (Docker) | FreeRTOS (FSI) | | Event Integrator | Docker on Linux | Docker on CCPLEX | | Communication Layer | Simple UDP + OPC UA (non-safe) | ESL/FSoE/Profisafe (safety-certified) | | Deployment | Docker Compose | Flash binary + Docker Compose | **Shared between SIL and production**: AI Perception (VSS Warehouse Blueprint) pipeline, Safety Core fusion algorithms, ATL decision logic, and event formats — ensuring SIL validation reflects production behavior. ## Next Steps * [Isaac Sim Configuration](/halos/outside-in-safety/testing/sil/components/isaac-sim) - Isaac Sim configuration details * [Isaac Sim Action Graphs](/halos/outside-in-safety/testing/sil/components/action-graph) - Forklift ROS2 Action Graph * [Forklift Controller](/halos/outside-in-safety/testing/sil/components/forklift-controller) - The service that drives the forklift along a route * [SIL — 2D Perception](/halos/outside-in-safety/testing/sil/sil-2d) - Deploy SIL system (2D) * [SIL — 3D Perception](/halos/outside-in-safety/testing/sil/sil-3d) - Deploy SIL system (3D)