> This page is for version 1.4.0 (Latest) (default).
> For other versions, use one of these documentation indexes:
> - 1.4.0 (Latest) (default): https://docs.nvidia.com/halos/outside-in-safety/1.4.0/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/halos/outside-in-safety/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/halos/outside-in-safety/_mcp/server.

# SIL — 2D Perception

Deploy Halos SIL for Software-in-Loop testing.

> **Note**
>
> Halos SIL requires VSS Warehouse 3.3 as its perception backend. Deploy VSS Warehouse first, as in [Deploy AI Perception](/halos/outside-in-safety/getting-started/inference-pipeline), with the SIL overrides of Step 2, before proceeding with Halos SIL deployment.

## Prerequisites

See [Prerequisites](/halos/outside-in-safety/getting-started/prerequisites) for hardware and software requirements.

## Deploy with Halos Deployment Skill (recommended)

For an automated bring-up with a coding agent (Claude Code, Codex, Cursor, and similar), use the `hoisa-deploy-profile` skill — it replaces Steps 1–6 below with a single agent prompt. See [Agentic Deployment](/halos/outside-in-safety/getting-started/agentic-deployment).

Manual instructions below remain supported and are required if you are not using a coding agent.

## Step 1: Download Packages

Clone the Halos Outside-In Safety repository (ships the compose files, profiles, configs, and scripts in-tree), then download the SIL data tarball from NGC:

```bash
# Clone the Halos Outside-In Safety repo
git clone https://github.com/NVIDIA/halos-outside-in-safety.git -b v1.4.0
cd halos-outside-in-safety

# Download SIL data
ngc registry resource download-version \
  "nvidia/halos-outside-in/sample-sil-data:v1.4.0"

tar -xvf sample-sil-data_v*/halos-outside-in-sil-data.tar.gz \
  --directory=/home/$USER/ # change me
```

**Output**:

* `halos-outside-in-safety/` - Cloned repo (compose files, profiles, configs, scripts)
* `sil-data/` - Isaac Sim scenes and configs

## Step 2: Deploy VSS Warehouse

Deploy VSS Warehouse 3.3.0, the GitHub release `v3.3.0` of the [VSS repository](https://github.com/NVIDIA-AI-Blueprints/video-search-and-summarization), as in [Deploy AI Perception](/halos/outside-in-safety/getting-started/inference-pipeline), or with the VSS repository's `vss-build-vision-ai` skill (warehouse profile). Apply the overrides below **before** `docker compose up` — if perception builds its TensorRT engine with the wrong config, it has to build it again.

Before the first `up`:

* **Log Docker in to `nvcr.io`** with your NGC API key. The release images, `nvcr.io/nvidia/vss-core/*:3.3.0`, do not pull anonymously; without the login a pull stops at `unauthorized: authentication required`.

  ```bash
  echo "$NGC_CLI_API_KEY" | docker login nvcr.io --username '$oauthtoken' --password-stdin
  ```

* **Download the app data**, `nvidia/vss-warehouse/vss-warehouse-app-data:3.3.0`, one 2.1 GB archive. Without `--org nvidia --team vss-warehouse` the download stops at `Missing org - If Authenticated, org is also required.`

  ```bash
  ngc registry resource download-version \
    "nvidia/vss-warehouse/vss-warehouse-app-data:3.3.0" --org nvidia --team vss-warehouse
  tar -xzf vss-warehouse-app-data_v3.3.0/vss-warehouse-app-data.tar.gz \
    --directory=/path/to # change me
  ```

  `VSS_DATA_DIR` is the extracted `vss-warehouse-app-data/` directory. It carries no models: perception downloads them into `<vss_data>/models/` at its first start.

* **Prepare the data directories** on a fresh extract. The containers run as different users, so their data directories must exist and be writable by all of them; otherwise Redis restarts in a loop on `Can't open the log file: Permission denied`. This is the data-directory step of the `vss-build-vision-ai` skill:

  ```bash
  DATA=/path/to/vss-warehouse-app-data   # VSS_DATA_DIR
  for d in analytics_cache calibration_toolkit elastic/data elastic/logs kafka redis/data redis/log \
           vss_video_analytics_api vst/clip_storage nvstreamer/vst_data; do
    mkdir -p "$DATA/data_log/$d"
  done
  mkdir -p "$DATA/agent_eval/dataset" "$DATA/agent_eval/results" "$DATA/models"
  chmod -R a+rwx "$DATA/data_log" "$DATA/agent_eval" "$DATA/models" "$DATA/videos"
  ```

> **Note**
>
> `<wh_ops>` below is the VSS warehouse-operations directory, `<vss_repo>/deploy/docker/industry-profiles/warehouse-operations/`. Edit the override files there in place, but **do not run** `docker compose` **from** `<wh_ops>/` — its `compose.yml` is only an app fragment and pulls in no infrastructure (Kafka, Redis, `sdr-controller`). Deploy from `<vss_repo>/deploy/docker/` with `containers.env`, `industry-profiles/warehouse-operations/.env` and `industry-profiles/warehouse-operations/overrides.env`, in that order.

### 2.1 `overrides.env` Overrides

Set these in `<wh_ops>/overrides.env`, which Docker Compose reads last:

```bash
HOST_IP='<HOST_IP>'                                             # get with: hostname -I | awk '{print $1}'
COMPOSE_PROFILES=${COMPOSE_PROFILES_WH_KAFKA_2D_MINIMAL}        # default: …_WH_KAFKA_2D, which adds Elasticsearch and monitoring
SAMPLE_VIDEO_DATASET="warehouse-loading-dock-3cams-synthetic"   # default: warehouse-4cams-20mx20m-synthetic
NUM_STREAMS=3                                                   # default: 4; matches 3 Isaac Sim cameras
HARDWARE_PROFILE=L40S                                           # default: H100; the perception GPU
LLM_NAME_SLUG=none                                              # default: nemotron-3.5-lightning-30b-a3b; no LLM for SIL
VSS_APPS_DIR="<vss_repo>/deploy/docker"
VSS_DATA_DIR="/path/to/vss-warehouse-app-data"
```

`MODE=2d`, `BP_PROFILE=bp_wh_kafka`, `STREAM_TYPE=kafka`, `LLM_MODE=none` and `VLM_MODE=none`
already ship; keep them. The Safety Core reads its events from Kafka: with `bp_wh`, or a
profile list without Kafka, it receives none while every container still looks healthy.

Export `NGC_CLI_API_KEY` in the shell that runs Docker Compose: perception downloads its
models with it at start. `containers.env` pins the images to `nvcr.io/nvidia/vss-core`, tag
`3.3.0` (`VSS_CONTAINER_TAG`); leave it.

### 2.2 DeepStream Config

File: `<wh_ops>/warehouse-2d-app/deepstream/configs/ds-main-config.txt`

Disable SEI extraction in `[source-list]` (the app config ships it **enabled**):

```text
[source-list]
# extract-sei-type5-data=1
# sei-uuid=NVDS_CUSTOMMETA
```

Set the timing keys in `[streammux]`:

```text
[streammux]
attach-sys-ts-as-ntp=1       # timestamp each frame with host (wall-clock) arrival time
# extract-sei-sim-time=1     # SEI off — do not use Isaac SEI sim-time as the timestamp
# drop-backward-sei=1        # moot with SEI extraction off
```

Isaac Sim 6.1 RTSP *does* embed SEI, but the Safety Core does not consume sim time — using it as the frame timestamp makes the Safety Core drop every event as **STALE**. System timestamps keep decisions flowing. Without these changes the perception pipeline also reports low FPS and bounding boxes flicker on VST.

### 2.3 Perception Model Config

Nothing to change. 2D runs the RT-DETR model that `warehouse-2d-app` downloads into `<vss_data>/models/` at start, and the shipped `warehouse-loading-dock-3cams-synthetic` calibration is used as-is. (3D downloads its Sparse4D model the same way; see [SIL — 3D Perception](/halos/outside-in-safety/testing/sil/sil-3d).)

### 2.4 VST Config

File: `<wh_ops>/warehouse-2d-app/vst/configs/vst_config.json`

> **Warning**
>
> Several copies of `vst_config.json` exist in the deploy tree. Edit the one the VST container actually mounts.

```json
"rtsp_streaming_over_tcp": true,
"use_sensor_ntp_time": false,
"bbox_tolerance_ms": 100
```

* `rtsp_streaming_over_tcp: true` — ingest Isaac's self-hosted RTSP over TCP. UDP ingest
  can stall RTP delivery silently (see [Isaac Sim Streaming](/halos/outside-in-safety/known-limitations#isaac-sim-streaming)).
* `use_sensor_ntp_time: false` — pair with `attach-sys-ts-as-ntp=1` above so VST and
  DeepStream agree on wall-clock arrival time.
* `bbox_tolerance_ms: 100` — widen the metadata-to-frame match window (default `0`) to
  reduce bbox flicker on VST.

This file also carries a `halo_safety_*` block. Leave `halo_safety_udp_port` at its shipped `-1` for the `sil` profile — SIL drives the safety indicator through the comm-layer / ROS 2 path, not the VST overlay. Only the `base` profile sets it to `12345`.

### 2.5 Wait for the TensorRT Engine

Start VSS Warehouse. The first startup downloads the model and builds the TensorRT engine, which takes several minutes. Poll `vss-rtvi-cv` until all 3 cameras report a non-zero FPS:

```bash
docker logs vss-rtvi-cv 2>&1 | grep -E 'Active sources|PERF' -A3 | tail -6
```

> **Warning**
>
> Do not use `docker logs -f` here — it follows the stream and never returns.

Expected output when ready:

```text
Active sources : 3
**PERF:
15.00000 (14.99911)     source_id : 2 stream_name Camera_02
15.00000 (14.99919)     source_id : 1 stream_name Camera_01
15.00000 (15.00077)     source_id : 0 stream_name Camera
```

All 3 sources must report non-zero FPS. A source stuck at `0.00000` isn't arriving — Isaac Sim is not streaming yet, the RTSP URL is wrong, or the engine is still building.

Then confirm the Safety Core event seam has data:

```bash
docker exec kafka kafka-console-consumer \
  --bootstrap-server localhost:9092 \
  --topic mdx-events --max-messages 1 --timeout-ms 30000
```

> **Info**
>
> Do **not** proceed to Halos SIL until both signals fire: 3 cameras at FPS > 0, and `mdx-events` has data.

## Step 3: Configure Halos SIL Environment

```bash
nano deployments/profiles/sil.env
```

Required settings:

```bash
MDX_SAMPLE_APPS_DIR="/path/to/halos-outside-in-safety"  # this repo checkout root
MDX_DATA_DIR="/path/to/sil-data"                        # the extracted sil-data dir from Step 1
HOST_IP='<HOST_IP>'                                      # same value as the VSS overrides.env
ISAAC_GPU_DEVICE=1                                       # an RT-core GPU that does not run perception
ROS_DOMAIN_ID=0                                          # MUST be unique per host (0-232); see Troubleshooting
DOCKER_GID=999                                           # run: getent group docker | cut -d: -f3
```

`PSF_IMAGE` (the PSF 1.4 Safety Core image,
`nvcr.io/nvidia/halos-outside-in/outside-in-safety:nv-psf-halos-1.4-6b92731`) and
`ISAAC_SIM_IMAGE` (`nvcr.io/nvidia/isaac-sim:6.1.0`) are pre-set — leave them. The 1.4
configurations do not run on the 1.3 image.

`PSF_LAUNCH_MODE` (Safety AI Monitor) and `PSF_PCM_MODE` (Perception Monitor) ship `skip`.
To run one `active`, also set its key to `1` in
`closed-loop-testing/safety-core/configs/nvpss.conf` (`required_safety_monitor` or
`required_perception_monitor`). Otherwise the Safety Core stops at start with
`Error: --saim-mode active conflicts with required_safety_monitor = 0 in /opt/nvidia/psf/bin/nvpss.conf.`
See [Required monitors](/halos/outside-in-safety/deployment/custom-deployment#required-monitors).

Two more settings choose the run, and their defaults are the 2D test scenario:

* `SCENARIO=warehouse_20x20_1fl` names the scene, fleet file, cameras config and waypoints
  together (`closed-loop-testing/isaac-sim/sil/configs/scenarios.yaml`); Isaac and every
  controller read it.
* `PSF_APP` (unset = `atl`) picks the decision maker. `pxc` and `both` (proximity) expect the
  3D feed; see [SIL — 3D Perception](/halos/outside-in-safety/testing/sil/sil-3d) and
  [Proximity Monitoring](/halos/outside-in-safety/deployment/applications#proximity-monitoring) in the Deployment Guide.

## Step 4: Deploy Halos SIL

```bash
cd deployments
../closed-loop-testing/scripts/setup.sh sil              # Create required directories and set permissions
../closed-loop-testing/scripts/cleanup_all_datalog.sh sil # Clean previous data logs
docker compose --env-file profiles/sil.env up -d --build
```

Services started: `safety-core`, `comm-layer`, `isaac-sim`, `forklift-controller`.

## Step 5: Run Test Scenario

```bash
docker exec -d isaac-sim bash -lc 'cd /isaac-sim/sil/scripts && \
  ./run_sdg.sh --start --headless --enable-vst'
```

`SCENARIO` from `sil.env` supplies the IRA config, the fleet file and the cameras config.
`-c`, `--robots-config` and `--cameras-config` still win over it for a one-off run.

For script usage details and additional options, see [Execution Script](/halos/outside-in-safety/testing/sil/components/isaac-sim#execution-script).

**What happens**:

1. **Loads** warehouse scene (20×20m)
2. **Spawns** the forklift from the fleet file (`robots.yaml`) and 3 digital humans
3. **Initializes** ROS2 Action Graph for forklift
4. **Runs** forklift playback sequence
5. **Starts** RTSP streaming — Isaac 6.1 self-hosts one RTSP server per camera (`8554/camera`, `8555/camera_01`, `8556/camera_02`). See [RTSP Multi-Camera Graph](/halos/outside-in-safety/testing/sil/components/action-graph#rtsp-multi-camera-graph) for the graph that builds them.
6. **Registers** 3 cameras with VST (`--enable-vst` deletes existing sensors, then adds the Isaac cameras once the render is warm)

**Startup time**:

* **First run**: 10–15 min — Isaac Sim loads its cache and compiles render shaders, then starts the simulation and brings up RTSP streaming.
* **Subsequent runs**: 5–10 min — cache and shaders are already compiled.
* **Simulation**: runs until you stop it. `--start` streams continuously rather than for a fixed frame count.

## Step 6: Monitor Safety Commands

The log paths below use `MDX_DATA_DIR` from your `sil.env`. From the `deployments/` directory, load it into your shell first (re-run it in each terminal you monitor from):

```bash
set -a; source profiles/sil.env; set +a
```

Monitor OPC server log:

```bash
tail -f $MDX_DATA_DIR/comm-layer/opc_server.log
```

Expected output:

```text
INFO:udp_receiver.safety_receiver:Received: Seq#0 | HEARTBEAT | 💓 Heartbeat | ts=2026-04-29T13:33:28.055302+00:00
INFO:udp_receiver.safety_receiver:Received: Seq#3 | MUTE (ALLOW OPERATION) | 🟢 Safety muted - Loading allowed | ts=2026-04-29T13:33:38.946800+00:00
INFO:udp_receiver.safety_receiver:Received: Seq#9 | UNMUTE (PREVENT OPERATION) | 🟡 Safety active + Alarm on | ts=2026-04-29T13:34:06.987138+00:00
```

🟢 **MUTE (ALLOW OPERATION)**: Forklift in trailer, no humans detected (allow loading)

🟡 **UNMUTE (PREVENT OPERATION)**: Human detected or forklift exiting (alarm active)

💓 **HEARTBEAT**: Periodic keep-alive (every 5s) — confirms PSF→comm-layer link is healthy

Monitor PSF:

```bash
tail -f $MDX_DATA_DIR/psf-log/pss.log
```

Expected output:

```text
2026-04-29T13:33:38.770555+00:00 host nv_mdx_client[59]:  Timestamp: 2026-04-29 13:33:38:770421 Endpoint: NVPSB_PSS_SOURCE Data: Safety event reported: EVENT_0 (rule: Forklift tripwire OUT)
2026-04-29T13:34:06.823078+00:00 host nv_mdx_client[59]:  Timestamp: 2026-04-29 13:34:06:822968 Endpoint: NVPSB_PSS_SOURCE Data: Safety event reported: EVENT_1 (rule: Forklift tripwire IN)
2026-04-29T13:34:06.986286+00:00 host NVPSB_PSD_CLIENT[34]:  Timestamp: 2026-04-29 13:34:06:986216 Endpoint: NVPSB_PSD_CLIENT  Data: PSD-Gateway: received DecisionRequest id=1 with 1 events
```

**EVENT_0 / EVENT_1**: Tripwire crossings reported by perception (forklift OUT/IN trailer).

**DecisionRequest**: PSF decision-maker invoked — produces the corresponding MUTE/UNMUTE command on the OPC log above.

## Step 7: View Camera Streams

**VST UI** — `http://<HOST_IP>:30888/vst/`

```bash
# Open in browser
http://<HOST_IP>:30888/vst/
```

View live camera streams from Isaac Sim.

![Camera stream view in VST](/halos/outside-in-safety/_fern-files/_img/5200e9bea2c920c12fd001ac1222c1206b99e2c10886761b93357114cd5636b0.webp)

## Cleanup and Restart

### Restart the Scenario (Live Stack)

`--start` runs the scenario until it is stopped. To relaunch the Isaac scenario on an already-running stack, use the wrapper — it pauses VST stream processing for the boot window (so the fresh RTSP mounts don't hit the cold-start "no caps" race), then resumes it once the streams are live:

```bash
bash closed-loop-testing/scripts/restart_isaac.sh   # reuses the running driver's own args
```

Expect the mounts to warm in \~1–2 min (cached shaders), VST to resume, and DeepStream back at 3/3 cameras \~30 s later.

### Reset to a Fresh Deployment

When a stack misbehaves in ways a restart doesn't fix, return it to the clean first-bring-up state **without** paying the TensorRT/shader rebuild. Stop the scenario and the Halos stack, tear VSS down with `down -v` (it drops the VST sensor database volume; the models and the TensorRT engine live under `VSS_DATA_DIR/models/` and are kept), redeploy VSS, then clear Redis before relaunching:

```bash
docker exec redis redis-cli FLUSHALL
docker restart sdr-controller
docker restart vss-rtvi-cv
```

Keep `VSS_DATA_DIR/models/` and the `isaac-cache/` shader dirs — rebuilding them costs several minutes each.

### Full Cleanup

```bash
# Stop Halos SIL
cd deployments
docker compose --env-file profiles/sil.env down
bash ../closed-loop-testing/scripts/cleanup_all_datalog.sh sil
docker volume prune -f

# Stop VSS Warehouse: see "Stop VSS Warehouse Blueprint" in
# the Deploy AI Perception page (same env files as the deployment)
```

> **Warning**
>
> `docker volume prune -f` removes every dangling volume on the host, not only this
> stack's. With the VSS stack down that includes the VST sensor database. Skip it unless
> you want that reset, or remove the specific volumes instead.

## Troubleshooting

### Safety Core (PSF) Cannot Connect to Kafka

Warehouse (Kafka) must be running before Halos SIL. Check status:

```bash
# Check Warehouse running
docker ps | grep kafka
```

If Kafka is not running, redeploy VSS Warehouse following [Step 2: Deploy VSS Warehouse](#step-2-deploy-vss-warehouse).

### STALE Events in Safety Core (PSF)

First check that DeepStream stamps frames with system time (2.2). `nvpss.conf` ships `timeWindowSize = 6000` (ms) for SIL; raise it only if events still arrive STALE:

```bash
nano closed-loop-testing/safety-core/configs/nvpss.conf
# Increase: timeWindowSize
cd deployments && docker compose --env-file profiles/sil.env restart safety-core
```

### Safety Indicator Flickering (Multi-Machine)

If multiple SIL systems run on the same network, each must use a unique `ROS_DOMAIN_ID` (0-232) in `deployments/profiles/sil.env`. Without this, ROS2 nodes from different machines publish to the same `/safety/is_muted` topic, causing cross-machine interference and safety indicator flickering.

```bash
# In deployments/profiles/sil.env — assign a unique number per machine
ROS_DOMAIN_ID=42
```

Restart Halos SIL after changing. Verify with:

```bash
docker exec comm-layer bash -c \
  "source /opt/ros/jazzy/setup.bash && ros2 topic info /safety/is_muted -v"
```

Should show `Publisher count: 1`.

## Known Issues

Upstream Isaac Sim and VST issues affect the stream path; their status on Isaac Sim 6.1 and the workarounds are in [Isaac Sim Streaming](/halos/outside-in-safety/known-limitations#isaac-sim-streaming). They are not specific to 2D.

## Next Steps

* [SIL Architecture](/halos/outside-in-safety/testing/sil/architecture) — System architecture and data flow
* [Isaac Sim Configuration](/halos/outside-in-safety/testing/sil/components/isaac-sim) — Isaac Sim configuration
* [Isaac Sim Action Graphs](/halos/outside-in-safety/testing/sil/components/action-graph) — Forklift control details
* [Quick Start Guide](/halos/outside-in-safety/testing/srr/quickstart) — Once SIL is healthy, run the **Regression Testing Reporter** to score the safety decision